Randomization in Matlab Grader

If I'm teaching a class with hundreds of students and I don't want them to each have the same problem, is it possible to have each problem be different on a per-student basis? For example if Matlab grader has access to an ID number which is unique to that student then when programming the question I could use that number when generating the student's question, and reference solution. Something like the following, where the guts of ##xxx## gets evaluated to produce numbers which are unique to the student, and shown to the student:

Problem Description and Instructions:

Assign x=##ID%8## and y=##ID%10##

Calculate x/y and assign the result to z.

Reference Solution:

x = ##ID%8##

y = ##ID%10##

z = x/y

Answer 1

[Alex Pedcenko]

There is a way to achieve some level of individualisation in Grader. Unfortunately it is not possible to generate problem description, which is individual for each student. The workaround is that you can make a list (printed or electronic sheet of sorts), which states parameters of each question for each student.

_________________________

Example:

Question in Grader: Plot a graph of a function y(x) from in your Question 1 (refers to the sheet with values) for N values of x in the the domain of . Values of N, x_min and x_max are individual for each student.

Assign your Student ID number to the variable id.

__________________________

Comment: function y(x), values of N, x_min and x_max are provided to each student individually (e.g. you can make a sheet with their id numbers and list y(x), N, etc for each student id and give the sheet during a test or on Moodle etc. if Grader could generate (or alter) question text based on externally attached script, life could be much easier, but I guess security concerns...

To provide Grader with individual values of y,N.x_min,x_max we can use "Files referenced" section in question specs. We can upload additional m-file or p-file which contains all student id's in an array and other arrays with all other parameters. E.g. I call this file "testit.m" (same as the name of the function inside). The file testit.m is below (not visible to students):

function [ y,n,x_min,x_max ] = testit(student_id)
    % functions and id's are stored in cell arrays

    % can also load all that from some data file
    IDs={'111111','222222','333333'}; % sample student id numbers, 
    % one of id's (e.g. 1st) can be fake for testing purposes

    functions_to_plot={@sin,@cos,@exp_cos}; % functions corresponding to student id numbers 
    % for the "assessment stage", can be built-in or custom made (see example of custom function below)

% other numerical parameters made individual for each student:
    N=[100,200,300]; 
    x_1=[0,-pi,0]; % x_min for plotting
    x_2=[2*pi,pi,3*pi]; % x_max for plotting

% finding which id to use
    id=num2str(student_id);
    index=find(contains(IDs,id)); %searching for id in cell array
    if strcmp(cell2mat(IDs(index)),id) % in case only part of ID was used
        disp(['ID found: ', id]);
    else
        index='';
        disp('ID not found!');
    end
    
	y=functions_to_plot(index); % function for student with given student_id 
	n=N(index); % 
	x_min=x_1(index);  
	x_max=x_2(index); 

end


% example of custom function for a student
function y = exp_cos(x)
	y = exp(x).*cos(x)
end

Then Referense solution function looks like this:

function [y, x, id, h] = myFunction(id)
    
    if ~exist('id','var') % just to pass "ref. solution validation" stage
             id=111111; % should be one of id's in testit.m
    end
       
        [func,n,x_min,x_max] = testit(id);
        x=linspace(x_min,x_max,n);
        % take vector x and evaluate y
        y=func{1}(x); % this is the only way I found as "eval" and alike won't work
        h=plot(x,y);
end

The learner template:

function [y,x,id,h] = myFunction(id) % lock this line
id='222222'; % Note to student: make sure you define your Student ID here (teacher: leave value of id empty)

%% following lines are as an example solution for this forum: they are not shown to student in actual question

x=linspace(-pi,pi,200); % values for id 222222
y=cos(x); % can also try sin(x) [id=111111] or exp(x).*cos(x) [id=333333]
%% 


% assign plot to handle h (optional in case you want to check plot properties, like line style, color etc)
h=plot(x,y); %  

"How to call this function" box

% do not change anything in this box
id=0;
myFunction(id);

Finally the Assessment function

% Run learner solution:
[y,x,id,h] = myFunction();

% Run reference solution and get ref. variables:
[yRef, xRef, idRef, hRef] = reference.myFunction(id); % is is the only place where "id", 
% provided by the student is actually passed to the ref. function to get the correct solution


% can also extract values from "h" and "hRef", e.g. to check if plot parameters are right

% Compare student result with reference solution:
assessVariableEqual('x', xRef,'Feedback','vector x appear to be wrong');
assessVariableEqual('y', yRef,'Feedback','vector y appear to be wrong');

Now, If one is concerned that student will not use their ID and pass solution validation with ID of their mate. ID should not necessarily be real student ID, they may be some unique strings of random characters pre-generated before the test. In this case all you need to change is just one line in textit.m with cell arrays of used id's.

Also after the test you can obviously check if ID they used is ID they should have used, but the main purpose I guess is to avoid extra work :)

Also testit.m can contain all info for all the questions in the assignment, not just one (may need to use unique variables for parameters of each question though)

[Justin Wyss-Gallifent]

Thanks for the details on this. In truth I also considered that this sort of thing would be an option but I hadn't thought it through completely so this helped. Thanks again.

[Justin Wyss-Gallifent]

I'm just adding this for future reference. I wrote a simpler version so I figured I'd share.

The problem description is:

This problem is customized to your University ID Number. White a script which does each of the following in order:

1. Assign uid to your University ID Number.
2. If the last digit of your UID is even, calculate . If it is odd, calculate . Assign the result to a1.
3. Calculate where equals the sum of the digits in your UID. Assign the result to a2.

The reference solution is:

% This just needs to pass.

There are two assessment tests, one is:

[a1ref,a2ref] = testit(uid)
assessVariableEqual('a1',a1ref);

and the other is:

[a1ref,a2ref] = testit(uid)
assessVariableEqual('a2',a2ref)

The file is:

function [a1,a2] = testit(uid)
if mod(uid,2) == 0
    a1 = sin(0.3);
else
    a1 = cos(0.3);
end
syms x;
s = sum(str2num(num2str(uid)'));
a2 = int(sqrt(x),0,s);
end

[Alex Pedcenko]

In this particular example, all this can be realized in the reference solution without a need of any additional ref. files or functions (like testit.m). This is because the answer and all constants are directly [mathematically] mapped to uid and there is no any ambiguity for refrence solution. All what you do in "testit" can be done in usual way in reference solution block of Grader becuse there is small number of possible "cases" and they all can be resolved programmatically like you did.

The disadvantage of the approach you use is that student knows from the question what is your mapping between uid and the outcome, so they can write universal code for any uid.

My approach, which basically uses "lookup table", does not expose the mapping between uid and outcome, and could have been useful if there is, for example ,huge number of possible question variations, where you cant mathematically/programmatically link uid with the outcome of solution in a simple way.

[Justin Wyss-Gallifent]

Oh, absolutely. What I'm trying to do is get away from having an external sheet for each student. If I have an external sheet then testit.m can be set to match that sheet with the answers and make it impossible to write generic code. And I may have to go there, but I'd rather not because then each time I run the course I have to generate those external sheets for each student.

[Alex Pedcenko]

Sure, but external "sheet" can be done automatically as well. Imagine before test, you just feed a list of student IDs to e.g. some matlab function, which e.g. permutes list of UIDs vs problems, then generates "sheet" for each UID. You can even make it completely obscure, and give it as a p-code function, which takes ID of particular student as an input and spits out all the input variables they need (e.g. based on the internal lookup table)....

[Justin Wyss-Gallifent]

Totally, and this is heading back to how I handled the course I'm teaching before I switched to Matlab grader. This issue is that I need to be able to hand the course off to other faculty members and not require them to do anything beyond run the course. As soon as the requirements spill over to external things people start to react negatively. :-)

[Alfonso da Silva Saavedra]

Hi! I found this thread a couple of weeks ago because I wanted something similar to the original question, and, after some approximations using functions, I finally found a solution using scripts.

I published all the code, with some examples, here: https://github.com/alfonsovng/matlab-grader-utils

Answer 2

[Cris LaPierre]

MATLAB Grader allows you to use random numbers to accomplish what you want. Both the reference and learner solutions share the same seed, so generate the same random numbers.

There are two example problems that do this in the Getting Started with MATLAB Grader problem collection.

1. Coordinate transformations Navigating a robot
2. Finding a signal through the noise

Typically you would create the variable for the students in the Learner Template and lock that line of code to ensure they do not accidentally change it. You can use this approach to create a scalar, vector or matrix of values for them. Every time they run the problem, the variable will generate new values.

To answer your other questions, MATLAB Grader does not have access to the student id, and does not have the capability to display a variable value in the problem description.

[Justin Wyss-Gallifent]

This was a bit confusing - first you said that Matlab Grader can use random variables to accomplish what I want, but what you showed is not what I want, and at the end you explicitly state that Matlab Grader does not have the capability to display a variable value in the problem description, which is exactly what I want.

So the short answer is "No".

Are there any plans to implement such functionality? It would be very helpful in preventing cheating, especially in courses with large numbers of students.

[Cris LaPierre]

I must have misunderstood what it was you wanted to achieve. Apologies.

To clarify then, my response shows how to set up a problem in Grader so that every student solves the same problem using different numbers.

[Jeff Alderson]

Justin, can you clarify if having randomly generated inputs is sufficient for randomizing the outputs upon which students are graded? Based on the response given above in your example, you can simply assign your values of X and Y to the outputs of the randi function as Cris illustrated.

Is it critical for the learning objectives that a student is assigning a static value that they obtained in the problem description, and not generated in code, or by a function you provide? What other aspects of a MATLAB Grader problem do you believe would need to be randomized?

When you say that you are trying to prevent cheating, what other countermeasures are you using to detect if cheating has occurred? Have you exported solutions from MATLAB Grader assignment reports and used them in a similarity scoring engine previously?

[Justin Wyss-Gallifent]

No, what I would ideally like is that each student has a slightly different version of the actual question. The issue with randomly generated inputs is that students can just copy and paste each others' code anyway. If the questions are slightly different then even if they're sharing code they at least need to make modifications appropriate to the custom nature of their problem.

This is my first time using Matlab Grader and so currently there are no other countermeasures in place. Exporting solutions and testing for code similarity is useless from a perspective of convincing a university honor committee.

[Justin Wyss-Gallifent]

Addendum - I've been teaching an online Matlab course with 500+ students every semester for several years now. The way we managed this previously was a little more convoluted and not via Matlab Grader. Happy to discuss it outside of a public chat if you're interested. Our hope is that moving to Matlab Grader simplifies the situation for everyone.

[Annie Hui]

I have a similar problem dealing with large classes. Before using Matlab Grader, I would run a program to generate a large random pool of questions. Each question has a unique set of parameters statically embedded within the question itself. I could also generate additional features such as username, course id and embed them within the question. Each question has its own set of numeric answers which I programme into the test pool.

I tried Matlab Grader today, and I'm still trying to find out whether these features could possibly be incorporated into the Grader. Any additional information will be helpful. Thanks in advance.

[Jeff Alderson]

Annie, you may want to reach out to your Customer Success Engineer and describe what you are trying to do. We may be able to hep you use some of the randomization and referenced files features of MATLAB Grader to support this today. We would also value learning from you so that we can incorporate these use cases into a future release.

[Duncan]

i'm just looking over this thread and wonder if any more recent additionf to Matlab Grader that enable individualised questions for large groups. Just want to go with the simplist option (i really like that Grader will now allow students to open in Matlab online).