Update matrix values inside MATLAB Function block

9 views (last 30 days)
Szilard Hunor Toth
Szilard Hunor Toth on 11 Dec 2021
Edited: Andy Bartlett on 13 Dec 2021
Hi all!
I'm having trouble with updating a matrix inside a MATLAB function block. I'm trying to implement a Q-learning algorithm using only basic MATLAB and Simulink for vehicle control. When I try to update my Q-table, which I defined as a parameter in the MATLAB Function workspace, it gives me the following error:
'Expected either a logical, char, int, fi, single, or double. Found an mxArray. MxArrays are returned from calls to the MATLAB interpreter and are not supported inside expressions. They may only be used on the right-hand side of assignments and as arguments to extrinsic functions.'
My outside Simulink initialization code is:
%tuneable parameters
epsilon = 0.9;
epsilonDecayRate = 0.001;
epsilonMin = 0.01;
%sampleTime = 0.1;
gamma = 0.95;
Pedal = 10; %#
Steer = 30; %deg
%discretizing state and action space
discVx = [5 6 7 8 9 10 11 12 13 14 15];
discVy = [-5 -4.5 -4 -3.5 -3 -2.5 -2 -1.5 -1 -0.5 0];
discR = linspace(0,1,11);
discPed = linspace(0,1,Pedal+1);
discSteer = linspace(-200,100,(300/Steer)+1);
%skeleton arrays for search
actInfo = [];
for i=1:(Pedal+1)
for j=1:((300/Steer)+1)
actInfo(end+1,:) = [discPed(i) discSteer(j)];
end
end
numofacts = (Pedal+1)*((300/Steer)+1);
obsInfo = [];
for i=1:length(discVx)
for j=1:length(discVy)
for k=1:length(discR)
obsInfo(end+1,:) = [discVx(i) discVy(j) discR(k)];
end
end
end
numofstats = length(discVx)*length(discVy)*length(discR);
%Q-table
Q = zeros([numofstats numofacts]);
Q = randi([-5000 -3000],[numofstats numofacts]);
And the MATLAB Function is:
function [pedal,steer] = trainMyQtable(v_x, v_y, r, v_x_, v_y_, r_,...
reward, Q, obsInfo, actInfo, epsilon, epsilonDecayRate, epsilonMin,...
discVx, discVy, discR, lastPed, lastSteer, gamma)
%discretize and detect states
[dump,index1] = min(abs(discVx-v_x_));
[dump,index2] = min(abs(discVy-v_y_));
[dump,index3] = min(abs(discR-r_));
laststate = [discVx(index1);discVy(index2);discR(index3)];
[dump,index1] = min(abs(discVx-v_x));
[dump,index2] = min(abs(discVy-v_y));
[dump,index3] = min(abs(discR-r));
newstate = [discVx(index1);discVy(index2);discR(index3)];
lastObsIndx = ismember(obsInfo, laststate);
newObsIndx = ismember(obsInfo, newstate);
%update table value
actIndx = ismember(actInfo, [lastPed lastSteer]);
TD_error = reward + gamma * max(Q(newObsIndx,:)) - Q(lastObsIndx,actIndx);
Q(lastObsIndx,actIndx) = Q(lastObsIndx,actIndx) + alpha*TD_error;
%get action with exploration
choice = rand(1);
if choice > epsilon
pedal = randsample(actInfo(:,1),1);
steer = randsample(actInfo(:,2),1);
else
[~,newActIndx] = max(Q(newObsIndx,:));
[pedal,steer] = actInfo(newActIndx,:);
end
epsilon = max([epsilon*epsilonDecayRate epsilonMin]);
The problem is with the line
Q(lastObsIndx,actIndx) = Q(lastObsIndx,actIndx) + alpha*TD_error;
Can anyone give me a way to get around this issue? Or maybe a more clever way to implement a Q-table? Any insight would be appreciated. Thank you!
  2 Comments
Szilard Hunor Toth
Szilard Hunor Toth on 12 Dec 2021
Thank you for the suggestion! I tried all of these types the error mentions, but it persists. The Simulink says
Expected either a logical, char, int, fi, single, or double. Found an mxArray. MxArrays are returned from calls to the MATLAB interpreter and are not supported inside expressions. They may only be used on the right-hand side of assignments and as arguments to extrinsic functions. Function 'Agent/MATLAB Function' (#33.855.869), line 20, column 51: "alpha*TD_error"
Though in the model manager, the variable is recognized as a fi type, so there shouldn't be a problem.

Sign in to comment.

Answers (1)

Andy Bartlett
Andy Bartlett on 13 Dec 2021
Edited: Andy Bartlett on 13 Dec 2021
Something to be aware of is that MATLAB Function Block primarily uses code generation mode and not interpreted mode.
Interpreted Mode
Let's call running code in "regular" MATLAB usage as "interpreted mode."
In interpreted mode, you can think of all numeric variables as being represented as MxArrays.
MxArray are effectively a structure that contains or points to meta data like type, dimensions, complexity, etc.
The MxArray structure also provides pointers to access stored values that live elsewhere in memory.
As an example let's consider
myVar = uint16([4, 5, 6])
The MxArray representing myVar contains meta data like, type is 'uint8', complexity is real, dimensions 1 by 3, etc.
The MxArray also provides access to the 6 bytes that encode myVar's three values 4, 5, and 6.
Code Gen Mode
When generating code for MATLAB code and/or a Simulink model for simulation or production code generation, the variables will be stripped down to "leanest essence."
Normally, MxArray's are no longer to be found. The outer structure and all the meta data make inefficient in the context of code generation.
When generating C code, the example above, myVar, would appear in code as just the following.
uint16_T myVar[3] = {4U, 5U, 6U}
The meta data will not directly appear in the generated code, but will appear indirectly as needed. For example, myVar was declared as type uint16_T with dimension 3.
Same Meaning But Clearly Different
Representing myVar as an MxArray for interpreted mode as a structure that points to meta data and raw data is clearly different from representing myVar in C code as raw numeric type value. In both cases, myVar represents the same thing and the data values like 3, 4, and 5 should be the same. But clearly, their representations are not directly compatible.
The error from MATLAB Function Blocks is indicating that an attempt is being made to mix these incompatible representations. MATLAB Function Block, like MATLAB Coder, attempts to do everything in code generation mode. Somehow a MxArray is coming into play and causing an incompatibility that code generation wants to be plain C/C++.
Consider Debuging with MATLAB Coder
If you have MATLAB Coder available, it might be helpful to debug the issue using MATLAB Coder.
Put your function trainMyQtable in a standalone M-file.
Setup a call to codegen to define the input arguments.
Call codegen and use the created code gen report to narrow down the source of the issue.
For my own work, I find that approach helpful 90% of the time.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by