Problem with single agent Simulink using RL toolbox

I am using RL toolbox to train a single agent with the following specifications:

for type=1
    
    


%  obsMat = [1 1];
   obsMat = [4 3; 5 3; 6 3; 7 3; 8 3; 9 3; 5 11;  6 11; 7 11; 8 11; 6 12; 7 12; 3 12; ];

% obsMat = [ ];
sA0 = [2 5];


switch type
    case 1
        s0 = [sA0];
end


Ts = 0.1;
Tf = 100;
maxsteps = ceil(Tf/Ts);
switch type
    case 1
        mdl = "rlAreaCoverage1";
end
open_system(mdl)


% Define observation specifications.
obsSize = [12 12 4];
oinfo = rlNumericSpec(obsSize);
oinfo.Name = "observations";

% Define action specifications.

 numAct=131
switch numAct 
    case 131
        actionSpace = {1	2	3	4 5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27			34	35	36	37	38	39	40	41	42	43	44	45	46	47	48	49	50	51	52	53	54	55	56	57	58	59	60	61	62	63	64	65	66	67	68	69	70	71	72	73	74	75	76	77	78	79	80	81	82	83	84	85	86	87	88	89	90	91	92	93	94	95	96	97	98	99	100	101	102	103	104	105	106	107	108	109	110	111	112	113	114	115	116	117	118	119	120	121	122	123	124		129	130	131	132	133	134		136	137		140	141	142	143	144};

end

ainfo = rlFiniteSetSpec(actionSpace);
ainfo.Name = "actions";
switch type
    case 1
        blks = mdl + ["/Agent A (Red)"];
end

switch type
    case 1
        env = rlSimulinkEnv(mdl,blks,{oinfo},{ainfo});
 end

env.ResetFcn = @(in) resetMap(in, obsMat);
rng(0)

but I see this error:

Error using rlSimulinkEnv>localValidateIOSpecs (line 178)
Invalid size or type for observation specification.

Error in rlSimulinkEnv (line 90)
    localValidateIOSpecs(numAgents,observationInfo,actionInfo);

Do you have any idea what is wrong in my setup?

Answer 1

[Shantanu Dixit]

Hi Ali, it seems that you are facing an issue while training a single agent using RL toolbox in Simulink. Following could be the possible steps to troubleshoot the issue.

• Correct the initialization of simulink environment as

     env = rlSimulinkEnv(mdl,blks,oinfo,ainfo);

• For a single agent block ensure that the block is defined as a string or character vector containing the block path.

     blks = mdl + "rlAreaCoverage1"

• Ensure that the action space defined matches the action space in the model.
• Verify that the defined observation size matches the dimensions expected by the model.
• Verify the output dimensions of the model and ensure that they match with the above specified dimensions.

Additionaly refer to the below MathWorks documentation to learn how to create environment object from a Simulink model

• www.mathworks.com/help/reinforcement-learning/ref/rlsimulinkenv.html