Valid indices are restricted in parfor loops

Can anybody help me to understand what is wrong in the following code? Everything works until I add lines 24-27 with the for loop, where I get the "valid indices are restricted in parfor loops" for state and action variables. I have a lot about classification variables in parfor loops but I still haven't understood where I am wrong.

%% SARSA PARFOR TEST
alpha = 0.1;
epsilon = 0.05;
states = 1:1:100;
ls = length(states);
la = length(actions);
actions = [0 1 2];
num_agents = 10;
Q = zeros(ls,la,num_agents);
state = zeros(time_steps,num_agents);
action = zeros(time_steps,num_agents);
equalmax = zeros(time_steps,num_agents);
idx = zeros(time_steps,num_agents);
time_steps = 1000;
parfor k = 1:num_agents
    state(1,k) = states(randi(ls),1);
    if rand < epsilon
        action(1,k) = actions(1,randi(la));
    else
        [maxq(1,k),action(1,k)] = max(Q(state(1,k),:,k));
        equalmax(1,k) = find(Q(state(1,k),:,k) == maxq(1,k));
        if numel(equalmax(1,k)) > 1
            idx(1,k) = randi(numel(equalmax(1,k)));
            action(1,k) = idx(1,k) -1;            
        end
    end
    
    for t = 2:time_steps
     reward(t,k) = state(t-1,k)*action(t-1,k);
     
    end
    
end

Answer 1

[Matt J]

You need to get familiar with Sliced Variables and their restrictions. Basically, if you have an expression like state(expr,k) involving a sliced variable, then expr must remain the same if you index the same variable later in the loop. You violate this where you have state(1,k) and then later state(t-1,k). Moreover, t-1 is not a valid choice for expr. You cannot have complex expressions in the index that doesn't involve the loop variable. It has to be something like state(t,k).

Here is one way to get rid of the errors:

parfor k = 1:num_agents
    
    for t = 1:time_steps-1
    
                if t==1
                   
                    state(t,k) = states(randi(ls),1);
                    if rand < epsilon
                        action(t,k) = actions(1,randi(la)); %#ok<*PFBNS>
                    else
                        [maxq(1,k),action(t,k)] = max(Q(state(t,k),:,k));
                        equalmax(1,k) = find(Q(state(t,k),:,k) == maxq(1,k));
                        if numel(equalmax(1,k)) > 1
                            idx(1,k) = randi(numel(equalmax(1,k)));
                            action(t,k) = idx(1,k) -1;            
                        end
                    end
                    
                end
   
        Reward_tmp(t,k) = state(t,k)*action(t,k);
     
    end
    
end

reward(2:end,:)=Reward_tmp;

[Matt J]

Memo Remo's comment moved here:

Hi,

I am getting the same error. I tried to follow the abovementioned instructions to fix the problem but it seems my problem is somehow different. Please let me know how can I modify the indices of the IDent to make it compatible with the parfor.

Thanks!

%Q_IN is a (ZsplitNo*YsplitNo*XsplitNo) by 3 matrix.

parfor k=1:ZsplitNo*YsplitNo*XsplitNo

            
            X=Q{Q_IN(k,1),Q_IN(k,2),Q_IN(k,3)};
            

            partition1=1;

            [Ident1] = Ex_PartitionScanner(partition1,X);

            
            IDent{Q_IN(k,1),Q_IN(k,2),Q_IN(k,3)}=Ident1;


            p=p+1;

end

[Matt J]

idx=sub2ind(size(Q), Q_IN(:,1),Q_IN(:,2),Q_IN(:,3) );


XCell=Q(idx);
IDentCell=IDent(idx);


parfor k=1:ZsplitNo*YsplitNo*XsplitNo

            
            X=XCell{k};
            

            partition1=1;

            [Ident1] = Ex_PartitionScanner(partition1,X);

            
            IDentCell{k}=Ident1;


            p=p+1;

end


IDent(idx)=IDentCell;

[Memo Remo]

Hi Matt J,

Thank you so much!

Best,