A problem with RL toolbox: wrong size of inputs of actor network.

I have a problem with getSize which shows a wrong size, my input is a scalar with a size [1 1], but get size returns 2. I am using RL toolbox and an actor network with three inputs:

in1=[1]; %scalar

in2=[1]; %scalar

in3=image; %of size [12 12 4]

but the getSize returns {[2] [2] [12 12 4]} instead of {[1] [1] [12 12 4]} . I am not sure why?

Answer 1

[Namnendra]

Hello Ali,

The issue you're encountering with the `getSize` function returning an incorrect size for scalar inputs in the Reinforcement Learning (RL) toolbox might be due to how the toolbox interprets scalar inputs. Scalars can sometimes be treated as 2-element vectors in certain contexts, which could explain why `getSize` is returning `[2]` instead of `[1]`.

Here's a way to troubleshoot and potentially resolve the issue:

Step-by-Step Troubleshooting

1. Check Network Input Layers: Ensure that the input layers of your actor network are correctly defined to accept scalar values. Use `featureInputLayer` for scalar inputs and `imageInputLayer` for image inputs.

 in1 = featureInputLayer(1, 'Normalization', 'none', 'Name', 'in1');
    in2 = featureInputLayer(1, 'Normalization', 'none', 'Name', 'in2');
    in3 = imageInputLayer([12 12 4], 'Normalization', 'none', 'Name', 'in3');

2. Verify Network Connections: Make sure the network connections are correctly defined, and the layers are connected properly.

3. Check Input Sizes: Explicitly check the input sizes before passing them to the network to ensure they are correctly formatted.

 % Ensure in1 and in2 are 1x1 scalars
    in1 = reshape(in1, [1 1]);
    in2 = reshape(in2, [1 1]);
    
    % Ensure in3 is [12 12 4]
    in3 = reshape(in3, [12 12 4]);
    
    % Verify sizes
    disp(size(in1));  % Should display [1 1]
    disp(size(in2));  % Should display [1 1]
    disp(size(in3));  % Should display [12 12 4]

4. Test with a Simple Network: Create a simple network to isolate the issue and verify the input sizes.

% Simple network for testing
    lgraph = layerGraph();
    
    % Add input layers
    in1Layer = featureInputLayer(1, 'Normalization', 'none', 'Name', 'in1');
    in2Layer = featureInputLayer(1, 'Normalization', 'none', 'Name', 'in2');
    in3Layer = imageInputLayer([12 12 4], 'Normalization', 'none', 'Name', 'in3');
    
    lgraph = addLayers(lgraph, in1Layer);
    lgraph = addLayers(lgraph, in2Layer);
    lgraph = addLayers(lgraph, in3Layer);
    
    % Create the actor representation
    actor = rlStochasticActorRepresentation(lgraph, {in1Layer, in2Layer, in3Layer}, ...
        {'action'}, actorOptions);
    
    % Check sizes
    sizes = getSize(actor);
    disp(sizes);

Explanation

1. Define Input Layers: We define three input layers: two `featureInputLayer` for scalar inputs and one `imageInputLayer` for the image input.

2. Create Layer Graph: We create a layer graph and add the input layers.

3. Add Fully Connected Layers: We add fully connected layers to process each input.

4. Connect Layers: We connect the input layers to the fully connected layers and then concatenate their outputs.

5. Add Output Layer: We add an output layer to produce the final action.

6. Create Actor Representation: We create the actor representation using the layer graph and specified input layers.

7. Check Sizes: Finally, we use `getSize` to check the sizes of the inputs to ensure they are correct.

By following these steps, you should be able to correctly define and verify the sizes of your inputs and resolve the issue with `getSize` returning incorrect sizes.

Thank you.