How do I Specify the Input Size on a Fully Connected Layer?

I'm working on implementing the LeNet5 convolutional neural network for synthesis into VHDL. (A diagram of the network can be found at the top of this page: LeNet5 Diagram) My plan was to use Matlab's Deep Learning Toolbox to train the model, then extract the coefficients for use in the VHDL model. I've trained the model in Matlab successfully, but I've discovered something unusual about the model's layers. This is the declaration of the model's layers, with a comment to indicate where my issue is:

lenet5 = [...
    imageInputLayer([28 28 1], 'Name', 'inputLayer')
    convolution2dLayer(5, 6, 'Name', 'conv1')
    reluLayer('Name', 'relu1')
    maxPooling2dLayer(2, 'Stride', 2, 'Name', 'pool1')
    convolution2dLayer(5, 16, 'Name', 'conv2')
    reluLayer('Name', 'relu2')
    maxPooling2dLayer(2, 'Stride', 2, 'Name', 'pool2')
    % Issue is at this interface
    fullyConnectedLayer(120, 'Name', 'fc1')
    reluLayer('Name', 'relu3')
    fullyConnectedLayer(84, 'Name', 'fc2')
    fullyConnectedLayer(10, 'Name', 'fc3')
    softmaxLayer('Name', 'softmax')
    classificationLayer('Name', 'class')...
];

The problem is that the activation map generated by the pool2 layer has the correct dimensions of 5x5x16, per the diagram linked above.

actPool2Size = size(activations(referenceLenet5, checker, 'pool2'))

actPool2Size =

       5     5    16

If this 3D activation map was stretched into a vector, it would have 400 elements. However, when I examine the fc1 layer, the matrix dimensions are 120x256, shown below.

weightsFC1Size = size(referenceLenet5.Layers(8).Weights)

weightsFC1Size =

     120   256

Since the fc1 layer is just a matrix multiply, these dimensions don't align, making the multiplication impossible, as it would be a matrix multiply with dimensions 120x256 times 400x1. I've read the doc page for the fully connected layer function and I've tried setting the 'InputSize' parameter to 400 to correct this; however, I receive the following error message:

layer = fullyConnectedLayer(120);
layer.InputSize

ans =

      'auto'
  layer.InputSize = 400;
  You cannot set the read-only property 'InputSize' of FullyConnectedLayer.

It appears that the FullyConnectedLayer function isn't correctly computing the input size. How can I manually override this behavior and set the correct input size for the fully connected layer?

Answer 1

[Johannes Bergstrom]

The fully connected layer automatically calculates the input size. The input to 'fc1' in the lenet5 layer array is 4-by-4-by-16. Use analyzeNetwork(lenet5) to see all the layer sizes. Note that your image input size is 28-by-28, while in the LeNet5 Diagram that you link to, it's 32-by-32.

Answer 2

[Dengsheng Zhang]

This is how you got the total number of weights: 120x256:

• At layer 4, the map size is 12x12
• At layer 7, the map size is 4x4=16
• At layer 8, the total weights: 16(map size)x16(depth)x120=120x256