MATLAB Answers

0

My convolutional network doesn`t learn properly. Can someone explain why and give me some advice?

Asked by Joaquin Carrasco on 4 Jan 2018
Latest activity Edited by Matt J
on 19 Feb 2019
Hi everyone, I`m pretty new at neural network, and I`m trying to make learn my cnn. From a set of about 15000 images i made a training set (70% using random images) and 30% as validation set. The problem is I usually get this kind of graph:
As you can see, the validation star to diverge and it can`t learn anything more. I tried various training options, actually I`m using Max Epochs at 70, validation frequency at 100 (tried with 30 and 50), initial learn rate set at 0.01 with a learn rate drop factor of 0.1; the learn rate drop period is about 15 (tried with different settings; 3, 5, 7, 10....) and my minibatch size is 60. I tried different layer configurations but I`m showing you the actual one. The num_clases value is 120 because it`s the number of different classes I`m trying to classify. Anyone can give me a hint what I`m doing wrong or which changes should I do to make it work properly?
layers = [
imageInputLayer([128 128 3])
convolution2dLayer(5,64,'Padding',1)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
convolution2dLayer(5,128,'Padding',1)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
convolution2dLayer(3,64,'Padding',1)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
convolution2dLayer(3,128,'Padding',1)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
convolution2dLayer(3,256,'Padding',1)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
convolution2dLayer(3,512,'Padding',1)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
convolution2dLayer(1,64,'Padding',1)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
convolution2dLayer(1,128,'Padding',1)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
convolution2dLayer(1,256,'Padding',1)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
convolution2dLayer(1,512,'Padding',1)
batchNormalizationLayer
reluLayer
maxPooling2dLayer(2,'Stride',2)
dropoutLayer
fullyConnectedLayer(2000)
dropoutLayer
fullyConnectedLayer(2000)
fullyConnectedLayer(num_clases)
softmaxLayer
classificationLayer];

  0 Comments

Sign in to comment.

1 Answer

Answer by Don Mathis on 19 Feb 2019

This question is over a year old but I'll post an answer anyway: It's a classic case of overfitting.
Your training accuracy is higher than your validation accuracy because your network is learning details of the training data that don't generalize to the validation data. Similarly, the training loss os lower than the validation loss.

  0 Comments

Sign in to comment.