Mini batch size for SeriesNetwork

11 visualizzazioni (ultimi 30 giorni)
Hi!
I have got some issue, it seems that miniBatchSize does not divide my training data into batches, whole matrix of 2e6x15 goes though training per one iteration. I can see in options function that miniBatchSize gets a value, tried everything from 2 to 1M. Thank you for your answers :)
  2 Commenti
Maria Duarte Rosa
Maria Duarte Rosa il 5 Ago 2019
Hi Maxim, can you provide more details on what made you conclude that the whole matrix 2e6x15 goes through training per one iteration?
Maxim Abdullayev
Maxim Abdullayev il 6 Ago 2019
For example in the gui wondow during training process, iterations per epoch is equall to 1, and it really is equal in the graph and so on. I dont have possibility to post screenshot of training process, but here is the code. I have tried differen parameter values, but nothing works.
size = 50000;
layers = [ sequenceInputLayer(15)
% sequenceFoldingLayer
% lstmLayer(2,'OutputMode','sequence')
fullyConnectedLayer(100)
dropoutLayer(0.5)
fullyConnectedLayer(1)
% sequenceUnfoldingLayer('unfold')
regressionLayer];
validationFrequency = floor(numel(trainOUT)/size);
options = trainingOptions('adam', ... % sgdm rmsprop adam
'MaxEpochs',50, ...
'InitialLearnRate',1e-2, ...
'LearnRateSchedule','piecewise', ...
'LearnRateDropFactor',0.1, ...
'LearnRateDropPeriod',20, ...
'Shuffle','never', ...
'MiniBatchSize',size, ...
'ValidationData',{testIN',testOUT'}, ...
'ValidationFrequency',validationFrequency, ...
'ValidationPatience',5,...
'Plots','training-progress', ...
'Verbose',false);
net22 = trainNetwork(trainIN',trainOUT',layers,options);

Accedi per commentare.

Risposta accettata

Maria Duarte Rosa
Maria Duarte Rosa il 6 Ago 2019
Hi Maxim,
Thanks for providing more details.
If your data is in a D x S matrix format (D being 2e6 and S being 15) MATLAB assumes that this is a single observation problem with 15 time-series each being 2e6 points long. For each epoch, we have only 1 iteration and so the mini-batch size option is ignored because it doesn't apply to just 1 observation.
If you'd like to break the time-series into smaller chuncks of data that are treated as different observations, you can do that using the sequenceLength parameter in trainingOptions, by providing a positive integer with the desired sequence length:
Depending on how many smaller sequences this generates then the mini-batch size parameter can be used to control de size of the mini-batches as you would expect.
If you'd like to input more than one observation (each observation being D x S, i.e. D data points and S time-series) then it's better to use N x 1 cell arrays (each cell containing a D x S matrix), where N is the number of observations. Please see here for more information:
I hope this helps.

Più risposte (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by