MATLAB Answers

How do I save the training progress images generated in "trainNetwork" when 'Plots' is set to 'training-progress' in the training options?

373 views (last 30 days)
When training my neural net with "trainNetwork", I have passed in training options with the 'Plots' field set to 'training-options'. This causes a plot to be generated regularly throughout the training process, updating the user on mini-batch loss and accuracy, validation data loss and accuracy, and a few other metrics.
Is there any way I can save these images?

Accepted Answer

MathWorks Support Team
MathWorks Support Team on 18 Oct 2019
One way to do this is to make use of the "OutputFcn" field in the training options structure. You can customize a function, and assign it as the value of this field when calling "trainingOptions". See the 'OutputFcn' section in the below MATLAB documentation.
This function will execute once before the training, once after the training, and once after every iteration. It takes as input an information structure with information about the progress of the training. This is useful since you could, for instance, check the "State" field of the info structure to see if the training is over, and then save the final result figure using "savefig".
As a caveat, the figures produced during the training are hidden by default, so the function "findall" may be needed to find the handles that are invisible.
>> findall(groot, 'Type', 'Figure')

  3 Comments

Chee Ho Ng
Chee Ho Ng on 6 Jan 2018
Dear MathWorks Support Team, I'm having this issue also. However, I'm not familiar with the "findall" function. Could you provide a simple example code on "how to save the training progress plot"?
Thank you in advance!
Erica Kiderman
Erica Kiderman on 15 May 2018
Thank you for your feedback. I have updated the answer above to include an example of how to use the "findall" function

Sign in to comment.

More Answers (7)

Peter
Peter on 24 Apr 2018
The following works for me: add the following to the trainingOptions:
'OutputFcn',@(info)savetrainingplot(info),
Add the following function at the end of the script
function stop=savetrainingplot(info)
stop=false; %prevents this function from ending trainNetwork prematurely
if info.State=='done' %check if all iterations have completed
% if true
saveas(gcf,'filename.png') % save figure as .png, you can change this
end
end

  4 Comments

Show 1 older comment
Erica Kiderman
Erica Kiderman on 15 May 2018
The blank figures could be due to the fact that figures produced during training are hidden by default. To access hidden figures, use the "findall" function instead of "gcf". The answer above has been updated to include an example of how to use the "findall" function.
Leo Nunnink
Leo Nunnink on 19 Aug 2018
OK I have a variation on this problem. I am trying to train multiple networks sequentially in a loop. Workflow is: load data, train network, save training plot, save accuracy. All of it works except saving the training plot. I need to be able to create a dynamic file name for each new training plot in each iteration of the loop. The problem is I can't passs the loop index variable to the savetrainingplot function eg: when I modify training options as follows: 'OutputFcn',@(info)savetrainingplot(info,i), and the function as follows: function stop=savetrainingplot(info,loopnumber) It will work perfectly for a few iterations then crash with error message 'Handle input must be scalar, vector, or cell array of vectors.' I've tried making the loop variable global but it still isn't visible to the function so can't be used in generation of dynamic file names for the training plots.
Any advice would be appreciated.
function stop=savetrainingplot(info,loopnumber)
stop=false;
if info.State=='done' %check if all iterations have completed
% if true
AllCodes=readtable('C:\Users\Leo\Documents\ActiveCodes.txt');
CurrentCode=AllCodes{loopnumber,1};
currentfig=findall(groot,'Type','Figure');
filename=strcat('C:\Users\Leo\Documents\MATLAB\Training Plots\',string(CurrentCode),'.jpg');
saveas(currentfig,filename)
end
end
George Tzagkarakis
George Tzagkarakis on 16 Jul 2019
Hello everyone,
I followed the above reccomendations from Peter and Keke Keke Zhang,and my code is as follows:
options = trainingOptions('sgdm', ...
'MiniBatchSize',10, ...
'MaxEpochs',5, ...
'InitialLearnRate',1e-4, ...
'Shuffle','every-epoch', ...
'ValidationData',augimdsValidation, ...
'ValidationFrequency',1, ...
'Verbose',true, ...
'Plots','training-progress',...
'OutputFcn', @(info)savetrainingplot(info));
function stop=savetrainingplot(info)
stop=false; %prevents this function from ending trainNetwork prematurely
if info.State=='done' %check if all iterations have completed
% if true
saveas(findall(groot, 'Type', 'Figure'),'training_process.jpg') % save figure as .png, you can change this
saveas(findall(groot, 'Type', 'Figure'),'training_process.fig')
% savefig(findall(groot, 'Type', 'Figure'),'training_process.fig')
end
end
The problem is that when I save the training-progress-plot as a .fig file(either with saveas or wth savefig,as in the code above),the created plot shows the training and the smoothed training accuracies and losses with exactly the same color/style(whereas the non-smoothed ones should have "fading" effects).Both in the 2 figures,as well as in the legends in the right side...
Any ideas of how to fix it(apart from editing "manually" the plot afterwards)..?
Thanks in advance!
P.S. : Apart from the above issue,whenever I open the saved .fig "training-progress" figure,it cannot be closed be any means-excpet "killing" Matlab from the Task Manager.Tested in Matlab R2018a,R2018b and R2019a...

Sign in to comment.


Aqib Mumtaz
Aqib Mumtaz on 25 Mar 2018
I am also looking for some quick solution to save progress plot instead of plotting first my own then saving?

  0 Comments

Sign in to comment.


Rohan Kapoor
Rohan Kapoor on 15 Oct 2018
Edited: Rohan Kapoor on 15 Oct 2018
Hi,
Is there a way to make a video of the training progress?
Cheers!

  0 Comments

Sign in to comment.


Jifeng Chu
Jifeng Chu on 28 May 2019
I also have this problem, maybe there has another convenient way.
You can try the following script to output the data, and draw the curves of training progress by yourself.
[trainedNet,traininfo] = trainNetwork(___)

  0 Comments

Sign in to comment.


Feruza Amirkulova
Feruza Amirkulova on 10 Aug 2019
When training process is over, use on command window:
>> h= findall(groot,'Type','Figure');
>> h.MenuBar = 'figure';
It will make traning window look like Matlab Figure, and save in jpg or eps format. I was able to save as Matlab .fig files but I wasn't able to open them.

  0 Comments

Sign in to comment.


Daniel Hussey
Daniel Hussey on 6 Nov 2019
Another solution I have found is to turn on MATLAB's diary feature, which will save the output of the training to the console in text format. This actually seems to be in Markup format, so I used an online tool (https://tableconvert.com/?output=excel) to convert the data back into something more useful, like Excel, where I could play around with the formatting and such better. Just another option.

  0 Comments

Sign in to comment.


Mendi
Mendi on 5 Dec 2019
Go inside trainNetwork.m
Change Line 1:
function [trainedNet, info, fig] = trainNetwork(varargin)
Change line 168:
[trainedNet, info, fig] = doTrainNetwork(layersOrGraph, opts, X, Y);
Change line 175:
function [trainedNet, info, fig] = doTrainNetwork(layersOrGraph, opts, X, Y)
In line 271 (before function "doTrainNetwork" end) add those lines:
idxTrainingPlotReporter=find(cellfun(@(r)isa(r,'nnet.internal.cnn.util.TrainingPlotReporter'),reporters.Reporters),1);
if ~isempty(idxTrainingPlotReporter)
fig=reporters.Reporters{idxTrainingPlotReporter}.TrainingPlotPresenter.TrainingPlotView.Figure;
else
fig=[];
end
Now in you can call:
[net,info,fig] = trainNetwork(trainingData,net,options);
saveas(fig,'myfig.png')

  0 Comments

Sign in to comment.

Sign in to answer this question.


Translated by