How to calculate mini-batch MSE value in trainNetwork function?

KR:

neural network toolbox의 trainNetwork함수에서 minibatch MSE는 어떻게 계산하죠? adam 알고리즘으로 training 할때 weight와 bias가 업데이트되는 것을 설정할 수 있나요?

EN:

In neural network toolbox, how "trainNetwork" function calculates mini-batch loss & mini-batch MSE? and while they training by adam algorithm(trainingOptionsAdam), how can I check updated the weight and bias at each epoch?

Answer 1

[Krishna]

Hello,

I see you're interested in learning more about the trainNetwork function in MATLAB. This function operates similarly to the process of training a deep learning network. It begins with forward propagation, followed by calculating gradients using backpropagation on the loss function. Subsequently, the weights are updated based on the algorithm you're using, which in your case is 'Adam'. All these steps are automatically handled by the trainNetwork function.

Since Adam is a stochastic optimizer, it processes data in mini-batches. During forward propagation, data is passed in these mini-batches, and all necessary steps are completed before moving on to the next set of mini-batches. However, if you wish to access the weights and biases, you will need to implement a custom training loop in MATLAB and record the weights and biases at each iteration.

You can view the min-batch loss at every iteration and plot the training-loss and validation-loss graph of mini-batches at every iteration using trainNetwork, by setting the plot to "training-progress" in trainingOptions.

Please go through the following documentation to learn more regarding custom training loop and how to implement them,

https://in.mathworks.com/help/deeplearning/deep-learning-custom-training-loops.html

Please go through the following documentation to learn more regarding 'trainNetwork' function,

(Not recommended) Train neural network - MATLAB trainNetwork - MathWorks India

Also, currently MATLAB does not recommend 'trainNetwork' function instead you can use 'trainnet' function.

https://in.mathworks.com/help/deeplearning/ref/trainnet.html

Please go through the following documentation to learn more about how to ask question on MATLAB answer and get a fast response,

https://in.mathworks.com/matlabcentral/answers/6200-tutorial-how-to-ask-a-question-on-answers-and-get-a-fast-answer

Hope this helps.