Neural network fitting result problems

Hi, Does any one can teach me how can read the neural network fitting result ? Why my MSE and R is NaN ? And this is good ?

thanks.

Answer 1

[Harsh]

Hi,

It looks like you're trying to figure out why you're seeing NaN for both MSE (mean squared error) and R (regression coefficient) after training your neural network. Let's dive into some reasons and fixes for these issues:

MSE turning into NaN could be because:

• The training might not be converging, leading the cost to shoot up infinitely.
• Invalid operations, like a divide-by-zero or taking the log of zero, might be happening. This often occurs if the learning rate is set too high.
• Please refer the following thread to learn more: https://community.deeplearning.ai/t/when-does-mse-becomes-nan/385197

R turning into NaN might be due to:

• Exploding or vanishing gradients, which can sometimes be managed by using batch normalization.
• A learning rate that's too high. Lowering it can often stabilize the training process.
• Operations that result in NaN during the training, such as division by zero or logarithm of zero.
• NaN values in your input data or errors in preprocessing (like normalization with zero variance), which can propagate NaNs through the network. Refer the following thread to mitigate this issue: https://stackoverflow.com/questions/66381703/linear-regression-contain-nan-values

I hope this explanation clarifies the potential reasons behind the NaN values you're observing and offers some pathways to resolve them. Good luck!

[WEI TSE KOUNG]

but how can I modify the learning rate ? I saw the neural network fitting app only can change the spilt and layer size. tks

[Harsh]

You can not change the learning rate directly in the "Neural Network Fitting" app. To adjust the learning rate, you'll need to write a MATLAB script. For example, if the app has set up a two-layer feedforward network, you can recreate it in your script using 'feedforwardnet' and then make your adjustments there.

Learning rate can be modified by modifying the value of the 'LearnRateDropFactor' property of trainingOptions while training a neural network, you can also adjust additional properties like 'LearnRateDropPeriod' and 'LearnRateSchedule' with MATLAB.

To view an example on the same, copy the following command in the MATLAB console:

openExample('nnet/SpecifyTrainingOptionsExample')

Please refer the following MATLAB Answer and the documentation for more information on adjusting the learning rate:

1. https://www.mathworks.com/matlabcentral/answers/427646-how-to-change-learning-rate-and-momentum-in-trainlm-neural-network
2. https://www.mathworks.com/help/deeplearning/ref/trainingoptions.html

I hope this helps, thanks!

[WEI TSE KOUNG]

So, I export generate code and add the Training Options in the follows code is that correct ? tks

[Harsh]

Yes!