This question is closed. Reopen it to edit or answer.
How do I determine the best model order of my Network? Also clarify some doubts regarding free parameters such as weights
1 view (last 30 days)
I have a two dimensional input and one dimensional target output. My NN has one hidden layer with 10 neurons (hidden) in it
[I N] = size(input); % I = 2, N = 100
[O N] = size (target); % O = 1
I used nftool to do the curve fitting. I trained it and saw the plots,etc all.
My questions are as follows -
1. How do I see the equation with which the NN determined the 'estimated outputs'? 2. I didn't see how I could plot my estimated output vs input mesh-plot in 3D. X and Y axis assuming to be two dimensions of input and Z as the estimated output. Since I have only target value Z* (lets say), the syntax
doesn't work. Can anybody kindly help me out here?
3. The point of asking the equation of estimated outputs in terms of inputs is to determine the model order. And I want to change the model order in order to determine the optimized model complexity. Even if I get the equation, there was no where I can see, how could I change the model order of the polynomial from say 2nd degree polynomial to 4th or 5th degree polynomial. Whereas I have seen literature on fitting the curve from straight line to 10th order polynomial.
4. My task is to find the optimum level of Regularization in order to find the optimal order of my Network. Can anybody explain me what it is with a simple example or show me the direction I should head to for understanding the same. To me, optimization always meant to try varying no. of neurons in hidden layer and change the training, validation and testing samples by hit and trial to improve the error.
Now to clarify some doubts
1. I saw @GregHeath mentioned many a times the no. of unknown weights as
Nw = (I +1)*H+(H+1)*O
Can you kindly clarify this formula? I = 2 and 3 neuron in hidden layer (H = 3) and 1 output gives 9 free weight parameter to be adjusted ( Neural Networks Demystified [Part 3: Gradient Descent] ). Why is there +1 in your I and H terms above while having a product?
2. Is 'training function' in 'nntool' is a sort of backpropagation method? What is then meant by 'adaptive learning function' in the same?
3. Can somebody kindly explain with an example the terms related to 'connection' having created the network -
biasConnect: [1; 1]
inputConnect: [1; 0]
layerConnect: [0 0; 1 0]
outputConnect: [0 1]
Thank you very much.
Akshay Kumar on 4 Sep 2018
3. biasConnect: Defines which layers have biases.The presence (or absence) of a bias to the ith layer is indicated by a 1 (or 0).
inputConnect: Defines which layers have weights coming from inputs.The presence (or absence) of a weight going to the ith layer from the jth input is indicated by a 1 (or 0).
layerConnect: Defines which layers have weights coming from other layers. The presence (or absence) of a weight going to the ith layer from the jth layer is indicated by a 1 (or 0) .
outputConnect: Defines which layers generate network outputs. The presence (or absence) of a network output from the ith layer is indicated by a 1 (or 0).
You can go through the below link to know more about these properties.