Relation between input data points and hyper parameters that needs to be tuned

Hi All,

Can anyone please let me know the relationship between the number of input data points and the hyperparameters/number of layers that needs to be present in any machine learning model?

Thanks for your time and help

[KSSV]

Read about hyperparameters function in matlab.

Answer 1

[Greg Heath]

 [ I N]  = size(input)
 [ O N ] = size(target)

 % (MATLAB DEFAULT)
   Ntst    = round(0.15*N)
   Nval    = Ntst
   Ntrn    = N-(Ntst+Nval)%  ~ 0.7*N

 % Design parameters 
   Ndes = Ntrn*O           % No. of design equations ~ 0.7*N*O
   H                       % No. of hidden nodes for I-H-O net
   Nw   = (I+1)*H+(H+1)*O  % No. of unknown weights

 Require  Ndes >= Nw   ==> H <= Hub = (Ntrn*O-O)/(I+O+1)
 Desire   Ndes >> Nw   ==> H << Hub

My typical goal: Minimize H subject to the requirement

         MSE < = 0.01*var(target',1)   % Rsquare >= 0.99

My approach:

 1. Apply the requirement to the training data
 2. Loop over H to find the minimum H to satisfy the 
    requirement.

I have hundreds of examples in the NEWSGROUP comp.soft-sys.matlab as well as ANSWERS.

Hope this helps

Thank you for formally accepting my answer

Greg

[Venkat]

Hi Greg,

Thanks for your time and input. I understood what you have said. My problem is I am using CNN. My inputs are images of size 16x512 and I have 30,000 image samples per class, totaling 60,000 representing my 2 classes.

In order to decide on the number of layers in CNN along with the number of filters in each convolutional layer, I am doing an iterative process, but that is going to take a long time. So I am trying to see if I can derive some generic numbers I can start with rather than iterating from 1 to N as far as the number of filters is concerned.

Can I apply the same rule? If yes, can you please explain a little bit more

Thanks

Answer 2

[Greg Heath]

Each case is different. However, things tend to be relatively straightforward if you have at least as many training equations as you have unknowns.

[Venkat]

Thank a lot.