What are the reasons for having different outputs after several runs?

I use the same initial condition for my network, but I still get different outputs. What are the other factors that can cause this difference?

Answer 1

[Greg Heath]

The short answer is that train(only if all weights are zero) or configure( anytime) assign weights depending on the state of the RNG.

If you are training in a loop over random initial weights. The best statistically unbiased choice for the best net is determined by the minimum mse of the validation set.

 rng(4151941)             % Initialize RNG with famous birthday
 for i = 1:Ntrials
   s{i} = rng;            %Save the ith state of the rng (may not need a cell) 
   net  = configure(net,x,t);
   [ net tr ] = train(net,x,t);
    mseval(i) = tr.best_vperf;  % Best mseval over all epochs of ith run
 end

 [ minmseval ibest ] = min(mseval);
 rng     = s{ibest};            % For repeating the best design
 bestnet = configure(net,x,t);
 bestIW0 = bestnet.IW
 bestb0  = bestnet.b
 bestLW0 = bestnet.LW
 [ bestnet tr ] = train(bestnet,x,t) % NO SEMICOLON TO REVEAL ALL DETAILS!!!

Hope this helps.

Thank you for formally accepting my answer

Greg

Answer 2

[Walter Roberson]

By default, Neural Networks are initialized randomly.

[Sanaz]

I thought the default is zero, because when I use net.iw and net.b.. before training, the values are zero. If the default is random, how I can see those random initials?

[Walter Roberson]

http://www.mathworks.com/matlabcentral/answers/22100

[Greg Heath]

The obsolete newfit, newpr and newff initialize when the net is created.

The current fitnet, patternnet and feedforwardnet are either initialized by configure before training OR, if configure is not used, they will be automatically initialized by train.

In the latter case it is difficult to obtain those values. If you really need them, use configure then save or print the weights before calling train.