Avoiding Overfitting by Averaging Multiple Neural Network

I’m using a narnet neural network to model stock time series data so I can get step-ahead and multi-step predictions. I’m concerned about overfitting in this model and have been reading in Matlab about techniques to avoid overfitting. I would like to try averaging multiple networks to improve generalization and train multiple neural networks and average their outputs (as in the example provided). The example shows this technique on a feedforward net. So, my question is (and please forgive me is I ask stupid questions since I’m still a relatively new user of Matlab and neural networks) can I use the averaging technique on my narnet network? If so, would the same example work? If not and I need to use the feedforward net, how then can I use that output in my model to find step-ahead and multi-step predictions for my stock time series? I would appreciate any input and/or suggestions. Many thanks. Robert Henson

Answer 1

[Greg Heath]

The best way to mitigate overtraining an overfit net is

MINIMIZE THE NUMBER OF HIDDEN NODES SUBJECT TO A MAXIMUM ALLOWED ERROR RATE.

The conventional I-H-O NN has I Input nodes, H Hidden nodes, O output nodes and Nw unknown weights where

 Nw = (I+1)*H+(H+1)*O = (I+O+1)*H +1

With Ntrn training examples the total number of training equations is

 Ntrneq = Ntrn*O

To prevent overfitting: No. eq >= No. unknowns:

 Ntrneq >= Nw

or

 H <= Hmax <= Hub = (Ntrn*O-1)/(I+O+1)

 I have posted zillions of examples in BOTH 
comp.soft-sys.matlab and ANSWERS.

Search on "greg" and one or more of the following Ntrneq, Hmax, Hub

Thank you for formally accepting my answer

Greg