Why neural network sometimes performs worse when additional input variables are used?
4 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
Why neural network sometimes performs worse when additional input variables are used?
I would expect a neural network trained with n+1 input variables not to perform worse than a neural network (exactly same structured) trained with n input variables (the n inputs are the same). If the added input variable has no value then during the training step shouldn't all the weights associated to it be 0 and the overall performance not be worse. Is my expectation wrong? If not, why I am getting worse performance sometimes (even with seeded random generator)?
0 Commenti
Risposte (1)
Greg Heath
il 7 Lug 2017
1. You don't say what n is
2. How is the net suppose to know that n inputs is sufficient?
3. All it does is start somewhere amongst a bunch of hills. Now since it is blind and can only feel within it's reach to determine where down is, it's lucky to get anywhere.
4. If you were kidnapped and dumped in a 4-dimensional mountain range do you think you would do better if you were in 3-D??
Hope this helps.
Greg tries to roll down in the steepest direction
2 Commenti
Greg Heath
il 8 Lug 2017
There is a dreaded NN phenomenom called
OVERTRAINING AN OVERFIT NET.
An overfit net is a net that has more unknown parameters than equations used to solve those parameters. The result is that, without other constraints to satisfy, there are extra degrees of freedom that can do what ever they want. As a result, the answer between input data points is somewhat arbitrary.
Simple case: Solve
x1 + 2*x2 + 3*x3 = 6
3*x1 + 2*x2 + x3 = 6
Whatever answer you give, it is not the one I wanted... UNLESS you chose the one with x2 = 0.
Hope I got the idea across
Greg
Vedere anche
Categorie
Scopri di più su Parallel and Cloud in Help Center e File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!