Precision network analysis NarX
1 visualizzazione (ultimi 30 giorni)
Mostra commenti meno recenti
Well I think I know how to do it: in the matrix of new inputs and new targets I can go poking above the predicted outputs and by looping multiple outputs could get ahead, because as I've seen in a code, to perform 300 iterations of a one time I need to give the 300 inputs (question that is impossible because of the future inputs do not know). Now the dilemma is this: I have done an analysis of details, I've closed the loop and I checked with more data if more or less fulfilled the prediction, but unfortunately I have not been successful. For this I used about 2100 but I think data are insufficient to achieve the desired accuracy. Besides the variables are correlated but do not show much correlation (0.35 or so). In openloop get accuracy R2a = 0.82 with MSE = 0.0016 and performing the analysis of R2 in training / validation / test none of them made me overfitting. You think that you should use more data? thank you very much.
0 Commenti
Risposta accettata
Greg Heath
il 26 Feb 2013
1. Use original trn/val/tst data and 'divideblock' to create and evaluate the open loop net.
2. If val and tst performance is unsatisfactory, try to design better nets via changing delays, numHidden, random initial weights and/or data-split ratios. Oherwise ...
3. Close the loop and evaluate using the ORIGINAL data. Unbiased val and tst performances are much more important than the biased trn performance (even if the latter bias is mitigated by a degree-of-freedom-adjustment).
4. Predict closed loop performance using NEW test input data. If corresponding NEW target data is available, evaluate the performance.
I think data are insufficient to achieve the desired accuracy. Besides the variables are correlated but do not show much correlation (0.35 or so).
Are you referring to zero lag correlations obtained from corrcoef or dynamic correlations obtained from nncorr, xcorr or crosscorr? Nonzero lag correlations are usually significant above 0.14 or so.
In openloop get accuracy R2a = 0.82 with MSE = 0.0016 ...
Relatively irrelevant. What are R2val and R2tst?
and performing the analysis of R2 in training / validation / test none of them made me overfitting.
Overfitting is usually detected by high R2trn but low R2val, R2tst and possibly R2trna.
You think that you should use more data?
If Nw << Ntrneq, you probably have enough training data.
If std(MSEtst) << MSEtst you probably have enough test data and similarly for val.
Look up the expression for std(MSE) assuming MSE is Chi-square distributed in Wikipedia (I only remember it for binomially distributed classifications).
Hope this helps.
Thank you for formally accepting my answer.
Greg
4 Commenti
Greg Heath
il 27 Feb 2013
I don't know. How are your t1, t2, t3 and MSE00 defined?
What are I, N, O, Ntrn, Nval, Ntst, ID, FD, and H?
Let
mxD = max(ID,FD) % size of delay buffer
then
1. The most unbiased approach is to
a. Only use the prior information stored in netc
b. Only use test set data that was used in no way for design
2. If Ntst is not sufficiently large to satisfy std(MSEtst) << MSEtst it may be useful to add a small bias by using the preceeding mxID vectors from the design data (val or trn if Nval = 0).
Più risposte (0)
Vedere anche
Categorie
Scopri di più su Modeling and Prediction with NARX and Time-Delay Networks in Help Center e File Exchange
Prodotti
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!