Neural Networks toolbox - How to choose which 'divideFcn' to use for time series prediction?

I'm using a layer-recurrent network for time series prediction (predicting joint angles from EMG recordings). My inputs are data from four EMG channels, formatted as a 4-by-N cell array for the four channels across N time steps (target signal is a 1-by-N cell array). The layer delays make my ANN an infinite-impulse response filter (right?), and thus the output depends on the order that data is fed into it. Based on that, it seems to me I should set

 net.divideFcn = 'divideblock';
 net.divideMode = 'time';

so that training data is presented in sequence. However, after training a few networks with 'divideblock' and a few with 'dividerand', I got better results for 'dividerand' (I have not tried different architectures, just one with layerDelays=1:2 and hiddenLayerSize = 5). My best guess right now is that 'dividerand' can still take some advantage of temporal structures because I have a 1 kHz sampling rate but most of the input signals are below 100 Hz and most of the target signal is below 20 Hz (that is, it doesn't matter if you skip a few data points).

I thought I had a reasonable understanding of how artificial neural nets work both mathematically and in the Matlab toolbox, but this isn't clicking for me. I'm curious if there's something I don't understand about the training process, about how to format the inputs/outputs (and am doing something wrong), or if everything is working as it ought to and I'm just slightly confused. (I'm also not 100% clear on when one ought to use multiple inputs vs. one input of size > 1 for time series, but I don't think that's the issue.)

To summarize, the question is whether 'divideblock' is the appropriate choice for a recurrent network doing time series prediction, and, if not, why it's okay to use 'dividerand' given that input order matters.

If anyone has words of wisdom to help clarify, I'd greatly appreciate it. Thanks much!

Answer 1

[Greg Heath]

1. Standardize all signals to have zero-mean/unit-variance

2. Find the significant positive lags of the target autocorrelation function

3. Find the significant nonnegative lags of EACH input/target crosscorrelation function

4. For examples search the NEWSGROUP and ANSWERS using the keywords

 greg nncorr

5. Use the results of 2 and 3 to determine

     a. Which inputs to use
     b. Which timeseries function to use
     c. Which combination of delays to use

5. Unfortunately, delays for all inputs must be the same. However, it may be possible to improve performance by combining outputs of multiple nets with different input/delay combinations.

6. In order to maintain temporal correlations I typically use multiple designs with divideblock to determine both the number of hidden nodes and corresponding weights.

7. In your case where dividerand appeared to work better than divideblock, try divideind or divideint.

Hope this helps.

Thank you for formally accepting my answer

Greg

[David]

Thanks for your response. So in short, divideFcn choice is important in ANNs that predict data based on temporal patterns?

Re: 5b - by "timeseries function," do you mean narxnet vs. narnet vs. layrecnet, etc?

Thanks again.

Answer 2

[Shashank Prasanna]

You can use any approach that works best for you. For time-series data, the toolbox uses lagged observations to create new predictors for your network. This means it preserves the temporal structure of the data for the number of lags you choose. Once this is done, any random sampling of the data for validation and testing is appropriate. Hope that helps.

[Greg Heath]

I do not agree. Spacing should be the same for all 3 datasets.

[Shashank Prasanna]

I don't follow, by spacing do you mean lags?

[Greg Heath]

The time interval between data points

[Calvin]

This is good dialogue and I hope it can continue so some of us can learn. Perhaps the random data partitioning (dividerand) affect(?) on the temporal structure of the input & target data can be framed by this question: http://www.mathworks.com/matlabcentral/answers/140637-does-dividerand-really-destroy-time-series-data-autocorrelations Please guys, share your wisdom!