Adapt and sim produces very different outputs

Hello!

I'm playing around with Neural Network Toolbox, and I'm trying to use a custom network for time series predictions. My problem is that adapt and sim produces completely different outputs, so while my error is relatively low during training (adapt) it's large while is try to use the network with sim. In fact the outputs also looks completely different plotted. I'm using the same network, same data, the only difference as follows:

[instance.net, predictions, errors, instance.final_inputs, instance.final_layer, instance.results] = adapt(instance.net, dataIn.inputs_shifted, dataIn.targets_shifted , dataIn.inputs_initial, dataIn.layer_initial);

[predictions, instance.final_inputs, instance.final_layer, errors, instance.results] = sim(instance.net, dataIn.inputs_shifted, dataIn.inputs_initial, dataIn.layer_initial, dataIn.targets_shifted);

One more thing, that i'm doing all preprocessing manually, so instance.net.inputs{1}.processFcns is empty for all inputs and connections.

Can anyone please help me?

[Bence]

I'm into the investigation of this problem, and found that the adapt function i'm using, adaptwb calculates network outputs by nnsim.a1(net,PD(:,:,ts),Ac(:,ts+AiInd),Q); while sim uses nnsim.a(net,Pd,Ai,Q,TS,fcns);

I'm gonna check out the difference, but it's still not clear to me why adaptwb is set up like that.

[Bence]

So i found the difference is that nnsim.a iterates through all time-steps while nnsim.a1 calculate only one step as required for training.

So my impression is that probably the training rate is too high and the final weights after training doesn't represent well the network behavior over the whole time but just in the last few steps.

I'll check if this is the case, so finally i may answer my own question for the record.

Answer 1

[Bence]

Okay, finally i've realized that the learning rate was to high and because of the cyclic behavior of time series it my training always ended in a high input level section so the input weights went small. While running sim it caused my network to be under-driven at most part of the time so my predictions was different.

Case solved, with learning rate zero the adapt and sim produces the same output.

For the record: I'll try lower learning rate so my network may generalize better through different time ranges so different dynamic ranges of the input data. (The input dynamics can contain significant information so removing doesn't seem to be good, but if nothing else will do i'll make further dynamic compression. (It's already 3 times logarithmically compressed.)

Any suggestions or comments are appreciated.

Answer 2

[Greg Heath]

ADAPT is a learning algorithm that updates weights after the presentation of each input vector. Therefore, each output corresponds to a different input vector and a different set of weights.

SIM is a calculation algorithm for a net with fixed weights. Therefore, each output corresponds to a different input vector but the same set of weights.

Hope this helps.

Greg