How to reduce the effect of a selected input on training a neural network?

Hello everybody,

I am trying to implement a recurrent neural network to classify the 4 phases of a movement.

The Output(t) is a function of Input(t): 42 signals from a single accelerometer and the output given 5 time samples before.

Now as soon as i tried this network on real time data acquisition i understand that it relies too much on the feedback input of the past output samples, sometimes predicting the future output instead of giving the present one.

Is there any way to regularize just a subset of the inputs trying to reducing this effect ?.