LSTM time series forecasting with 3 inputs using chickenpox example

Please let me know how to apply 3 inputs for the time series forecasting using LSTM example below.

chickenpox example link:

https://kr.mathworks.com/help/deeplearning/examples/time-series-forecasting-using-deep-learning.html?lang=en

In this example, input is one(numFeatures = 1;) and I'm having trouble with changing the codes for the multiple inputs.

I'm not sure with the data input structures, and following codes for it.

It would be appreciate to provide the example or explanation about it.

Thank you.

[Marlene Goetz]

Hey, did you found an answer? I am facing the same problem.

Thank you.

[Shuhei IKEDA]

Hi there, I'm facing the same issue.

Could anyone help us!?

[Uros Grivc]

Try this:

your input data will have to look like oil example in nnstart timeseries app, this example has two input wariables and the shape of the input works with lstm like chikenpox example...

one timestep with x y z n ... variables should look like this

[x;y;z;n;...]

it should be a column vector (this is why ; insted of , is used)

[diana haron]

Hi, did you manage to solve this ?

[Hanim Basarudin]

I think you have to use con2seq() to turn the multiple variables matrix into sequential vector like what Uros has described.

https://www.mathworks.com/help/deeplearning/ref/con2seq.html

[Rik]

Comment posted as flag by Amin mirboluki:

How can I use this?

Answer 1

[Jon Cherrie]

Rather than starting from the chickenpox example, you might be better off starting from the "Sequence-to-Sequence Regression Using Deep Learning" example:

• https://au.mathworks.com/help/deeplearning/ug/sequence-to-sequence-regression-using-deep-learning.html

In that case, the input is from 26 sensors and there is one output, an estimate of the remaining useful life of the engine.

[Ismaeel]

That example with 26 sensors is hard to follow. The question is simple:

How to predict the future if the input is a multi-input matrix?

The chickenpox example is very helpful to learn but it is challenging to edit it for a multi-input example. That example with 26 sensors uses the function "predict" to estimate one time-step ahead while the chickenpox example is used to loop over multiple time-step ahead.

Any help would be very much appreciated.

[Jon Cherrie]

Here's anothe example. Does it help with what you are looking for?

This is based on pieces from the following examples.

openExample('ident/ForecastingPredatorPreyPopulationsExample')

openExample('nnet/TimeSeriesForecastingUsingDeepLearningExample')

The data is taken from the first one and LSTM settings are inspired by the second.

Load Data

The data is a bivariate time series consisting of 1-predator 1-prey populations (in thousands) collected 10 times a year for 20 years. For more information about the data, see Three Ecological Population Systems: MATLAB and C MEX-File Modeling of Time-Series.

predPreyCrowding = load("PredPreyCrowdingData").y.';

Use the first 120 time steps for training the model and the rest of the data to test the model

trainingData = predPreyCrowding(:,1:120);
testData = predPreyCrowding(:,121:end);

tiledlayout("flow")
nexttile
plot(1:120,trainingData(1,:),121:201,testData(1,:))
title("Species 1")
legend("Training Data", "Test Data")
nexttile
plot(1:120,trainingData(2,:),121:201,testData(2,:))
title("Species 2")
legend("Training Data", "Test Data")

Prepare Training Data

In this case, the model will take one time step as input and provide the next time step as output. Prepare the data into this by creating a 'output' (y) that is one time step ahead of the 'input' (x)

XTrain = trainingData(:,1:end-1);
YTrain = trainingData(:,2:end);

Define Model

For the model, use an LSTM with two inputs (numFeatures = 2) and two outputs (numResponses = 2). That is one input and one output for each species.

The number of hidden units controls the complexity of the model.

numFeatures = 2;
numResponses = 2;
numHiddenUnits = 10;

layers = [ 
    sequenceInputLayer(numFeatures)
    lstmLayer(numHiddenUnits)
    fullyConnectedLayer(numResponses)
    regressionLayer
    ];

Train Model

options = trainingOptions('adam', ...
    'MaxEpochs',100, ...
    'GradientThreshold',1, ...
    'InitialLearnRate',0.005, ...
    'LearnRateSchedule','piecewise', ...
    'LearnRateDropPeriod',125, ...
    'LearnRateDropFactor',0.2, ...
    'Verbose',0, ...
    'Plots','training-progress');

net = trainNetwork(XTrain,YTrain,layers,options);

Run Inference: Predict the Future

net = resetState(net);

The network is initialized to work from the start of the training set. Before making predictions beyond the training set, need to run inference up to the point the test data starts:

[net,YBefore] = predictAndUpdateState(net,trainingData);

The last of the predictions from inputs 'before' is the first of predictions beyond the training data

YPred = YBefore(:,end);

Because of the way the model is doing one step ahead a prediction, the output from one time step, becomes the input to the next time step

numTimeStepsTest = 81;
for i = 2:numTimeStepsTest
    XThis = YPred(:,i-1);
    [net,YNext] = predictAndUpdateState(net,XThis);
    YPred(:,i) = YNext;
end

% Plot
tiledlayout("flow")
nexttile
plot(1:120,trainingData(1,:),121:201,testData(1,:),1:120,YBefore(1,:),121:201,YPred(1,:))
title("Species 1")
legend("Training Data", "Test Data", "Prediction", "Forecast")
nexttile
plot(1:120,trainingData(2,:),121:201,testData(2,:),1:120,YBefore(2,:),121:201,YPred(2,:))
title("Species 2")
legend("Training Data", "Test Data", "Prediction", "Forecast")

Compare with predictions running on the whole time series, not just beyond the training data

net = resetState(net);
[net, YPred] = predictAndUpdateState(net,predPreyCrowding(:,1));
for i = 2:201
    [net,YPred(:,i)] = predictAndUpdateState(net,YPred(:,i-1));
end

% Plot
tiledlayout("flow")
nexttile
plot(1:120,trainingData(1,:),121:201,testData(1,:),1:201,YPred(1,:))
title("Species 1")
legend("Training Data", "Test Data", "Predictions")
nexttile
plot(1:120,trainingData(2,:),121:201,testData(2,:),1:201,YPred(2,:))
title("Species 2")
legend("Training Data", "Test Data", "Predictions")

[Ismaeel]

Thank you Jon Cherrie, for your help.

It works for me now (with an edit in the standerization and destanderization of the input-output data).

Thank you very much.

[mhd z]

Hello @Jon Cherrie

Thank you for this valuable example. As you mentioned in the example, the model gets one input of each and gives one output for each one of spicies.

• my question is that do they effect each other when the model is trained or when the data are forecasted? or the model just treats them as separate data and forecasts them? ( for example in stochastic models with exogenous serirs, the exogenous sesris help the model to model the original series more accurately. is it the same?)
• The other question is if I I want to model (forecast) only one of them, how should I change the code? is it possible at all? ( I tried to get help from the Turbofan Engine example, I tried to change some parts of this code by copying some of adjustments in the Turbofan exampple, but I was not successful. the code does throws an error in the training section. there is only 2 time series and there is no need to miniBatch. so, by changing only the number of features and responses in the layers and options, the code should work. but it didn't. I tried with miniBatch and it did not pass the training section.)

the questions might be silly and too long, but I would be very grateful if you answer, as much as you can.

Error using trainNetwork (line 183)
Number of elements must not change. Use [] as one of the size inputs to automatically calculate the appropriate size for that dimension.
Error in MultiVarLSTM_Pred_Ex_test (line 66)
net = trainNetwork(XTrain,YTrain,layers,options);
Caused by:
Error using reshape
Number of elements must not change. Use [] as one of the size inputs to automatically calculate the appropriate size for that dimension.

[Imrana]

Hello mhd z,

Did you find your answer? I have same questions