Forecasting single variable time series data using LSTM

Hello all,

I am at the very early stage of LSTM and time series forecasting.. Currently I am trying to forecast a single variable time series data (length 10000) using LSTM, where train data length is 8000 and rest are the validation data. I am using a simple code as follows,

x = mydata(:,2);

input_size = 24;

output_size = 12;

train_size = round(0.8*numel(x));

x_train = x(1:train_size);

x_val = x(train_size+1:end);

XTrain = [];

YTrain = [];

for i = 1:(numel(x_train)-input_size-output_size+1)

XTrain = [XTrain; x_train(i:i+input_size-1)];

YTrain = [YTrain; x_train(i+input_size:i+input_size+output_size-1)];

end

XVal = [];

YVal = [];

for i = 1:(numel(x_val)-input_size-output_size+1)

XVal = [XVal; x_val(i:i+input_size-1)];

YVal = [YVal; x_val(i+input_size:i+input_size+output_size-1)];

end

% LSTM network architecture

numFeatures = 1; % Number of input features

numResponses = 1; % Number of output responses

numHiddenUnits = 200; % Number of LSTM hidden units

layers = [ ...

sequenceInputLayer(numFeatures)

lstmLayer(numHiddenUnits,'OutputMode','sequence')

fullyConnectedLayer(numResponses)

regressionLayer];

% Define the training options

maxEpochs = 200;

miniBatchSize = 128;

options = trainingOptions('adam', ...

'MaxEpochs',maxEpochs, ...

'MiniBatchSize',miniBatchSize, ...

'ValidationData',{XVal,YVal}, ...

'ValidationFrequency',10, ...

'GradientThreshold',1, ...

'InitialLearnRate',0.01, ...

'LearnRateSchedule','piecewise', ...

'LearnRateDropFactor',0.1, ...

'LearnRateDropPeriod',100, ...

'Verbose',0);

% Train the LSTM network

net = trainNetwork(XTrain,YTrain,layers,options);

With the input size 24 and output size 12, I have 24*7965=191160 number of XTrain and 12*7965=95580 number of YTrain data. The XVal (validation data) contains 24*1965=47160 and YVal contains 12*1965=23580 data.

However, I am getting an error (while training) as "The training sequences are of feature dimension 191160 but the input layer expects sequences of feature dimension 1". As per my understanding, I have a single variable and number of input features should be 1. The XTrain, YTrain, XVal and YVal are one dimensional arrays. Could you please help me to solve this error problem?

Answer 1

[Ben]

The XTrain, YTrain, XVal and YVal must all be cell arrays with size (Number of Observations) x 1, and where each entry XTrain{1} is a (NumFeatures) x (SequenceLength) array.

If the number of input features is 24 then you need to make the input size of the sequenceInputLayer 24 also, and if the number of output features is 12 then the final fullyConnectedLayer should also have hidden size equal to 12.

[Amitrajit Mukherjee]

Thank you so much Ben, I will check and update...

[Forough]

Hello Ben:

Thanks for your answer. I have problem in undertanding how can I associate my own dataset to these three parameters:

1) Number of Observations

2) NumFeatures

3) SequenceLength

Let's say I have 3D cartesian position of an object during 300 seconds of its movement. So I have 300 sample of x,y and z coordinates of the object. And then I want to train an LSTM network to predict its movement in future. What will be these three above parameters for my problem?

Thanks a lot

[Ben]

Hi @Forough,

It sounds like this is a sequence length of 300 (the time steps of the data), with 3 features (the x,y,z coordinates) and 1 observation.

[Forough]

Thanks a lot Ben. Does LSTM work for only one set of observation like I said?

[Ben]

Sure, you can train an LSTM on one observation. How well it can extrapolate beyond the training times will depend on how complex the dynamics of the object - generally the LSTM can only predict what it sees in the training data.