Forecasting Adjusted Close price of financial instrument through LSTM Neural Network (Help!)

Hello everybody :)

I am working on my master thesis project in which I am trying to build a Long-Short term memory neural network for predicting the Adjusted Close price of an ETF. I am starting with six different time series in an Excel file, the first is the one to be predicted and the other five are the inputs (Open, High, Low, Close and Volume, the ones you download from Yahoo Finance). I've splitted the dataset into 70% training and 30% testing through "cvpartition" and then converted the tables into arrays. Then I normalized the data through the function "normalize" and I divided the train dataset and the test dataset in X_train Y_train/ X_test Y_test. Subsequently I built the architecture and the options of the algorithm in this way:

%Architecture of LSTM

num_features = 5;

num_responses = 1;

num_hidden_units = 200;

layers = [

featureInputLayer(5);

lstmLayer(num_hidden_units, 'OutputMode','last')

fullyConnectedLayer(num_responses)

regressionLayer];

%Training Options

options = trainingOptions("adam", ...

MaxEpochs=200, ...

SequencePaddingDirection="left",...

InitialLearnRate=0.01,...

Shuffle="every-epoch", ...

GradientThreshold=1, ...

Verbose=false, ...

Plots="training-progress");

%Training LSTM

net = trainNetwork(X_Train, Y_Train,layers,options);

And predicted the output in this way:

y_pred = predict(net, X_Test, SequencePaddingDirection="left");

The result is good, maybe too good. I am reporting a Root Mean Square Error of 0.02, am I doing right or am I just prooving the obvious?

Can somebody Please give me a hint?

Thank You so much for the help

Answer 1

[Vandit]

Hello Davide,

Based on the code you provided, it seems that you have followed the correct steps for building and training your LSTM neural network. Achieving a low Root Mean Square Error (RMSE) of 0.02 indicates that your model is performing well on the test dataset.

However it is possible that your model is overfitting the training data, resulting in overly optimistic performance on the test data. To address this, you can apply regularization techniques like L2 regularization to prevent overfitting in LSTM models.

To know more L2 regularization, please refer to the link given below for your reference:

https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.layer.setl2factor.html

Hope this helps.