Is it possible to apply upper and lower bounds to predictions in an LSTM?
1 visualizzazione (ultimi 30 giorni)
Mostra commenti meno recenti
My predictions, which have been running very well as late, have hit an issue, where they are forecasting well below what should be the minimum possible value, i.e. forecasting at ~200, when the minimum should be 0. Is there anyway of simply applying a constraint/bounds, or indeed using the historical data to apply this?
0 Commenti
Risposte (1)
Ben
il 13 Mar 2023
I believe the default LSTM has outputs bounded in (-1,1) due to the activation functions used.
In any case you can try using activations to add constraints to your model, e.g. a tanhLayer has outputs bounded in (-1,1) and you can add a fixed linear transform to modify that to an arbitrary interval (a,b) as follows:
a = 0; b = 10;
% since tanh(x) is in (-1,1), (tanh(x) + 1)/2 is in (0,1).
% then scale to (0,b-a) and translate to (a,b);
boundToIntervalLayer = functionLayer(@(x) a + (b-a)*(tanh(x)+1)/2);
layers = [
sequenceInputLayer(1)
lstmLayer(1)
fullyConnectedLayer(1)
boundToIntervalLayer];
net = dlnetwork(layers);
% test it out
seqLen = 10;
x = dlarray(randn(1,seqLen),"CT");
y = forward(net,x);
% you can check y is in (a,b)
Alternatively if you use a custom training loop you could add a "soft constraint" by adding a penalty term to the loss function that is large when the network predicts values outside of your bounds.
0 Commenti
Vedere anche
Categorie
Scopri di più su Deep Learning Toolbox in Help Center e File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!