Azzera filtri
Azzera filtri

Convolutional LSTM (C-LSTM) in MATLAB

81 visualizzazioni (ultimi 30 giorni)
Jake
Jake il 9 Ott 2018
Modificato: Dieter Mayer il 29 Ago 2022
I'd like to train a convolutional neural network with an LSTM layer on the end of it. Similar to what was done in:
  1. https://arxiv.org/pdf/1710.03804.pdf
  2. https://arxiv.org/pdf/1612.01079.pdf
Is this possible?

Risposte (5)

Shounak Mitra
Shounak Mitra il 9 Ott 2018
Hi Jake,
Unfortunately, we do not directly support C-LSTM. We are working on it and it should be available soon.
-- Shounak
  7 Commenti
David Willingham
David Willingham il 26 Ago 2022
Hi Dieter,
Apologies for not updating this answers post sooner. This workflow is now supported. the following code will illustrated this:
% Load data
[XTrain,YTrain] = japaneseVowelsTrainData;
% Define layers
layers = [ sequenceInputLayer(12,'Normalization','none', 'MinLength', 9);
convolution1dLayer(3, 16)
batchNormalizationLayer()
reluLayer()
maxPooling1dLayer(2)
convolution1dLayer(5, 32)
batchNormalizationLayer()
reluLayer()
averagePooling1dLayer(2)
lstmLayer(100, 'OutputMode', 'last')
fullyConnectedLayer(9)
softmaxLayer()
classificationLayer()];
options = trainingOptions('adam', ...
'MaxEpochs',10, ...
'MiniBatchSize',27, ...
'SequenceLength','longest');
% Train network
net = trainNetwork(XTrain,YTrain,layers,options);
Dieter Mayer
Dieter Mayer il 29 Ago 2022
Modificato: Dieter Mayer il 29 Ago 2022
Hi David,
Thanks for your reply! Is this workflow shows a real convolution LSTM (LSTM carries out convolutional operations instead of matrix multiplication) and is not only implied to a input matrix, which is a result of a convolution net work applied before?
Sorry for asking that, I have to learn the syntax of using the deep learning toolbox, I am a beginner. The background is, that I will use such a Conv-LSTM to make precipitation forecasts for grids bases on precipitation radar inputs from several timesteps of the last minutes / hours as discussed in this paper publication

Accedi per commentare.


Yi Wei
Yi Wei il 17 Dic 2019
Hi, can matlab support C-LSTM now?
  5 Commenti
ytzhak goussha
ytzhak goussha il 23 Feb 2021
Hey,
Sorry I didn't follow this thread and didn't see the questions.
Here is a simplified C-LSTM network.
The input it a 4D image (height x width x channgle x time)
The input type is sqeuntial.
When you need to put CNN segments, you simply unfold->CNN->Fold->flatten and feed to LSTM layer.
Ioana Cretu
Ioana Cretu il 18 Mag 2021
Hi! When I try to train the model I have this error:
Error using trainNetwork (line 170)
Invalid network.
Caused by:
Layer 'fold': Unconnected output. Each layer output must be connected to the input of another layer.
Detected unconnected outputs:
output 'miniBatchSize'
Layer 'unfold': Unconnected input. Each layer input must be connected to the output of another layer.
I connected the layers using this:
lgraph = layerGraph(Layers);
lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');
What do you think the cause is?

Accedi per commentare.


Chen
Chen il 25 Ago 2021
Please refer to this excellent example in:
It is possible to train the hybrid together.

Jonathan
Jonathan il 4 Ago 2022
inputSize = [28 28 1];
filterSize = 5;
numFilters = 20;
numHiddenUnits = 200;
numClasses = 10;
layers = [ ...
sequenceInputLayer(inputSize,'Name','input')
sequenceFoldingLayer('Name','fold')
convolution2dLayer(filterSize,numFilters,'Name','conv')
batchNormalizationLayer('Name','bn')
reluLayer('Name','relu')
sequenceUnfoldingLayer('Name','unfold')
flattenLayer('Name','flatten')
lstmLayer(numHiddenUnits,'OutputMode','last','Name','lstm')
fullyConnectedLayer(numClasses, 'Name','fc')
softmaxLayer('Name','softmax')
classificationLayer('Name','classification')];
lgraph = layerGraph(layers);
lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');

David Willingham
David Willingham il 26 Ago 2022
Updating this answer. This workflow has been supported since R2021. The following example illustrates how to combin CNN's with LSTM layers:
% Load data
[XTrain,YTrain] = japaneseVowelsTrainData;
% Define layers
layers = [ sequenceInputLayer(12,'Normalization','none', 'MinLength', 9);
convolution1dLayer(3, 16)
batchNormalizationLayer()
reluLayer()
maxPooling1dLayer(2)
convolution1dLayer(5, 32)
batchNormalizationLayer()
reluLayer()
averagePooling1dLayer(2)
lstmLayer(100, 'OutputMode', 'last')
fullyConnectedLayer(9)
softmaxLayer()
classificationLayer()];
options = trainingOptions('adam', ...
'MaxEpochs',10, ...
'MiniBatchSize',27, ...
'SequenceLength','longest');
% Train network
net = trainNetwork(XTrain,YTrain,layers,options);

Categorie

Scopri di più su Sequence and Numeric Feature Data Workflows in Help Center e File Exchange

Prodotti


Release

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by