LSTM For classification of EMG sequences

hello,

I am trying to classify EMG signals with the movement that generated the original signal

I have tried using a patternnet but this gives very bad generalization (I think is due to my lack of knowledge in creating the ANN structure and input format) so I have been investigating LSTM networks using matlab,

am i right in following the sequence to label classification example that is provided in the documentation?

https://uk.mathworks.com/help/deeplearning/examples/classify-sequence-data-using-lstm-networks.html

Thanks

[bavi]

Hey ,@CHRISTOPHER MILLAR I am curious about your project.

I am also trying to classify the hand movement of data from semg sensor data.

I used the classification learer app whuch was failure.

Can you explain your approach?

[CHRISTOPHER MILLAR]

In the end i followed 1 of the examples in the documentation to develop a script and used the Experiment Manager App to do the all parameter tuning

at the beginning it was difficult to figure out but i got there in the end. Getting the data in the correct format is VIP then just follow the structure for the networks in the samples.

For Example an LSTM is created like this

layers = [

sequenceInputLayer(numFeatures,'Name','sequence')

lstmLayer(1500,'OutputMode','last','Name','lstm')

dropoutLayer(0.5,'Name','drop')

fullyConnectedLayer(numClasses,'Name','fc')

softmaxLayer('Name','softmax')

classificationLayer('Name','classification')];

you just need to amend the parameters to match your requirements

Hope this helps

Chris

Answer 1

[Raunak Gupta]

Hi,

For using the example that is mentioned I assume you have several features from the EMG Signal and each EMG signal is a sequence of those features. Since EMG Signal is kind of signal that has temporal relationship between time frames, LSTMs will perform good in capturing the pattern across the data. The preparation of data according to the problem statement is the only concern while using LSTMs. So, I suggest setting up the data such that each datapoint corresponds to a sequence of features and its class.

[CHRISTOPHER MILLAR]

Hi Raunak,

Thanks for your reply, can i just advise you of the format of my data that I am inputting into my LSTM and ask you to confirm that I am following the correct format protocols?

The device I am using to collect the EMG data has 8 channels so for each movement I have a 200 data point sequence across 8 separate channels and I have 12 different movement classes that I have recorded 20 times each so for example:

For each movement I have 20x1 cell that holds each 8 * 200 movement sequence

1. 8x200 double
2. 8x200 double
3. 8x200 double
4. 8x200 double
5. 8x200 double
6. 8x200 double
7. 8x200 double
8. 8x200 double
9. 8x200 double
10. 8x200 double
11. 8x200 double
12. 8x200 double
13. 8x200 double
14. 8x200 double
15. 8x200 double
16. 8x200 double
17. 8x200 double
18. 8x200 double
19. 8x200 double
20. 8x200 double

this gives me a complete data set of 240 (12 movements X 20) x 1 cells with each cell containing an 8 * 200 sequence. I then label each cell in accordance with the movement class that it corresponds with and then input in the LSTM network that i created using the example provided to specify the layers and training options

inputSize = 8;

numHiddenUnits = 50;

numClasses = 12;

dropoutLayer(0.3,'Name','drop1');

layers = [ ...

sequenceInputLayer(inputSize)

bilstmLayer(numHiddenUnits,'OutputMode','last')

fullyConnectedLayer(numClasses)

softmaxLayer

classificationLayer];

Can you confirm to me that is indeed the correct way of going about formatting my data and configuring the layers of my LSTM?

Thanks in advance of any help you can provide

Chris

[Raunak Gupta]

Hi,

This indeed looks correct, since the sequence size is same across all the data points you can skip the padding part mentioned in the example. rest all you may follow from the same example.

[CHRISTOPHER MILLAR]

Raunak

Thank you so much for taking the time to confirm this for me, I appreciate your help so much.

I just have 1 further question regarding LSTM

I want to use the same weights to train future LSTMs to perform comparisons I have found how to extract different layer weights but I am not sure how to assign these weights when configuring future LSTM networks?

Extracting Weights from a trained LSTM network

IW = lstmNet.Layers(2,1).InputWeights;
RW = lstmNet.Layers(2,1).RecurrentWeights;
B = lstmNet.Layers(2,1).Bias;
FCW = lstmNet.Layers(3,1).Weights;
FCB = lstmNet.Layers(3,1).Bias;

Can you advise if this is possible?

Regards

Chris

[Raunak Gupta]

Hi,

For initializing a new Network with some same weights, you can define the 'InputWeights' , 'RecurrentWeights' and 'Bias' Property in the layer itself.

You may look it here.

Below example would be according to your last comment. These can be layers for new network.

layers = [sequenceInputLayer(inputSize)
    bilstmLayer(numHiddenUnits,'InputWeights',IW,'RecurrentWeights',RW,'Bias',B,'OutputMode','last')
    fullyConnectedLayer(numClasses,'Weights',FCW,'Bias',FCB)
    softmaxLayer
    classificationLayer]

[CHRISTOPHER MILLAR]

Hi again Raunak,

Once again thanks for the quick response

I have tried what you have sugested but I get an error saying

'InputWeights' matches multiple parameter names: 'InputWeightsL2Factor', 'InputWeightsLearnRateFactor'. To avoid ambiguity, specify the complete name of the parameter.

I dont think either of thes options applies to what I am trying to do, do you have any suggestions on how to overcome this problem?

Regards

Chris

[Raunak Gupta]

Hi,

I am refering to bilstmlayer only and the above mentioned error should not appear. Can you share the code which is used to save the weights and biases into some variables and then these variables are assigned to the new network that is needed. It will help me figure out the error. Also it may be helpful if you provide the error message in quotes.

[CHRISTOPHER MILLAR]

Hi Raunak,

This first section is how i extract the weights for each layer from the initial trained LSTM's

for i = 1:NNTrials

IW{i} = lstmNet{i}.Layers(2,1).InputWeights;
RW{i} = lstmNet{i}.Layers(2,1).RecurrentWeights;
B{i} = lstmNet{i}.Layers(2,1).Bias;
FCW{i} = lstmNet{i}.Layers(3,1).Weights;
FCB{i} = lstmNet{i}.Layers(3,1).Bias;

end

I then pass these weights to a function to train new LSTM's after changing some parameter so as to get a fair comparison between LSTMs

iw = IW{i,1}; b = B{i,1};
 rw = RW{i,1}; fcw = FCW{i,1}; fcb = FCB{i,1};

newLayers = [ ...
    sequenceInputLayer(inputSize)
    bilstmLayer(numHiddenUnits,'InputWeights',iw,'RecurrentWeights',rw,'Bias',b,'OutputMode','last')
    fullyConnectedLayer(numClasses,'Weights',fcw, 'Bias',fcb)
    softmaxLayer
    classificationLayer];

And this is the Error message that is generated when I try to execute

trainNetwork(XTrain,YTrain,newLayers,newOptions);

Error using nnet.cnn.layer.BiLSTMLayer.parseInputArguments (line 371)
'InputWeights' matches multiple parameter names: 'InputWeightsL2Factor', 'InputWeightsLearnRateFactor'. To avoid ambiguity, specify the complete name of the parameter.

Thanks in advance of any help you can provide

Regards

Chris

[Raunak Gupta]

Hi,

From the error message I can see that this issue was there with R2018a and R2018b release and is fixed in R2019a and later release. So, I suggest installing the latest release.

[CHRISTOPHER MILLAR]

Raunak,

I downloaded the newest version of Matlab and this code now works

Thank you very much for all your help with this

Regards

Chris