Teacher forcing for a LSTM network

21 views (last 30 days)
Philip Hua
Philip Hua on 22 May 2022
Commented: Philip Hua on 7 Jun 2022 at 13:20
Is there a way to implement teacher forcing for a LSTM network in MATLAB? Hopefully there is an option buried somewhere.

Answers (2)

David Willingham
David Willingham on 24 May 2022
Hi Philip,
This example shows how teacher forcing can be implemented with LSTM's in MATLAB.Sequence-to-Sequence Translation Using Attention.
  1 Comment
Philip Hua
Philip Hua on 24 May 2022
hi David,
Thank you for your help but this is a attention model. I just need TF for a standard LSTM model.

Sign in to comment.


David Willingham
David Willingham on 24 May 2022
Edited: David Willingham on 24 May 2022
Please see the attached example, trainLSTM_seq2seq. Is this what you were looking for? I.e. an example for a standard LSTM model?
  5 Comments
Philip Hua
Philip Hua on 7 Jun 2022 at 13:20
hi David,
Thank you for your email. I just implemented the basic code without teacher forcing yesterday and during training it gives 30-40% accuracy. The author
seems to suggest that the model should perform much better. There are two things missing from my model currently:
1) i have not embedded the tokens (page 27) and
2) No teacher forcing
The problem with 1) is that these tokens are categorical tokens so after embedding (using the embed functions), I don't know how to retrieve the original tokens from the embedded data. I also presume that the data has to be discretized and there is no mention in the thesis of this so I am not sure what is going on TBH. Perhaps you can shed some light on this
2) i was going to try to use what you send to implement teacher forcing but I thought the open loop is a much neater way to implement the solution in this case. I am also dubious about TF - i suspect that all it's going to do is to overfit the data.

Sign in to comment.

Products


Release

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by