How to format sequences to store in experience buffer for DRQN?

16 visualizzazioni (ultimi 30 giorni)
For DRQN (Deep Recurrent Q Learning) in POMDP it is needed to store entire sequences instead of individual transitions in the replay buffer. For the object agent.ExperienceBuffer, how to construct the data? For example, for Observation element i have tried to have a 1x1 cell with inside the numchannel x sequencelength, and also to have a cell array directly numchannel x sequencelength. the idea was to then sample minibatch of sequences instead of minibatch of transitions.
For any trial I get an error
Error using rl.replay.rlReplayMemory/validateExperience
Observation dimensions must match the dimensions specified in the corresponding specifications.
More specifically, when debugging i see that in the first case (1x1 cell) the code crashes at :
for obsCh = 1:numObsChannels
if ~all(size(NewObs{obsCh}) == obj.InternalReplayMemory_.ObservationDimension{obsCh})
error(message('rl:general:errIncorrectObservationDim'));
end
And in the second case at:
if numObsChannels ~= numel(NewObs)
error(message('rl:general:errIncorrectObservationDim'));
end
In MATLAB it is possible to have dqn with recurrent layers, so there is certainly a way to store these sequences somehow.
Thank you,
Imola

Risposte (0)

Categorie

Scopri di più su Sequence and Numeric Feature Data Workflows in Help Center e File Exchange

Prodotti


Release

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by