使用PPO和TRPO算法在reinforcement learning design app输出连续动作时,动作值不在设定好的区间内
19 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
%Open model
mdl='FCEV';
blk='FCEV/RL Agent';
%open_system(mdl);
%(s,a)
obsInfo = rlNumericSpec([3 1]);
obsInfo.Name = 'observations';
obsInfo.Description = 'A, B, and C';
numObservations = obsInfo.Dimension(1);
actInfo = rlNumericSpec([1 1],'LowerLimit',0,'UpperLimit',35);
actInfo.Name = 'Pbat_factor';
numActions = actInfo.Dimension(1);
%env
env = rlSimulinkEnv(mdl,blk,...
obsInfo,actInfo);
需要的动作区间是在0到35连续输出,但是在reinforcement learning design app中使用PPO或者TRPO算法输出的动作不在区间内,甚至一直输出负值。请问是网络结构设置有问题还是什么原因?
0 Commenti
Risposte (1)
Aiswarya
il 27 Feb 2024
请注意,我将用英语回答这个问题。
(Please note that I will be answering the question in English.)
I understand that you are using the PPO/TRPO agent and trying to set the "LowerLimit" and "UpperLimit" of data space as "actInfo" using "rlNumericSpec" function. But the action output values are not in the range specified by you.
The action bounds depend on the type of agent. Both PPO and TRPO are on-policy agents and they don’t enforce constraints set in the action specification (specified using "rlNumericSpec") for continuous action spaces. If you want to enforce these limits you have to do it explicitly on the environment side.
You may refer to the "rlTRPOAgent" documentation which also mentions the fact that action bounds need to be set by the user within the environment:
Also note that this is not the case with agents like SAC, for which the action bounds can be enforced with "rlNumericSpec".
0 Commenti
Vedere anche
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!