- Set an initial guess for the mean in the bias of the “fullyConnectedLayer” in the mean calculation.
- However, because of the non-linearities like the “tanhLayer”, directly setting the bias to achieve a specific mean after scaling and non-linear transformations can be complex.
How to set initial estimate for mean in PPO actor critic network
1 visualizzazione (ultimi 30 giorni)
Mostra commenti meno recenti
I am using a PPO actor critic network. I created the actor following this example
https://www.mathworks.com/help/reinforcement-learning/ref/rl.function.rlcontinuousgaussianactor.html
How can I set an initial guess for the mean? Currently the actor always starts with an intial mean at time zero of zero.
0 Commenti
Risposte (1)
Aneela
il 22 Mag 2024
Hi Jason Butler,
To set an initial guess for the mean in PPO actor network, modify the initial weights or biases of the layers that contribute to calculating the mean.
Assuming the desired initial mean as 5, here’s a workaround:
desiredInitialMean = 5; % Adjust this value as needed
% Since you have 3 actions, create a bias vector with 3 elements
biasForDesiredMean = repmat(desiredInitialMean / actInfo.UpperLimit, [prod(actInfo.Dimension), 1]);
% Modify the meanPath definition to include the bias initialization as a vector
meanPath = [
tanhLayer(Name="tanhMean");
fullyConnectedLayer(prod(actInfo.Dimension), ...
'Bias', biasForDesiredMean, ...
Name="fcMean");
scalingLayer(Name="scale", ...
'Scale', actInfo.UpperLimit)
];
For more information on “Bias” in the “fullyConnectedLayer”, refer to the following MathWorks documentation: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.fullyconnectedlayer.html?s_tid=doc_ta#:~:text=Layer%20biases%2C%20specified,single%20%7C%20double
0 Commenti
Vedere anche
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!