Action of the RL agents actions change when deployed in a different enviornment
4 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
Jayalath Achchige Damsara Udan Jayarathne
il 19 Mar 2023
Commentato: Jayalath Achchige Damsara Udan Jayarathne
il 23 Mar 2023
Hi all,
I have an RL agent trained in a environment (env 1- a simulink model). The sample time of the agent is 0.1s. It uses a variable step solver (DormanPrince) to solve each episode. After training is complete I export the same agent to a different enviornemnt (env 2 - a more complex environement compared to env 1) and deploy without changing any parameters. This environement does not have any randomness built into it. Also it is solved suing a variable step size solver (DormanPrince). However, when I run simulations with env 2 with the same initial condtions, I get different results. (Les say the trajectory I am calculating changes each time I run the simulation).
Why does this happen even when there is no randomness in the simulation model? If I run the simulation without the agent with same initial conditions I get a solution which does not change everytime I run it.
Plese let me know if anyone has encountered this. Thank you!!
0 Commenti
Risposte (1)
Emmanouil Tzorakoleftherakis
il 20 Mar 2023
Modificato: Emmanouil Tzorakoleftherakis
il 20 Mar 2023
A couple of suggestions/comments:
1) You mentioned env1 and env2 are different - why are you expecting to see the same results? Variable step solvers can lead to different numerical results if the stiffness of the underlying equations that are integrated changes. Even if env1 and env2 are modeling the same system, the more complex version is likely to be more stiff, which can change the simulation results.
2) Which agent are you using? Some agents are stochastic by nature, so unless you fix the random see generator, you will see different results
7 Commenti
Emmanouil Tzorakoleftherakis
il 21 Mar 2023
Modificato: Emmanouil Tzorakoleftherakis
il 21 Mar 2023
Don't have any other ideas, maybe technical support can help if you can share a reproduction model.
Vedere anche
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!