Hello Lin,
The inconsistency between the simulation time set by reinforcement learning and the training time is expected due to the different and independent nature of these parameters. Here's an explanation to clarify the difference:
- Simulation Time is the time required to run the simulation, where each episode represents an interaction between the Reinforcement Learning Agent and the Environment.
- Training Time is the time taken to for the model parameters to converge to the optimal parameters.
The simulation time and training time are governed by different parameters and are inherently independent of each other. The simulation time depends on the number of episodes and the duration of each episode, whereas the training time depends on the number of epochs and the computation time required to update the model parameters.
Refer to the following MathWorks Documentation to understand more about Training RL Agents and training options.
- Train RL Agents: https://www.mathworks.com/help/reinforcement-learning/ug/train-reinforcement-learning-agents.html
- Agent Options: https://www.mathworks.com/help/reinforcement-learning/ref/rl.option.rltd3agentoptions.html
- Training and Simulation: https://www.mathworks.com/help/reinforcement-learning/training-and-simulation.html
Hope the above information is helpful.
