rlSimulationOptions

Options for simulating reinforcement learning environments

Description

example

simOpts = rlSimulationOptions returns the default options for simulating a reinforcement learning environment against an agent. You use simulation options to specify parameters about the simulation such as the maximum number of steps to run per simulation and the number of simulations to run. After you configure the options, use simOpts as an input argument for sim.

example

simOpts = rlSimulationOptions(Name,Value) creates an option set for simulation using the specified name-value pairs to override default option values.

Examples

collapse all

Create an options set for simulating a reinforcement learning environment. Set the number of steps to simulate to 1000, and configure the options to run three simulations.

You can set the options using Name,Value pairs when you create the options set. Any options that you do not explicitly set have their default values.

simOpts = rlSimulationOptions(...
    'MaxSteps',1000,...
    'NumSimulations',3)
simOpts = 
  rlSimulationOptions with properties:

                  MaxSteps: 1000
            NumSimulations: 3
               StopOnError: "on"
               UseParallel: 0
    ParallelizationOptions: [1x1 rl.option.ParallelSimulation]

Alternatively, create a default options set and use dot notation to change some of the values.

simOpts = rlSimulationOptions;
simOpts.MaxSteps = 1000;
simOpts.NumSimulations = 3;

simOpts
simOpts = 
  rlSimulationOptions with properties:

                  MaxSteps: 1000
            NumSimulations: 3
               StopOnError: "on"
               UseParallel: 0
    ParallelizationOptions: [1x1 rl.option.ParallelSimulation]

Input Arguments

collapse all

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: 'MaxSteps',1000,'StopOnError',"On"

Number of steps to run the simulation, specified as the comma-separated pair consisting of 'MaxSteps' and a positive integer. In general, you define episode termination conditions in the environment. This value is the maximum number of steps to run in the simulation if those termination conditions are not met.

Example: 'MaxSteps',1000

Number of simulations to run, specified as the comma-separated pair consisting of 'NumSimulations' and a positive integer. At the start of each simulation, sim resets the environment. You specify what happens on environment reset when you create the environment. For instance, resetting the environment at the start of each episode can include randomizing initial state values, if you configure your environment to do so. In that case, running multiple simulations allows you to validate performance of a trained agent over a range of initial conditions.

Example: 'NumSimulations',10

Stop simulation when an error occurs, specified as "off" or "on". When this option is "off", errors are captured and returned in the SimulationInfo output of sim, and simulation continues.

Flag for using parallel simulation, specified as the comma-separated pair consisting of 'UseParallel' and either true or false. Setting this option to true configures simulation to use parallel computing. To specify options for parallel simulation, use the ParallelizationOptions property.

Using parallel computing requires Parallel Computing Toolbox™ software.

For more information about training using parallel computing, see Train Reinforcement Learning Agents.

Example: 'UseParallel',true

Parallelization options to control parallel simulation, specified as the comma-separated pair consisting of 'ParallelizationOptions' and a ParallelTraining object. For more information about training using parallel computing, see Train Reinforcement Learning Agents.

The ParallelTraining object has the following properties, which you can modify using dot notation after creating the rlTrainingOptions object.

Randomizer initialization for workers, specified as one the following:

  • –1 — Assign a unique random seed to each worker. The value of the seed is the worker ID.

  • –2 — Do not assign a random seed to the workers.

  • Vector — Manually specify the random seed for each work. The number of elements in the vector must match the number of workers.

Send model and workspace variables to parallel workers, specified as "on" or "off". When the option is "on", the host sends variables used in models and defined in the base MATLAB® workspace to the workers.

Additional files to attach to the parallel pool, specified as a string or string array.

Function to run before simulation starts, specified as a handle to a function having no input arguments. This function is run once per worker before simulation begins. Write this function to perform any processing that you need prior to simulation.

Function to run after simulation ends, specified as a handle to a function having no input arguments. You can write this function to clean up the workspace or perform other processing after simulation terminates.

Output Arguments

collapse all

Option set for simulating reinforcement learning environments, returned as a rlSimulationOptions object. The property values of simOpts are initialized to the default values or to the values you specify with Name,Value pairs. You can further modify the property values using dot notation. Use the options set as an input argument with sim when you simulate reinforcement learning environment against an agent.

See Also

Introduced in R2019a