validateEnvironment

Validate custom reinforcement learning environment

Syntax

validateEnvironment(env)

Description

validateEnvironment(env) validates a reinforcement learning environment. This function is useful when:

• You create a custom environment from a template class using rlCreateEnvTemplate.

• You create a custom Simulink^® environment using rlSimulinkEnv or createIntegratedEnv.

When you create a custom environment using rlFunctionEnv or rlNeuralNetworkEnvironment, the software runs validateEnvironment automatically after creating the environment object.

validateEnvironment resets the environment, generates an initial observation and action, and simulates the environment for one or two steps (see Algorithms). If there are no errors during these operations, validation is successful, and validateEnvironment returns no result. If errors occur, these errors appear in the MATLAB^® command window. Use the errors to determine what to change in your observation specification, action specification, custom functions, or Simulink model.

Examples

Validate Simulink Environment

This example shows how to validate a Simulink environment.

Create and validate and environment for the rlwatertank model, which represents a control system containing a reinforcement learning agent (For details about this model, see Control Water Level in a Tank Using a DDPG Agent.)

open_system("rlwatertank")

Create observation and action specifications for the environment.

obsInfo = rlNumericSpec([3 1],...
    LowerLimit=[-inf -inf 0  ]',...
    UpperLimit=[ inf  inf inf]');
obsInfo.Name = "observations";
obsInfo.Description = 'integrated error, error, and measured height';
numObservations = obsInfo.Dimension(1);

actInfo = rlNumericSpec([1 1]);
actInfo.Name = "flow";
numActions = numel(actInfo);

Create an environment from the model.

env = rlSimulinkEnv( ...
    "rlwatertank", ...
    "rlwatertank/RL Agent", ...
    obsInfo,actInfo);

Now you use validateEnvironment to check whether the model is configured correctly.

validateEnvironment(env)

Error using rl.env.SimulinkEnvWithAgent/validateEnvironment (line 187) 
Simulink environment validation requires an agent in the MATLAB base workspace 
or in a data dictionary linked to the model. Specify the agent in the Simulink model. 

validateEnvironment attempts to compile the model, initialize the environment and the agent, and simulate the model. In this case, the RL Agent block is configured to use an agent called agent, but no such variable exists in the MATLAB® workspace. Thus, the function returns an error indicating the problem.

Create an appropriate agent for this system using the commands detailed in the Control Water Level in a Tank Using a DDPG Agent example. In this case, load the agent from the rlWaterTankDDPGAgent.mat file.

load rlWaterTankDDPGAgent

Now, run validateEnvironment again.

validateEnvironment(env)

Input Arguments

env — Environment to validate

environment object

Environment to validate, specified as a reinforcement learning environment object, such as:

• A custom MATLAB environment you create with rlCreateEnvTemplate. In this case, validateEnvironment checks that the observations and actions generated during simulation of the environment are consistent in size, data type, and value range with the observation specification and action specification. It also checks that your custom step and reset functions run without error. When you create a custom environment using rlFunctionEnv or rlNeuralNetworkEnvironment, the software runs validateEnvironment automatically before returning the environment object.

• A custom Simulink environment you create using rlSimulinkEnv. If you use a Simulink environment, you must also have an agent defined and associated with the RL Agent block in the model. For a Simulink model, validateEnvironment checks that the model compiles and runs without error. The function does not dirty your model.

For more information about creating and configuring environments, see:

• Reinforcement Learning Environments

• Create Custom Simulink Environments

Algorithms

validateEnvironment works by running a brief simulation of the environment and making sure that the generated signals match the observation and action specifications you provided when you created the environment.

MATLAB Environments

For MATLAB environments, validation includes the following steps.

1. Reset the environment using the reset function associated with the environment.

2. Obtain the first observation and check whether it is consistent with the dimension, data type, and range of values in the observation specification.

3. Generate a test action based on the dimension, data type, and range of values in the action specification.

4. Simulate the environment for one step using the generated action and the step function associated with the environment.

5. Obtain the new observation signal and check whether it is consistent with the dimension, data type, and range of values in the observation specification.

If any of these operations generates an error, validateEnvironment returns the error. If validateEnvironment returns no result, then validation is successful.

Simulink Environments

For Simulink environments, validation includes the following steps.

1. Reset the environment.

2. Simulate the model for two time steps.

If any of these operations generates an error, validateEnvironment returns the error. If validateEnvironment returns no result, then validation is successful.

validateEnvironment performs these steps without dirtying the model, and leaves all model parameters in the state they were in when you called the function.

Version History

Introduced in R2019a