Policy

Reinforcement learning policy

Since R2022b

• 

×

Libraries:
Reinforcement Learning Toolbox

Description

Use the Policy block to simulate a reinforcement learning policy in Simulink^® and to generate code (using Simulink Coder™) for deployment purposes. This block takes an observation as input and outputs an action. You associate the block with a MAT-file that contains the information needed to fully characterize the policy, and which can be generated by generatePolicyFunction or generatePolicyBlock.

Examples

Generate Policy Block for Deployment

Generate a policy block to deploy a trained policy.

Open Live Script

Ports

Input

observation — Environment observations

scalar | vector | nonvirtual bus

This port receives observation signals from the environment. Observation signals represent measurements or other instantaneous system data. If you have multiple observations, you can use a Mux block to combine them into a vector signal. To use a nonvirtual bus signal, use bus2RLSpec.

Output

action — Policy action

scalar | vector | nonvirtual bus

Action computed by the policy based on the observation input. Connect this port to the inputs of your system. To use a nonvirtual bus signal, use bus2RLSpec.

Note

Policy blocks generated from a continuous action-space rlStochasticActorPolicy object or a continuous action-space rlACAgent, rlPGAgent, or rlPPOAgent object, do not enforce the constraints set by the action specification. In these cases, you must enforce action space constraints within the environment.

Parameters

Policy block data MAT-file — Policy block data MAT-file

blockAgentData.mat (default) | file name

Enter the name of the MAT-file containing the information needed to fully characterize the policy. This file is generated by generatePolicyFunction or generatePolicyBlock. When you generate the block using generatePolicyBlock and specify a non-default dataFileName argument, then the generated block has this parameter set to the specified file name, so that the block is associated with the generated data file.

To use a Policy block within a conditionally executed subsystem, such as a Triggered Subsystem (Simulink) or a Function-Call Subsystem (Simulink), you must generate its data file from an agent or policy object which has its SampleTime property set to -1. Doing so allows the block to inherit the sample time of its parent subsystem.

Programmatic Use

Block Parameter: MATFile

Type: string, character vector

Default: "blockAgentData.mat"

Tips

• When using Embedded Coder^® to generate parallel code, enabling the Generate parallel for loops optimization parameter improves the performance when the data being processed is large in size. However, if the network and the data is small, the overhead of initializing the threads for parallelization significantly reduces the performance. In this case, disable Generate parallel for loops. See Generate parallel for-loops (Embedded Coder) and coder.MexCodeConfig (MATLAB Coder) for more information.

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using Simulink® Coder™.

Version History

Introduced in R2022b