rlTable

Value table or Q table

Description

Value tables and Q tables are one way to represent critic networks for reinforcement learning. Value tables store rewards for a finite set of observations. Q tables store rewards for corresponding finite observation-action pairs.

To create a value function approximator using an rlTable object, use an rlValueFunction, rlQValueFunction, or rlVectorQValueFunction object.

Creation

Syntax

T = rlTable(obsinfo)

T = rlTable(obsinfo,actinfo)

Description

T = rlTable(obsinfo) creates a value table for the given discrete observations.

T = rlTable(obsinfo,actinfo) creates a Q table for the given discrete observations and actions.

Input Arguments

obsinfo — Observation specification

rlFiniteSetSpec object

Observation specification, specified as an rlFiniteSetSpec object.

actinfo — Action specification

rlFiniteSetSpec object

Action specification, specified as an rlFiniteSetSpec object.

Properties

Table — Reward table

array

Reward table, returned as an array. When Table is a:

• Value table, it contains N_O rows, where N_O is the number of finite observation values.

• Q table, it contains N_O rows and N_A columns, where N_A is the number of possible finite actions.

Object Functions

rlValueFunction	Value function approximator object for reinforcement learning agents
rlQValueFunction	Q-Value function approximator object for reinforcement learning agents
rlVectorQValueFunction	Vector Q-Value function approximator for reinforcement learning agents

Examples

Create a Value Table

This example shows how to use rlTable to create a value table. You can use such a table to represent the critic of an actor-critic agent with a finite observation space.

Create an environment interface, and obtain its observation specifications.

env = rlPredefinedEnv("BasicGridWorld");
obsInfo = getObservationInfo(env)

obsInfo = 
  rlFiniteSetSpec with properties:

       Elements: [25x1 double]
           Name: "MDP Observations"
    Description: [0x0 string]
      Dimension: [1 1]
       DataType: "double"

Create the value table using the observation specification.

vTable = rlTable(obsInfo)

vTable = 
  rlTable with properties:

    Table: [25x1 double]

Create a Q Table

This example shows how to use rlTable to create a Q table. Such a table could be used to represent the actor or critic of an agent with finite observation and action spaces.

Create an environment interface, and obtain its observation and action specifications.

env=rlMDPEnv(createMDP(8,["up";"down"]));
obsInfo = getObservationInfo(env)

obsInfo = 
  rlFiniteSetSpec with properties:

       Elements: [8x1 double]
           Name: "MDP Observations"
    Description: [0x0 string]
      Dimension: [1 1]
       DataType: "double"

actInfo = getActionInfo(env)

actInfo = 
  rlFiniteSetSpec with properties:

       Elements: [2x1 double]
           Name: "MDP Actions"
    Description: [0x0 string]
      Dimension: [1 1]
       DataType: "double"

Create the Q table using the observation and action specifications.

qTable = rlTable(obsInfo,actInfo)

qTable = 
  rlTable with properties:

    Table: [8x2 double]

Version History

Introduced in R2019a