Reinforcement Learning Toolbox Customize DQN Agent

In reinforcement learning toolbox, is it possible to customize the way that DQN Agent takes actions? For example in a snake game if the last action taken was "North" then agent should not even bother to try the action "South" and vice versa. Should I do that with Custom Agent template from scratch like here or is there any better way to do this ? For example only customizing rlDQNAgent's takeAction method instead of implementing all other things from scratch.

Answer 1

[Ryan Comeau]

Hello,

I can see the value in what you're proposing. In some cases it would be useful to have RL agents not fail the game immediately for simple mistakes like this. That is not however the purpose of the RL agent. Failing is an important step in the learning process. What you can do however, is modify the reward function to make some failures worse than others, which will heavily discourage the rl agents from making these mistakes.

So in the snake example, hitting a wall could be a -10 reward, hitting the extended body could be -20 and hitting itself after a North than South type could be -100. This would rapidly teach the agent to not do that. You can open up the class of environment in question and look for the function called step(). This function shapes the reward and takes the action. Alter the reward function to your liking.

Hope this helps,

RC