Saving simulation data during training process of RL agents

Hello everyone

Using the Logging tab of the scope blocks inside the Simulink, I managed to save the data to the MATLAB workspace and then save them to a text file. I need to do the same thing for every iteration (episode) during an RL agent's training process. I wonder how I can similarly save the log data for further analysis and visualization.

Answer 1

[Ari Biswas]

Elaborating on Emmanouil's suggestion:

There are two ways to log and visualize data during training.

Option 1 is to use the training statistics structure (output of the train function). You must enable the Signal Logging checkbox under Model Settings > Data Import/Export. Then specify the signals to log in the Simulink model. This can be easily done by right clicking a signal and selecting Log Selected Signals. After the training is complete you can access this data from the SimulationInfo field in the training statistics struct.

Option 2 is to use the Simulation Data Inspector (SDI). With this option you can visualize the data as the simulations are completed during training. You would need to enable the Record logged workspace data in SDI option under Model Settings > Data Import/Export in the Simulink model, then specify all signals to log in the model. If training in parallel, execute the following command prior to starting training for making the SDI compatible with parallel simulations. Parallel workers will send logged data to the SDI as the simulations are completed.

Simulink.sdi.enablePCTSupport('local') % if the parallel workers are from the local cluster

The SDI also has options to export logged data to the MATLAB workspace.

[awcii]

how can find the SimulationInfo ? Should it be in workspace ?

Answer 2

[Emmanouil Tzorakoleftherakis]

Have you tried logging the data with Simulation Data Inspector? Make sure to pick only signals you actually need since depending on duration of training, you might end up saving a lot of data.

[Nima Mathwork]

Thank you @Emmanouil Tzorakoleftherakis for your answer. I think using Simulation Data Inspector is a proper solution. However, I could not find an appropriate example in the documentation. Now, I managed to use the cell array, which is suggested here. I would be grateful if you can indicate an example for using Simulation Data Inspector. As a reminder, I need the raw data to draw my custom plots.

[Emmanouil Tzorakoleftherakis]

Maybe this page and link included are helpful. I don't think there is an example specifically for reinforcement learning in the documentation.