Export Data to Parquet File Using Record Block

When you log data using a Record block, you can chose to log data to the workspace, a file, or both. One file type that you can record data to is a Parquet file. Parquet is an open-source file format with efficient compression and encoding of column-oriented data that is often used when processing big data.

Use the Record block to log scalar and multidimensional data from a signal, message, bus, or array of buses to a Parquet file. The Record block also supports logging variable-size signals, but does not support visualizing or logging variable-size signals to a Parquet file. Use the Record block to log real and complex data of any built-in data type or user-defined data types such as buses, enumerations, and fixed-point data. For more information about how the Record block stores data in the Parquet file, see Parquet File Logging Format for Simulation Data.

The Record block can log data to a Parquet file during normal, accelerator, and rapid accelerator mode simulations. All signals connected to the Record block are written to the Parquet file when the simulation is paused or stopped. By default, when you record data to a Parquet file, the Record block stores all logged metadata associated with the Simulink^® simulation in a JSON sidecar. The sidecar contains information such as the block path, port index, and sample time.

Log Data to Parquet File Interactively

To log data to a Parquet file, double-click the Record block. Click the Record To File button arrow. From this menu, select Record to File and set the file type to *.parquet. By default, the filename is recording.parquet, and the file is saved in the local folder.

To change other options specific to Parquet files, such as the compression style, time column configuration, and row group options, select Parquet File Options. For example, by default, the Record block saves data to a Parquet file using shared time columns, which groups data to the right of a shared time vector. To instead save data to a Parquet file in pairs of time and signal data columns using individual time columns, select Record To File > Parquet File Options > Individual Time Columns.

Log Data to Parquet File Programmatically

You can also configure the Record block to log data to a Parquet file programmatically using the set_param function. First, set the Record block to record to a file. Then, specify the name of the Parquet file.

set_param("MyModel/Record","RecordToFile","on","FileName","myRecording.parquet")

Once the Record block is configured to record data to a Parquet file, you can also use the set_param function to set other Parquet file options, such as the compression style.

set_param("MyModel/Record","ParquetCompression","compact")