Most efficient way to read mdf file

Hello,

I have list of signals. And I need to extract information from all channels whose name (partially) matches with the list of signals. Currently I do it

mdfObj = mdf(inputFile);
signals = ['time', 'position','velocity']

for idx = 1: length(signals)      
    chanList = channelList(mdfObj, signals(idx));  
    data = read(mdfObj, chanList);
    % Process data
end

However, reading still takes lot of time and I want to know if it can be optimized further. If I read whole mdfObj in one go,

data = read(mdfObj);

the program runs 4x faster. But, memory usage would be more.

Can I read maybe 100mbs of data sequentially?

[Benjamin Thompson]

Can you post a sample MDF file?

[Monika K]

The MDF file is of Gbs and has various channels for e.g.

1. time.sec
2. time.nanosec
3. speed.x
4. speed.y

Particular signal e.g. time related belong to same Channel group number. I hope this helps as I am not sure how can I post sample MDF file.

Answer 1

[Aravind]

Hi @Monika K,

If you're looking to optimize the reading of a very large MDF file, you can do so by processing the data in chunks. Here’s a step-by-step approach to achieve this:

1. Identify Total Samples: Use the “numel” function to determine the total number of samples in the MDF file. You can find more information regarding this at https://www.mathworks.com/help/releases/R2022a/matlab/ref/double.numel.html
2. Define Chunk Size: Decide on a chunk size, which is the number of samples you want to read at a time. This size should be based on your memory constraints and performance needs. You can choose the chunk size based on the amount of continuous data you wish to read.
3. Loop Through Samples: Implement a loop to read the data in chunks. Within this loop, specify the range of samples to read in each iteration, ensuring you only handle a manageable amount of data at a time.
4. Read and Process Data: For each chunk, use the “channelList” function to obtain the list of channels that match your signals. Then, use the “read” function to read the specified range of samples for these channels. You can refer to this example of the “read” function for reading a range of data from specified time range values: https://www.mathworks.com/help/releases/R2022a/vnt/ug/read.html#bvcxtkn-1. You can then process the data as needed within the loop.

I hope this helps!