Why clearing memmapfile is taking too long?

Hi team,

I am trying to use fileDataStore with custom read function. In this custom read function I am using memmapfile to read data from file. In this function I am clearing memmapfile object. It is taking approximately 3 seconds to get cleared. So, my questions are:

1) Why it is taking too long to clear memmapfile?

2) Is there any other way to clear memmapfile?

Following is attached snippets for the code.

[filename,filepath] = uigetfile('*.*', 'Select a file');

readTime = ReadDataStoreMapTime(fullfile(filepath, filename));

function time = ReadDataStoreMapTime(filename)

tic;

chunksize = 2.5*1024*1024*1024;

ds = fileDatastore(filename, 'ReadFcn', @readBinaryTRCFile, ...

'FileExtensions', '.trc', 'ReadMode', 'bytes');

ds.BlockSize = chunksize;

while hasdata(ds)

dataChunk = read(ds);

end

time = toc;

end

function data = readBinaryTRCFile(filename, offset, size)

tic;

size = floor(size / 8);

file = memmapfile(filename, ...

'Offset', offset, ...

'Writable', true, ...

'Format', 'double', ...

'Repeat', size);

data = file.Data;

clear file;

disp(toc);

end

Answer 1

[Hassaan]

function time = ReadDataStoreMapTime(filename)
    tic;
    chunksize = 2.5*1024*1024*1024;
    ds = fileDatastore(filename, 'ReadFcn', @readBinaryTRCFile, ...
        'FileExtensions', '.trc', 'ReadMode', 'bytes');
    ds.BlockSize = chunksize;
    while hasdata(ds)
        dataChunk = read(ds);
    end
    time = toc;
end

function data = readBinaryTRCFile(filename, offset, size)
    tic;
    size = floor(size / 8);

    file = memmapfile(filename, ...
        'Offset', offset, ...
        'Writable', true, ...
        'Format', 'double', ...
        'Repeat', size);

    data = file.Data;
    disp(toc); % The memmapfile object 'file' will be cleared automatically when it goes out of scope
end

By relying on MATLAB's automatic cleanup when file goes out of scope, this might slightly improve the performance or at least the predictability of memory management. If these steps don't help, consider profiling the code with MATLAB's profiler to identify any other bottlenecks or issues.