Azzera filtri
Azzera filtri

Why clearing memmapfile is taking too long?

1 visualizzazione (ultimi 30 giorni)
yash
yash il 15 Apr 2024
Risposto: Hassaan il 15 Apr 2024
Hi team,
I am trying to use fileDataStore with custom read function. In this custom read function I am using memmapfile to read data from file. In this function I am clearing memmapfile object. It is taking approximately 3 seconds to get cleared. So, my questions are:
1) Why it is taking too long to clear memmapfile?
2) Is there any other way to clear memmapfile?
Following is attached snippets for the code.
[filename,filepath] = uigetfile('*.*', 'Select a file');
readTime = ReadDataStoreMapTime(fullfile(filepath, filename));
function time = ReadDataStoreMapTime(filename)
tic;
chunksize = 2.5*1024*1024*1024;
ds = fileDatastore(filename, 'ReadFcn', @readBinaryTRCFile, ...
'FileExtensions', '.trc', 'ReadMode', 'bytes');
ds.BlockSize = chunksize;
while hasdata(ds)
dataChunk = read(ds);
end
time = toc;
end
function data = readBinaryTRCFile(filename, offset, size)
tic;
size = floor(size / 8);
file = memmapfile(filename, ...
'Offset', offset, ...
'Writable', true, ...
'Format', 'double', ...
'Repeat', size);
data = file.Data;
clear file;
disp(toc);
end

Risposte (1)

Hassaan
Hassaan il 15 Apr 2024
function time = ReadDataStoreMapTime(filename)
tic;
chunksize = 2.5*1024*1024*1024;
ds = fileDatastore(filename, 'ReadFcn', @readBinaryTRCFile, ...
'FileExtensions', '.trc', 'ReadMode', 'bytes');
ds.BlockSize = chunksize;
while hasdata(ds)
dataChunk = read(ds);
end
time = toc;
end
function data = readBinaryTRCFile(filename, offset, size)
tic;
size = floor(size / 8);
file = memmapfile(filename, ...
'Offset', offset, ...
'Writable', true, ...
'Format', 'double', ...
'Repeat', size);
data = file.Data;
disp(toc); % The memmapfile object 'file' will be cleared automatically when it goes out of scope
end
By relying on MATLAB's automatic cleanup when file goes out of scope, this might slightly improve the performance or at least the predictability of memory management. If these steps don't help, consider profiling the code with MATLAB's profiler to identify any other bottlenecks or issues.

Categorie

Scopri di più su Parallel Computing in Help Center e File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by