How can I read an image file from disk asynchronously?
5 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
Robin Atkins
il 10 Ott 2013
Modificato: Robin Atkins
il 8 Set 2021
I am working on a large number of 12MB image files. The time required to process each image file is split almost equally between Reading the input, Processing, and Writing the output. Reading and Writing speed is limited by disk, whereas Processing speed is limited by CPU/GPU. I am reading and writing to/from different disks.
When using imread and imwrite I have to wait for the function to complete before processing each image, so the disk or CPU/GPU is sitting idle ~2/3 of the time. Is there some way to call imread and imwrite asynchronously in order to accelerate the script?
Output from Example code:
>> readExample
Single worker:
Reading: 0.13
Processing: 1.10
Writing: 6.99
Total: 8.22
Three workers:
Reading: 0.17
Processing: 13.70
Writing: 0.60
Total: 8.23
Example Code:
% Make an example image
imwrite(uint16(rand(1080,1920,3)*65535),'example.tiff');
% Make an example convolution filter
X = gpuArray.colon(-300,300);
S = 100;
H = exp(-(X.*X)/(2*S*S));
H = H'*H;
H = H/sum(H(:));
% Go through the loop with a single worker
N=2;
Tread = zeros(N,1); Tprocess=zeros(N,1); Twrite=zeros(N,1);
Ttotal = tic;
for n=1:N
% Load the file
tic;
Im = single(imread('example.tiff'));
Im = gpuArray(Im);
Tread(n) = toc;
% Apply the convolution
tic;
Im(:,:,1) = conv2(Im(:,:,1), H, 'same');
Tprocess(n) = toc;
% Save the result
tic;
Im = gather(Im);
imwrite(uint16(Im),'output.tiff');
Twrite(n) = toc;
end
fprintf('Single worker:\n');
fprintf('Reading:\t\t%0.02f\n',sum(Tread));
fprintf('Processing:\t\t%0.02f\n',sum(Tprocess));
fprintf('Writing:\t\t%0.02f\n',sum(Twrite));
fprintf('Total:\t\t\t%0.02f\n',toc(Ttotal));
% Go through the loop again with three workers
if matlabpool('size') < 3
matlabpool('open',3);
end
Tread = zeros(N,1); Tprocess=zeros(N,1); Twrite=zeros(N,1);
Ttotal = tic;
parfor n=1:N
% Load the file
tic;
Im = single(imread('example.tiff'));
Im = gpuArray(Im);
Tread(n) = toc;
% Apply the convolution
tic;
Im(:,:,1) = conv2(Im(:,:,1), H, 'same');
Tprocess(n) = toc;
% Save the result
tic;
Im = gather(Im);
imwrite(uint16(Im),'output.tiff');
Twrite(n) = toc;
end
fprintf('\nThree workers:\n');
fprintf('Reading:\t\t%0.02f\n',sum(Tread));
fprintf('Processing:\t\t%0.02f\n',sum(Tprocess));
fprintf('Writing:\t\t%0.02f\n',sum(Twrite));
fprintf('Total:\t\t\t%0.02f\n',toc(Ttotal));
2 Commenti
Ashish Uthama
il 14 Ott 2013
Could you update the question with some sample code so we understand the pattern? If you are processing files in a loop, subsequent imread ought to happen while the GPU is processing, most (if not all) GPU calls are async.
Risposta accettata
Più risposte (2)
Image Analyst
il 11 Ott 2013
Modificato: Image Analyst
il 11 Ott 2013
Perhaps get the Parallel Computing Toolbox?
2 Commenti
Image Analyst
il 11 Ott 2013
I think it probably already does things as fast as it can. I doubt they programmed in delays on purpose.
You can set the CPU priority higher. Right click MATLAB in the process list (Windows OS, control-shift-Escape) and do Set Priority->High. DO NOT USE REALTIME or you won't be able to do anything else.
Or try using a solid state drive to speed things up.
Jan
il 11 Ott 2013
Modificato: Jan
il 11 Ott 2013
A less compact idea than the Parallel Computing Toolbox is starting three Matlab session and let them communicate through http://www.mathworks.com/matlabcentral/fileexchange/28572-sharedmatrix . But then the control of the inter-process communication is not trivial. But you would not transport the image data through the SharedMatrix, but control, which session processes which file.
Which kind of file type do the images have? Most of the formats are read using standard libraries, such that a C-Mex function could read and write the file asynchronously. You can easily start two threads for this job as long as they do not call any Matlab commands (mx... and mex... functions).
Vedere anche
Categorie
Scopri di più su Parallel and Cloud in Help Center e File Exchange
Prodotti
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!