Issue with parfor when using containers.Map

2 visualizzazioni (ultimi 30 giorni)
Shlomo Geva
Shlomo Geva il 28 Mag 2023
Commentato: Shlomo Geva il 28 Mag 2023
Here is the code of a script that calls a function inside a loop. The function is passed a Map object (created previously). I entered in the comment the details of the Map. The function actually does nothing with the map. I run this code with a for loop and it takes little time of course:
"Elapsed time is 0.000417 seconds."
When I change the for to a parfor and run again - it goes into a black hole. I have to kill it. In the Activity Monitor (on a Mac) I can see all 8 processors are 100% busy.
I assume it may have to do with the distribution of a large map, but I tried it also on a server with 1.5TB of memory and it hangs there too.
Any suggestions or explanantions will be greatly appreciated.
Perhaps there is a better way to distribute the map as a shared variable (it is not modified inside the loop)
Here is the script:
% >> myMap
%
% myMap =
%
% Map with properties:
%
% Count: 10344256
% KeyType: char
% ValueType: any
%
% >>
tic
for i=1:8 % if this for loop is cahanged to parfor - it hangs
[x,y] = foobar(myMap,i);
end
toc;
function [x,y] = foobar(myMap,i)
x=[];
y=[];
end
  4 Commenti
Walter Roberson
Walter Roberson il 28 Mag 2023
Yes. parfor() and parfeval() do not use shared memory (or at least are not documented as doing so).
In a discussion within the last few days I showed from the R2023a documentation that as of R2023a it cannot be the case that parpool constants are "broadcast" to all workers in a single packet. I see you are using R2022b, at which time such a "broadcast" was at least a hypothetical possibility.
You might want to experiment with using backgroundPool and parfeval as background pools can use shared memory and are designed to reduce the amount of memory copying required.
Shlomo Geva
Shlomo Geva il 28 Mag 2023
Will do. Thanks for your time.

Accedi per commentare.

Risposte (0)

Categorie

Scopri di più su Parallel for-Loops (parfor) in Help Center e File Exchange

Prodotti


Release

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by