![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/177275/image.png)
Why does parallel.pool.const create a copy of the variable in memory for each worker sequentially instead of in parallel?
3 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
Joseph Hall
il 29 Giu 2017
Commentato: Joseph Hall
il 30 Giu 2017
When creating a parallel.pool.const on 9 workers prior to using parfor, I noticed that the memory usage ramps up in 9 successive steps instead of all at once. The attached image shows these steps in memory usage prior to entering the parfor using 'Resource Monitor' on Windows 7. This seems to mean that the copies for each worker are created sequentially instead of in parallel and this takes alot of time. Why are these copies not created in parallel for faster execution? I am running R2017a.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/165635/image.jpeg)
0 Commenti
Risposta accettata
Edric Ellis
il 30 Giu 2017
I suspect you're creating the parallel.pool.Constant using data created on the client. It's much more efficient to have the workers create the data, if possible. Consider two cases:
% Case 1: data created on the client
parallel.pool.Constant(ones(1e4));
% Case 2: use the Constant constructor with a function handle to create
% the contents directly on the worker
parallel.pool.Constant(@() ones(1e4));
This results in the following memory usage pattern. In the screen-shot, case 1 is indicated with a red arrow, and case 2 with a green arrow.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/177275/image.png)
As you can see, case 2 happens in parallel, and avoids the data transfer from the client to the workers (it's the data transfer that really causes the lack of parallelism).
If you really cannot create the data on the workers, you can use the parallel.pool.Constant constructor that accepts a Composite, like this:
% Build an empty Composite
c = Composite();
% Transfer the data from client only to worker 1
c{1} = ones(1e4);
c(2:end) = {[]};
spmd
% Use labBroadcast to copy data to all workers (labBroadcast
% is more efficient than the client/worker communication)
c = labBroadcast(1, c);
end
% Build the Constant from the Composite
c = parallel.pool.Constant(c);
% Flush memory on the workers by executing an empty SPMD block
spmd, end
Più risposte (0)
Vedere anche
Categorie
Scopri di più su Programming Utilities in Help Center e File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!