Using parfor in a sparse setting

3 visualizzazioni (ultimi 30 giorni)
Jakob Sievers
Jakob Sievers il 23 Nov 2019
Commentato: Jakob Sievers il 23 Nov 2019
Hi there
I have been solving a problem using a parfor loop. However it has come to my attention that occasionally a large number of serial entries do not in fact have to be run, and I suspect that it reduces the advantage of the parfor loop, relative to a normal for-loop, whenever it has to deal with lots of "non-entries". As an example here is an illustration of a hypothetical situation, and process, which needs to be solved.
A=[1;2;3;4;nan;nan;nan;nan;nan;10;11];
B=nan(size(A));
parfor ii=1:length(A)
if ~isnan(A(ii))
B(ii)=A(ii); %process
end
end
As you can see, 5 out of the 11 entries do not need to be run and so the parfor skips those accordingly (for the actual case the ratio is often more like 1/10 which is why I am exploring alternative solutions). However I wanted to explore a quick solution which would make the parfor loop inherently only evaluate real entries. My problematic code is shown here:
A=[1;2;3;4;nan;nan;nan;nan;nan;10;11];
B=nan(size(A));
ix=find(~isnan(A));
parfor ii2=1:length(ix)
ii=ix(ii2);
B(ii)=A(ii); %process
end
Matlab doesn't allow this. Veterans will likely look at this and go "obviously this is not possible" but I don't understand why, and more importantly, I don't understand how I might otherwise make a similar simple workaround (i.e.: not rebuild the entire actual process within the parfor loop) that would only evaluate the right numbers.
Can anyone help me out?
Thanks in advance!
Cheers
Jakob
  3 Commenti
Jakob Sievers
Jakob Sievers il 23 Nov 2019
If you copy the second code snippet into a script you'll see that Matlab throws the following error: "The PARFOR loop cannot run due to the way variable 'B' is used"
and in the details it elaborates:
Explanation
MATLAB runs loops in parfor functions by dividing the loop iterations into groups, and then sending them to MATLAB workers where they run in parallel. For MATLAB to do this in a repeatable, reliable manner, it must be able to classify all the variables used in the loop. The code uses the indicated variable in a way that is incompatible with classification.
Suggested Action
Fix the usage of the indicated variable.
For more information about variable classification and other restrictions on parfor loop iterations, see in the Parallel Computing Toolbox documentation
Jan
Jan il 23 Nov 2019
@Jakob: Thanks for sharing the error message. I cannot copy&paste&run the code without having the Parallel Computing Toolbox installed at home.

Accedi per commentare.

Risposta accettata

Jan
Jan il 23 Nov 2019
Modificato: Jan il 23 Nov 2019
The help page "Troubleshoot Variables in parfor-Loops" explains the problem:
A = [1;2;3;4;nan;nan;nan;nan;nan;10;11];
B = nan(size(A));
ix = find(~isnan(A));
parfor ii2 = 1:length(ix)
ii = ix(ii2);
B(ii) = A(ii); % Problem: Usage of B not clear
end
Matlab cannot know if ii is unique, so it is possible that 2 workers access B(1) at the same time.
Solution:
A = [1;2;3;4;nan;nan;nan;nan;nan;10;11];
B = nan(size(A));
ix = find(~isnan(A));
BB = nan(numel(ix), 1);
parfor ii2 = 1:numel(ix)
ii = ix(ii2);
BB(ii2) = A(ii); % No problem: no collisions possible
end
B(ix) = BB
  1 Commento
Jakob Sievers
Jakob Sievers il 23 Nov 2019
Ah ok, that makes sense. And the solution seems very reasonable. I will try this out. Thank you for your help!

Accedi per commentare.

Più risposte (0)

Categorie

Scopri di più su Parallel for-Loops (parfor) in Help Center e File Exchange

Prodotti


Release

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by