Slice called broadcast variable in parfor

5 visualizzazioni (ultimi 30 giorni)
Mariella Dreißig
Mariella Dreißig il 17 Dic 2019
I have a code I want to parallelize, something like:
parfor i = 1:length(all_idx(:,1))
combis = nchoosek(1:k,2);
results = zeros(length(combis(:,1)),1);
for j = 1:length(combis(:,1))
results(j) = fun(X(:,all_idx(i,combis(j,1))), X(:,all_idx(i,combis(j,2))));
end
...
end
X and all_idx are broadcast variables, especially X is a huge matrix and produces a significant overhead. I read something about slicing the variables, the computation results are strored in (variables in which I write), but is there a way to somehow slice the variables which are read within the loop? As far as I understand, the whole matrices X and all_idx have to be sent to every worker in order to extract the columns needed, or did I miss something?
  2 Commenti
Edric Ellis
Edric Ellis il 18 Dic 2019
It looks from your code very much like each iteration of your outer parfor loop needs the whole value of X available. In this case, it is unavoidable that X must be sent separately to each worker.
"Slicing" variables is good - when it works - i.e. when your parallelism can be expressed as operations over independent "slices" of your data
Mariella Dreißig
Mariella Dreißig il 18 Dic 2019
Okay thanks, then I got it right and I can't do anything against this broadcast variable.

Accedi per commentare.

Risposte (0)

Categorie

Scopri di più su Loops and Conditional Statements in Help Center e File Exchange

Prodotti


Release

R2016b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by