Parallel Computing, cores, parfor

Is it necessary for me to have the Parallel Computing Toolbox installed to execut parallel calculating to save time? How it can be possible to have 'parfor' in MATLAB without having this box? I replaced all the for by parfor in my program but I did not observe any real reduction of time!!! I have two scripts : the first about a system of 11 interconnected differential equations with ode23s resolution and the second where I call this function n times (for i=1:n yi=f(xi)).

I expected by 'parfor' that every (n/32 . yi) were calculated in parallel on each core! Is this the idea of parallel computing? If it is, why people advise me to revise the ODE script?

I have a 32 cores machine and I must use it for my program!

I ask you to give me real practical advice! what should I do?

Thanks a lot

Answer 1

[Adam]

Is it necessary for me to have the Parallel Computing Toolbox installed to execut parallel calculating to save time?

Yes

How it can be possible to have 'parfor' in MATLAB without having this box?

It isn't, otherwise there would be no point in the toolbox existing and people buying it!

[moulay ELMOUKRIE]

When I am looking for all the boxes installed, I DO NOT SEE the Parallel Computing toolbox. But the parfor works and does not give results for time reducing.

[Adam]

parfor is still valid syntax when you don't have the Parallel Computing Toolbox, which I assume is for reasons of code compatibility - if you do not have the toolbox it just behaves as a normal for loop so that you don't have to change all your code from parfor to for if you no longer have the toolbox.

I'm not actually sure if the caveat in

doc parfor

that iterations are not executed in a guaranteed order still holds in this case though.

[moulay ELMOUKRIE]

I do not understand the lasy sentence: I'm not actually sure if the caveat in

doc parfor

that iterations are not executed in a guaranteed order still holds in this case though.

Can you explain me this please?!!!!

[Jan]

parfor distributes the task to several cores. Therefore the order of processing cannot be predicted and all calculations, which depend on the order are not apllicable. Example:

for k = 1:4, disp(k); end

displays 1 2 3 4 in every case.

parfor k = 1:4, disp(k); end

can reply e.g. 4 2 3 1 when you run it with the PPT. The documentation is not clear, if the order is preserved as with for, when parfor runs without the PPT.

While

i = 1;
v = zeros(1, 4);
for k = 1:4
   i    = i + 1;
   v(k) = i;
end

is valid, the same is rejected with parfor, because the result depneds on the order of te evaluations. The resulting vector v would not be defined uniquely anymore.

Answer 2

[Jan]

A lot of builtin Matlab functions run on multiple threads already, e.g. sum, filter, linear algebra functions etc. Therefore a Matlab program can use multiple cores already without the Parallel Processing Toolbox. But for parfor this toolbox is required. Without this toolbox, parfor needs the same time as a standard for.

You cannot expect a code to run 32 times faster, when it runs on 32 cores. Some resources cannot be parallelized, as the RAM and the construction of the output. Managing the threads requires time also. There can be additional conflicts using a cache line: When all cores write to the same block of data, e.g. a 64 byte block, they wait for each other and might be slower than a single core machine.

To give you any detailed instruction, we must see your code. If somebody suggested to revise the ODE script, this might be a good idea.

Answer 3

[moulay ELMOUKRIE]

I did it one time before! You suggested to me many things to do on this function and it is done. We gained something of course but the problem that the n the number of f(Xi) that we need is so Big and we have to divid computing on the available cores to get an interesting result. I was scared when you said: " When all cores write to the same block of data, e.g. a 64 byte block, they wait for each other and might be slower than a single core machine". the two scripts are attached. At each run i from 1 to n, the six parameters Par take different values.

[Jan]

Who is "you"? Do not be scared by problem of parallelizations. There is always a solution, when the problem can be parallelized at all. And if not, it is not your fault. ;-)

Using globals is a bad idea, especially for parallelization. As fas as I can see, this is not an error here, but better remove the discontinuity "if t > tadd" from the function to be integrated. This might reduce the stepwidth of the integrator to a tiny limit and can increase the processing time dramatically. Prefer to intergrate from 0 to tadd and then from tadd to the final time using te output of the first part as initial value to the second part.