Parallel programming: defining tasks

Hi.

I made a simple function, compute(m,n), that calculates the number of primes between m and n. I used this function to test the performance improvement when using parfor and spmd against a simple for cycle. I used the following cycle to test it:

A = [3 480000 830000 1100000];
NumPrimes = 0;

t = tic;

for n = 1:3
    NumPrimes = NumPrimes + compute(A(n),A(n+1)); 
end

toc(t)

As expected using parfor, or rewriting the cycle with spmd, gave me better results than with the serial for cycle.

Now comes the problem. I tried to run a job, defining each task such that each worker would take care of a different subset of numbers. Lab 1 would compute the number of primes from 3 to 480000, Lab 2 from 480000 to 830000, etc... To do it I wrote the following code:

job = createJob('configuration','local','FileDependencies',{'compute.m'});

A = [3 480000 830000 1100000];
NumPrimes = 0;

t = tic;

for n=1:3
    createTask(job,@compute,1,{A(n),A(n+1)})
end

submit(job)
wait(job)
out = getAllOutputArguments(job);
NumPrimes = sum([out{:}]);
toc(t)

When I run this code there are 2 possible outcomes: If I have 3 labs (matlabpool open 3) the code will run but it will take about 3x the time it takes with a serial code. If I have 4 labs it will run forever, until I eventually cancel it.

My doubts are: - Why isn't my code speeding up in relation to the serial version? - When I use the function createTask can't I decide which Lab takes care of the task or is it assigned by order? - Can I have more tasks than workers? If so how will they take care of the tasks? Each lab will take care of one of the first 4 tasks, and when they complete them move on to the next uncomplete task?

If anyone could clear my mind I'd really appreciate. Thanks!

Answer 1

[Thomas Ibbotson]

Hi Daniel,

See my answer to your other related question: http://www.mathworks.com/matlabcentral/answers/62944-parallel-programming-with-createjob

Thanks, Tom