How to execute multiple spmd blocks in parallel?

Hello everyone,

I encountered a problem with the Matlab Parallel Computing functionalities, for which I could not find a solution in the Forum or Documentation.

I have a function, say "my_parallel_function" within which I use an spmd block to execute code in parallel. In the real case I also make use of the labSend/labReceive functionalities.

function [  ] = my_parallel_function( N_R, parameter )
    spmd(N_R)
        disp( [ 'Hello from lab ' num2str(labindex) ': P=' num2str(parameter) ] );
    end
end

Now I want to execute this function in parallel, for different values of "parameter". My first attempt was to create a job object with independent tasks and let each task evaluate the function:

N_T = 2;    % number of tasks
N_R = 3;    % number of "realizations" which each tasks is supposed to run in parallel

c = parcluster;
j = createJob( c, 'Name', 'my_job');

parameter = 1;
t1 = createTask( j, @my_parallel_function, 0, { N_R, parameter }, 'CaptureDiary', true );

parameter = 2;
t2 = createTask( j, @my_parallel_function, 0, { N_R, parameter }, 'CaptureDiary', true );

submit(j);

However, this fails with the follwing error: "Could not create an SPMD block to match the requested size: 3. The parallel pool size is: 0."

So, I figured out the problem is that within a task (which is apparently always associated with 1 worker) a parallel pool cannot be created.

I also tried to work with "batch" but the same problem occurs.

Is there a way to associate multiple workers with the execution of a task which itself uses spmd?

As far as I understand, nesting of spmd and/or parfor is also not possible.

Any help is very much appreciated.

Thanks, Jan

[Jan Nitzbon]

I figured out that using the "batch" command and specifying the 'Pool' variable works:

parameter = 1;
j1 = batch( @my_parallel_function, 0, { N_R, parameter }, 'CaptureDiary', true, 'Pool', N_R );

parameter = 2;
j2 = batch( @my_parallel_function, 0, { N_R, parameter }, 'CaptureDiary', true, 'Pool', N_R );

However, I would still like to use the job/task framework to get a better handling of the jobs and tasks. Does anyone know if a similar thing works for "createJob/Task"? Apparently there is no 'Pool' variable to pass to a task or job...

Thank you very much!

Answer 1

[Edric Ellis]

You have already discovered batch, which I believe does everything you need. The alternative is to use createCommunicatingJob, which has two variants. The 'Pool' type is basically the same as the batch invocation you've discovered. The 'SPMD' type is appropriate when you essentially want to run a single spmd block. In that case, you simply specify the task function to be the body of the spmd block. In other words, given

function myOuterFunction()
spmd
  mySpmdBody();
end
end

j = batch(@myOuterFunction, 0, {}, 'Pool', 3);

is pretty much equivalent to:

j = createCommunicatingJob('Type', 'spmd', 'NumWorkersRange', 3);
t = createTask(j, @mySpmdBody, 0, {});
submit(j);

The main difference here is that the createCommunicatingJob variant uses only 3 workers rather than 4 for the batch variant, since it doesn't need a separate worker to manage running the spmd block.

[Jan Nitzbon]

Thank you very much for the reply. I guess I will stick to using batch for now. The solution using createCommunicatingJob would require a lot of restructuring because I do not have something comparable to mySpmdBody. What I would find desirable is something more like a function createCommunicatingTask:

j = createJob();
t1 = createCommunicatingTask(j, @myOuterFunction, 0, {parameter1}, 'Pool', 3);
t2 = createCommunicatingTask(j, @myOuterFunction, 0, {parameter2}, 'Pool', 3);
submit(j);

to get a better handling ob the jobs/taks compared to using batch. But I guess I have to stick to batch for now.