How to define 'local' matlabpool size at runtime for Parallel Computing Toolbox use in compiled MATLAB code?

Is it possible for a user of a standalone compiled MATLAB code, which uses the Parallel Computing Toolbox, to define the matlabpool size at run time?

My code queries the local configuration to get the number of workers:

schd = findResource('scheduler', 'configuration', 'local');
numWorkers = schd.ClusterSize;

I then allow the user to revise that number down via a GUI contol (so they don't max out their system if they have other stuff running). When processing begins I call:

matlabpool('local', numWorkers)
... % do stuff with parfor
matlabpool close

It works fine on my machine, however, when I compile this code on my machine (where the standard 'local' configuration has 8 workers) and then run it on a machine with only 2 cores, the number of workers gets set to 8, even though only 2 are available. Clearly the Compiler has compiled up and included my 'local' config.

If I don't know in advance how many cores a user will have, how can I access THEIR 'local' matlabpool configuration at runtime, instead of mine?

I'm using R2011a.

Many thanks, Nick

[Nick]

As a brief footnote to this question, I just reinstalled MATLAB after replacing my hard drive. Now when I compile, distribute, and run the above code, it does EXACTLY what it was supposed to. AEven though the code is compiled on my 8 core machine, it collects the correct number of cores on any machine, rather than hardcoding to 8 like it was doing.

I don't know why it now works, but I suspect it might be due to the fact that I haven't tried to edit the 'local' scheduler on my rebuilt development PC, so the code is trully accessing the 'local' scheduler on each of the target PCs.

Weird, but a great outcome.

Nick

Answer 1

[Elwin Chan]

Hi Nick,

When you compile a PCT application, all of your configurations get bundled into the compiled application, so if your local configuration has a hard-coded ClusterSize value of 8, then when the compiled application runs, it will always use a ClusterSize of 8, regardless of the number of actual cores on the machine.

You can supply a custom configuration to use for the application using the mcruserdata flag. e.g, if your application is myExe.exe, and your exported configuration is myConfig.mat, then you would use this command:

 myExe.exe -mcruserdata ParallelConfigurationFile:myConfig.mat

See the documentation for more details.

In order to take advantage of this, your code need to be configuration agnostic and always use the default configuration. Your code would then look like this:

 matlabpool % Open a matlabpool using the default configuration
 ... %do stuff
 matlabpool close

In general, it is better for your compiled code to use the default configuration rather than a hard-coded one because it allows users of the compiled application to run it on different clusters without you having to modify the actual code.

Elwin

[Nick]

Great answer, thanks Elwin.

I guess was trying to be too clever.

The real problem is the code using all cores (the default) and making the user's computer unresponsive while the computation is happening. An alterantive solution to changing the number of cores would be to change the job priority? Is that possible with just PCT in a standalone Windows app, without manually editing the priority in the Task Manager?

Thanks again.

Nick

Answer 2

[Marco Lavalle]

Hi.

Last week I had a similar problem and I was glad to see this post.

However I have still an unsolved issue. I am trying to execute the program myExe.exe on a different machine than the one I used to compile the code.

1. I compile my code using mcc on the machine A, which has 8 cores.

2. On the machine A, I generate a parallel configuration file specifying 16 cores (=number of cores on the machine B). I tried several configuration options, like 'local' or jobmanager.

3. Then I connect to machine B.

4. I execute: myExe.exe -mcruserdata ParallelConfigurationFile:myConfig.mat

5. I get the following error: "Error when using the 'scheduler' section of the configuration myConfig.mat The ClusterSize for a local scheduler must be between 1 and 8".

How can I tell MCR to use all 16 cores on the machine B? And how should I prepare my configuration file? The documentation does not seem to provide an answer.

Many thanks, Marco.

[Nick]

I can only answer part of your question. The PCT only supports up to 12 cores in the latest version. In R2011a it only supported a maximum of 8 cores.

Nick

Answer 3

[Eduard]

Hi, I compiled my code and setmcruserdata('ParallelConfigurationFile','local.mat')

And then I tried to execute the code like this:

MAAT_Master.exe -mcruserdata ParallelConfigurationFile:local.mat

It didn't work and led to the following error message:

??? MAAT_Master.exe -mcruserdata ParallelConfigurationFile:local.mat | Error: Unexpected MATLAB expression.

Can you help me, what is wrong with this code?

[Walter Roberson]

Do not use that from the MATLAB command prompt. Or prefix it with '!' to run it from the MATLAB command prompt.