ParFor takes a lot of time to run

I am having an issue with the time that it takes to carry out my parfor loop. There are no MATLAB errors, but after running PROFILER I seem to have found the source of the issue. 805 out of 807 seconds total were spent on the following line of code:

r = q.poll(1, java.util.concurrent.TimeUnit.SECONDS);

This is found within:

function [tags, results] = getCompleteIntervals(obj, numIntervals)

Which is called within:

java.util.concurrent.LinkedBlockingQueue (Java-method)

according to PROFILER at least. This is all part of the parallel programming method as far as I can make out. I can find the 'poll' function within C: (Windows) and have opened it to have a look, but I don't really understand what it does or how to stop it taking up so much of my time!

Answer 1

[Edric Ellis]

The MATLAB profiler can only provide you with useful information about your client MATLAB instance (i.e. the one where you're invoking PARFOR) - it doesn't tell you anything about where the workers are spending time. What you're seeing there is the client MATLAB waiting for the results to come back from the workers.

There are a number of reasons why PARFOR loops can run slower than you expect. It might be the data transfer from the client to the workers, or it might be because the workers are running in single-threaded mode. If the body of your PARFOR loop happens already to be taking advantage of MATLAB's intrinsic multi-threading, then PARFOR will not make things any faster.

You can get closer to working out what's going on on the workers by launching your client MATLAB with the "-singleCompThread" option, and running the profiler without opening matlabpool.

Answer 2

[Roberto Olmi]

I didn't see this post before crating mine: http://www.mathworks.in/matlabcentral/newsreader/view_thread/301841 Anyway it does not help.

I removed a call to a nested function (sobstituted by a call to a normal function) and this seems to improve the performances.