Improve several Parfor performance

Hello!

Im currently working on finding the best configuration for some filters and in order to achieve that, Im doing some procedural tests on each one of them. Those tests goes as follows:

   parfor i = 1 : 10
      for j = 1 : 10
        for k = 1 : 10
            %filter A
        end
      end
   end

   parfor i = 1 : 10
      for j = 1 : 10
        for k = 1 : 10
            %filter B
        end
      end
   end

   parfor i = 1 : 10
      for j = 1 : 10
        for k = 1 : 10
            %filter C
        end
      end
   end

So, lets say filterA, with i = 1, takes 1 second to execute, with i = 2, 2 seconds and with i = 10, 10 seconds. In that way, i = 10 will bottleneck the parfor, and all threads would have to sit idle waiting for the longest execution to end.

A solution to that would be to create a single parfor containing filter A, B and C, but if filter A takes 1 minute to execute, B takes 10 minutes and C takes 100 minutes, then filter C would bottleneck the parfor, thus ending with the same problem as before.

Another solution would be to try and get the parfors to execute in an order, starting with i = 10, then i = 9, all the way to i = 1, but since they are working in parallel, I dont think I would be able to order them. (An execution doesnt depend on the others).

So my question here is: Is there a way to stack every call in an array and then send it to be executed in a parfor? The array would end up being something like this:

Array = { filterA(i=1), ...filterA(i=10), filterB(i=1)...filterB(i=10), filterC(i=1)...filterC(i=10) }; (not necessarily with the indexes ordered) And then on a parfor I would only need to loop the entire array and call its elements. This solution would still eventually be bottlenecked (lets say the last execution chosen is also the longest one).

Also, is there actually a better solution to this?

My goal here is to reduce the overall execution time, by having less threads sitting idle waiting for a single thread to execute a long method.

Thank you for your time, and sorry if this got a little confused.

Answer 1

[Edric Ellis]

You can use parfeval to "stack up" all the calls. This should give you reasonable load-balancing (i.e. ensure all workers are kept busy). With parfeval, tasks are started in the order that you call parfeval, so if you know a priori that some are long-running, it makes sense to schedule those first. So, you might do this:

futures = [];
numOutputs = 1;
for ijk = 1:1000
  futures(end+1) = parfeval(@filterC, numOutputs, ijk);
end
for ijk = 1:1000
  futures(end+1) = parfeval(@filterB, numOutputs, ijk);
end
for ijk = 1:1000
  futures(end+1) = parfeval(@filterA, numOutputs, ijk);
end
wait(futures); % block until all complete
fetchOutputs(futures) % get all outputs

In practice, there's some overhead to calling parfeval, so you might want to batch things up a little more.