`parfor` doesn't seem to work with Python lists

Hellp everyone,

I'm trying to use a parfor loop that includes a Python list as an input argument to a class array:

 allEvents(1:size(hexData, 2), 1) = Event(); 
 parfor n = 1:size(hexData, 2)
      allEvents(n) = createSingleEvent(hexData{n}, dataFormat, range, hexToBinMap);
 end

where hexData{n} is the mentioned Python list, and allEvents is a preallocated array of class Event() (value class, not handle).

Unfortunately, no matter how I turn it, the parfor loop won't run. Needless to say, a regular for loop compiles properly.

The error I receive is a "Python error: Index exceeds list dimensions". When I manually check possibly-problematic indices I recieve no such error.

By adding a "char(hexData{n})" inside the parfor I receive "Index exceeds matrix dimensions" in the first occasion inside the function where I need to use the hexData{n} value, even if I see that the index definitely exists in that variable:

 startOfString = hexDataAsChar(1); % This errors out

Only when I erase all traces that hexData was ever a Python list does MATLAB kindly allows me to parfor this loop.

Any help will be deeply appreciated. Thanks!

Answer 1

[Robert Snoeberger]

If you are using a parpool, then parfor needs to save and load the Python list. Saving (serializing) Python objects into a MAT-file is listed as a limitation .

[Hagai Hargil]

Dear Robert, Thank you for your quick reply. I wasn't aware that this 'save and load' process occurs during the parfor loop. Is there a workaround around this limitation, which can enable me to parallelize processing of Python objects?

[Robert Snoeberger]

Unfortunately, I'm not aware of a workaround.

The only idea that I can think of is to wrap the Python object in your own MATLAB class with customized save and load . You might be able to use pickle to do the serialization. I haven't tried it so I don't know if the idea is viable.