Using .Net Assembly Methods in parfor Parallel loops.

I am trying to use methods from a .Net Assembly class inside a parfor loop in MATLAB 2017a. Here is the relevant section of code, omitting the section where the method inputs are generated. I run the addAssembly command on all workers (which posts a lot of unnecessary info to the window, but I can deal with that):

gcp;
pctRunOnAll NET.addAssembly('SinapsXNet');
pctRunOnAll SP = SinapsXNet.SindaPlotting;
parfor i = 1:NumRecords
     SP.getAllAtRecord('T',RecordNumbers(i));
     TemperatureOut(i,:) = double(getXArray(SP));
end

And this creates the error "Warning: Unable to load .NET object. Saving (serializing) .NET objects into a MAT-file is not supported." This function works without an issue if constructed with a basic 'for' instead of 'parfor'. Is there a workaround for this error?

Answer 1

[Edric Ellis]

In your example, the variable SP is being transferred to the workers, and that uses the same machinery as save and load (despite your use of pctRunOnAll, the body of your parfor loop is using the SP instance transferred from the client). I don't have the necessary code to try this out, but perhaps it might work to do the following:

gcp;
pctRunOnAll NET.addAssembly('SinapsXNet');
SP_c = parallel.pool.Constant(@SinapsXNet.SindaPlotting);
parfor i = 1:NumRecords
    SP = SP_c.Value;
    SP.getAllAtRecord('T',RecordNumbers(i));
    TemperatureOut(i,:) = double(getXArray(SP));
end

This avoids the problem by using parallel.pool.Constant with a function handle to build the .NET object directly on the workers.

[Frank Forkl]

Thank you for the response. I know it's challenging to debug code you can't run. It still crashed on the same error, but in a different place in the process. Previously, the crash location was described as follows:

Warning: Unable to load .NET object. Saving (serializing) .NET objects into a MAT-file is not supported.
> In parallel.internal.pool.deserialize (line 9)
  In parallel.internal.pool.deserializeFunction (line 12)
  In remoteParallelFunction (line 33)
Warning: Unable to load .NET object. Saving (serializing) .NET objects into a MAT-file is not supported.
> In parallel.internal.pool.deserialize (line 9)
  In parallel.internal.pool.deserializeFunction (line 12)
  In remoteParallelFunction (line 33)
Warning: Unable to load .NET object. Saving (serializing) .NET objects into a MAT-file is not supported.
> In parallel.internal.pool.deserialize (line 9)
  In parallel.internal.pool.deserializeFunction (line 12)
  In remoteParallelFunction (line 33)
Error using ReadSINDASav2 (line 129)
Attempting to access the property or method of an invalid object.

With the revision you proposed, I get this error instead:

Warning: Unable to load .NET object. Saving (serializing) .NET
objects into a MAT-file is not supported. 
> In parallel.internal.pool.deserialize (line 9)
  In distcomp.remoteparfor/getCompleteIntervals (line 149)
  In parallel_function>distributed_execution (line 820)
  In parallel_function (line 587)
  In ReadSINDASav2 (line 129) 
Error using ReadSINDASav2 (line 129)
Message: Object reference not set to an instance of an object.
Source: SinapsXNet
HelpLink:

[Edric Ellis]

That looks like a loop output is somehow a .NET object (that last warning is being thrown at the client when trying to interpret results from the workers). Does the parallel.pool.Constant appear to do the right thing at all? I.e. can you access the .Value field on the workers (using parfor or even spmd), and does it appear to contain the correct object? I.e.

spmd
  disp(SP_c.Value)
end

[Frank Forkl]

So, with a bit more context, what is going on makes a bit more sense. Here is enough to code to actually run if you have the .dll and data file (same as the other post you replied to):

%Read in SINDA Save Files using .NET assemblies
clear;
FP = 'F:\Read Sav File\2017-09-19_ResUp3Ch.sav';
gcp;
pctRunOnAll NET.addAssembly('SinapsXNet');
SP = SinapsXNet.SindaPlotting;

%Load in Data File
reOpen(SP, FP, 'PC');

%Count Numbers, load indicies
RecordNumbers = int64(getIntYArray(SP));
[~,NumRecords] = size(RecordNumbers);

%Iterate through data and output to a matrix:
parfor i = 1:NumRecords
    SP.getAllAtRecord('T',RecordNumbers(i));
    TemperatureOut(i,:) = double(getXArray(SP));
end

(reOpen, getIntYArray, getAllAtRecord, and getXArray are all methods of the class SinapsXNet.SindaPlotting)

So, the SP object is carrying information with it, specifically the data from the file it loaded. This is why the suggestion to create the object on the workers didn't work, the object wasn't loaded with the data. This code still throws this error:

Warning: Unable to load .NET object. Saving (serializing) .NET objects into a MAT-file is not supported.
> In parallel.internal.pool.deserialize (line 9)
  In parallel.internal.pool.deserializeFunction (line 12)
  In remoteParallelFunction (line 33)
Error using ReadSINDASav5_SlimPar
(line 16)

I'm stuck as to a solution to this in R2107A (though it does work with a few reversions in R2013A). For parfor to work, SP would need to get passed to the workers, and it appears there isn't a way to do that.