Matlab efficiency - Pass by reference

My Matlab project uses a main class object to manipulate data. The class object is fairly complex from a data structure perspective, with data trees, tables, structures of multiple depths, etc. Some of the methods call external functions to process the data. Many are, I'll say, "pass by reference" e.g.

[R.Struct1] = my_external_function(R.Struct1,var1,R.struct2)

R is the class object, and Struct1 is a property of the object. my_external_function modifies R.Struct1 with tight restrictions: Struct1 is not allowed to change size, and, in general, assignments are made using indexing, e.g. (within the my_external_function),

Struct1.node_tree.count(1) = Struct1.node_tree.count(1) + 1;

The problem I have is performance when running in the Matlab environment. The Matlab Profiler says that, say, 7.586 seconds (569 calls) are spent in child function "my_external_function". Yet when I dive into the child function, it reports that only 0.286 seconds (569 calls) were spent there. Is the unaccounted-for time due to unintended pass-by-value/overhead?

This is with R2014b. R2015b is about 20% worse. :(

I'll note here that when the my_external_function is converted to C (using Matlab coder), Struct1 is correctly passed as a single pointer (pass by reference). All my code must remain Coder compliant.

Thanks for any insight.

aj

[Adam]

Can you not use a handle-derived class instead of a struct? Then you will achieve actual pass-by-reference behaviour rather than relying on hoping the in-place optimisation works for structs.

I don't know what your class does obviously, but personally I almost never use structs - there are no situations in my usual programming where I would consider a struct to be a better solution to a problem than a class.

Having an external function making changes to a class property also sounds a little dodgy, but that is just an issue of design so is not really important to this discussion other than that obviously you wouldn't have to pass your struct anywhere if the function doing the work was a class-member function rather than an external function.

[Philip Borghesani]

If R2015b pre-release is 20% worse it would be helpful if you could submit a bug report with simplified code showing the slow down. We are actively working on the performance of R2015b and future versions and 20% loss is significant enough for us to take a good look at.

[AJ]

I love the concept of classes, and have used them on occasion, but I tend to avoid them for three reasons:

1. Matlab Coder does not support them.

2. They are not MEX friendly.

3. My benchmarking experiments (albeit several versions ago) showed that classes (methods and properties) are slower than functions and structures.

[Adam]

I have personally used classes in combination with both Matlab Coder and mex. Our company explicitly did a pilot project to see how Matlab Coder behaves with classes since they are how I program so we didn't want to buy a license for it before seeing if it would have any value at all for my OOP programming.

Certainly passing classes to Mex is not explicitly doable (though you can convert a class to a struct in some situations), but it depends what parts you want to do in mex really, you can just pass the things that mex needs explicitly by pulling them out of the class but use them in the class for every other usage.

I have also done various experiments on the speed of classes and also had massive slow-downs initially. Whether they have been optimised in the last few Matlab versions or not I am not sure, but it feels as though they have. Mostly though I just learned how to use classes and how not to use them when it comes to speed.

If speed is your only concern then yes, classes are not ideal, although I'd be very surprised if structs are overly performant either, but I don't use them so I couldn't really comment on that.

[James Tursa]

The new classdef style classes are certainly not mex friendly as you note. Mainly because the only routines for accessing them, mxGetProperty and mxSetProperty, use deep data copies.

In contrast, the old style classes using the @classname directory structure only, are very mex friendly because internally the variables are stored as structs, so all of the struct API functions such as mxGetField etc that work with pointers to the field elements (rather than deep data copies of the field elements) can be used on these objects.

If you want to work with pointers to the properties of classdef objects you have to use hacks. E.g., this one will get a pointer to a classdef property:

http://www.mathworks.com/matlabcentral/fileexchange/30672-mxgetpropertyptr-c-mex-function

The reverse task of putting a shared data copy of a variable into a classdef property instead of a deep data copy is not yet posted to the FEX, although I have a working prototype.

[James Tursa]

Update: The FEX submission above now contains the code for putting shared data copies of variables into classdef properties.

Answer 1

[James Tursa]

I don't know if the in-place parsing is smart enough to deal with struct fields or not. E.g., see this Blog by Loren:

http://blogs.mathworks.com/loren/2007/03/22/in-place-operations-on-data/

And this related post:

http://www.mathworks.com/matlabcentral/answers/223142-passing-by-reference-an-object-in-a-function

[AJ]

As an experiment I did the following:

Temp = R.Struct1;
[Temp] = my_external_function(Temp,var1,R.struct2);
R.Struct1 = Temp;

The total execution time did not change much, but, interestingly, lines 1 and 3 account for 98% of the time. The actual function call was almost negligible. This suggests that my attempt to pass-by-reference was foiled. I hope Loren or some other expert can shed some light on this.

[Jan]

The profiler disables Matlab's JIT acceleration, because the JIT can change the order of commands to improve the speed. Therefore the profiler is not a reliable tool to examine the execution time. Notice that e.g. the end of a for loop can take a remarkably chunk of the execution time in the profiler, although this command performs a simple jump only.

This is very sad, because profiling is important for improving the performance. But currently (since Matlab 6.5) the profiler is not powerful enough to examine Matlab code under standard conditions (enabled JIT). tic / toc is reliable.

[Philip Borghesani]

Try this code instead:

 Temp = R.Struct1;
 R.Struct1=[]; % Make sure there are no other references to Temp 
 [Temp] = my_external_function(Temp,var1,R.struct2);
 R.Struct1 = Temp;

The profiler does not disable the jit, It does introduce quite a bit of overhead, code that performs well in the jit tends to show the effect of this the most and may now run slowly again.

[AJ]

That was a good experiment. Unfortunately, there was no improvement. The total execution time remains the same, with proportions [0.42, 0.19, 0.03, 0.36] among the four lines. Somehow, a deep copy is occurring, I suspect. The piper must be paid. I also tried turning the profiler off, but using just tic/toc showed no improvement.

I tried one other test (no assignment):

my_external_function(R.struct1,var1,R.struct2);

This reduced execution time but about 60%, but again, only a small portion of the time is within the function.

[James Tursa]

I will add a few comments, which basically demonstrate that I don't understand what is going on yet. Philip's attempt at avoiding the deep data copy is a good one, but the timings just don't make sense to me.

Temp = R.Struct1;

The above line of code should have taken a very small fraction of the time, since it only results in a shared data copy of R.Struct1 into Temp. I have no idea why the profiler is showing a 0.42 fraction.

R.Struct1=[]; % Make sure there are no other references to Temp

The above line does not necessarily ensure there are no other shared data copies with Temp. It only ensures that the R.Struct1 is not one of them. If there are other shared data copies (or reference copies, etc) of R.Struct1 floating around, they will still be shared with Temp, so any subsequent changes to Temp would cause a deep data copy of Temp. MATLAB gives the user no tools to determine the shared status of a variable (shared data copy, reference copy, shared parent copy, etc), so the only way to ensure there is no sharing is to be very careful how you use the variable. (There is a way to determine some of this in a mex routine, but that is beyond the scope of this thread)

R.Struct1 = Temp;

Again, this should only produce a shared data copy of Temp in R.Struct1 which should only take a small fraction of time. I have no idea how it is getting a 0.36 fraction of time.

Maybe Jan is on the right track and the profiler can't really be trusted to show which lines are taking the bulk of the time in this case.

[James Tursa]

Update: There is now an FEX submission to give you some insights on variable sharing:

https://www.mathworks.com/matlabcentral/fileexchange/65861-isshared-determines-data-sharing-status-among-workspace-variables

Answer 2

[Philip Borghesani]

Does R.Struct1 contain objects (especially handle objects) or data that could be in other objects? Of particular concern is any back references to R.

Structures must be searched when added to and removed from an object via property access to determine if there are circular references and keep track of them properly. With large data structures and classes this can represent significant overhead.

If only some fields of Struct1 are being accessed by my_external_function then you might be better off passing them separately or passing the entire object if many fields are being modified.

[R.struct1.fld1,R.struct1.node_tree.count,...]=my_external_function(R.struct1.fld1,R.struct1.node_tree.count,...

[AJ]

Definitely no back-references. All types are strictly defined (and fixed size) to be compatible with Matlab Coder. Even though in the Matlab environment R is a class, I have a tool that rips the properties and methods into a structure and set of individual functions so I can get around the limitation of Coder which doesn't support classes. Once I go to C code, my performance problems are solved. It's merely inconvenient to have to wait for execution while developing in the modeling environment. Since R is a class object, does you comment about having to search for circular references still have merit?

[James Tursa]

Again, I haven't yet seen evidence that MATLAB is smart enough to do struct fields in-place. E.g., this causes a deep data copy on my machine (R2014a, Win64):

function firstfun2
disp('s.x = rand(1,10000000)')
tic
s.x = rand(1,10000000);
toc
disp('mxArray_Tag(s.x)')
mxArray_Tag(s.x);

disp('s.x = secondfun(s.x);');
tic
s.x = secondfun(s.x); % Attempt to change one element in-place
toc
disp('mxArray_Tag(s.x)')
mxArray_Tag(s.x);
end

function X = secondfun(X)
X(1) = 1;
return
end

The output is as follows. Note that the data pointer pr gets changed, indicating a deep data copy has occurred. The mxArray_Tag is a custom mex function of mine that shows variable type, sharing (CrossLink), data pointers (pr), etc.

>> firstfun2
s.x = rand(1,10000000)
Elapsed time is 0.111474 seconds.
mxArray_Tag(s.x)
------------------------------------------------------------------
Structure Address = 000000000A2BB2E0
Variable Type     = 3 SubElement
CrossLink         = 0000000000000000
pr                = 00000000D8FD0060
First three values:
 0.814724
 0.905792
 0.126987
s.x = secondfun(s.x);
Elapsed time is 0.062011 seconds.
mxArray_Tag(s.x)
------------------------------------------------------------------
Structure Address = 000000000A2BB2E0
Variable Type     = 3 SubElement
CrossLink         = 0000000000000000
pr                = 0000000174030060
First three values:
 1
 0.905792
 0.126987

I think your Temp code above was promising ... and I don't yet understand why it didn't work unless R.Struct1 was already a shared copy of something else.

I won't bother to post the output, but will simply state that this code did not produce a deep data copy:

z = rand(1,10000000);
z = secondfun(z);

[James Tursa]

@AJ: You say there are no back references, but are you completely sure about this? How is R.Struct1 actually created?

[Philip Borghesani]

James your are correct there is currently no in place optimization for

 s.a=foo(s.a)

So the "temp trick" would be needed to get the in place optimization.

That does not seem to be AJ's issue though, the time appears to be in extracting the struct from the object and putting it back in. Because R is a class (still don't know if it is a handle class or if that makes any difference) extracting and inserting a large structure can have high overhead. How many nodes are in node_tree?

[AJ]

OK, I did an analysis on the class structure. R is a classdef object that has 178040 mxArray pointers within it (in a recursive sense). None are handles or classes, although there are some enumerations. If I look at the mxGetData() pointers for each, there is a HUGE number of duplicate values, for two reasons:

1. When I initialize fields of a structure to, say, 0.0, all similar fields get the same data pointer. Interesting.

2. When I have arrays of structures (all fixed size), I used repmat to create them. This, of course, creates references, not copies.

3. During processing, relationships are established among the data structures, and I'll bet there is a lot of "cross-pollinating" of data references.

It will not be possible (within practicality) to make my structures reference-free. So I guess I'll have to suck it up for now. Thanks for everyone's contributions.

aj