How to avoid recursion with parenReference

Using Matlab 2021B version

I create a class which inherits both handel & matlab.mixin.indexing.RedefinesParen.

classdef Abc <handle &  matlab.mixin.indexing.RedefinesParen

I create object arrays using this class.

I want to overload the parenthesis for the class so that it can perform special handling in addition to indexing into the object array ( normal operation ).

For example

Abc('special') - performs special handling

Abc(2:3) - down selects only objects 2 to 3 (normal operation without RedefinesParen)

I have written the parenReference function as follows

function varargout = parenReference(obj,indexOp)
    if isscalar(indexOp)
        varargout{1} = obj(indexOp(1).Indices);
    else
%    Other special handling and forward indexing
%                
    end
end

The problem is that no matter how I try to select the subset of objects to return in the third line, it calls parenReference recusively and faults when recursion limit is reached. I have even tried

varargout{1} = builtin('subsref',obj,substruct('()',indexOp(1).Indices))

How do I select a subset of obj in this function without it being called recursively.

Thanks

[Matt J]

Interestingly, this recursion problem doesn't seem to afflict RedefinesDot (in R2021b). In the test classdef below, you can see that I have many expressions of the form obj.something inside my overload of the dotReference() method. They don't trigger any recursions, though.

classdef myclass<matlab.mixin.indexing.RedefinesDot 
%A test class which uses RedefinesDot to make indexing into a struct member more brief.

    properties 
        params  %this is a struct but to access its fields, we want to 
                %do things like obj.a instead of obj.params.a
        c
    end

   methods
        function obj=myclass()

          obj.params.a=1;
          obj.params.b=2;

        end

        function disp_a(obj)
           disp(obj.a);
        end
   end

    methods (Access=protected)
            
        function varargout = dotReference(obj,indexOp)
            if ismember(indexOp.Name,fieldnames(obj.params))
             [varargout{1:nargout}] = obj.params.(indexOp.Name);
            else
              [varargout{1:nargout}]=obj.(indexOp.Name);
            end
        end

        function obj = dotAssign(obj,indexOp,varargin)
            if ismember(indexOp.Name,fieldnames(obj.params))
             [obj.params.(indexOp.Name)] = varargin{:};
            else
              S.type='.'; 
              S.subs=indexOp.Name;
              obj= builtin('subsasgn',obj,S,varargin{1});
            end
        end
        
        function n = dotListLength(obj,~,~)
            n = numel(obj);
        end

    end

end 

>> obj=myclass;

>> obj.a=10; obj.b=20; obj.c=30

obj =

myclass with properties:

params: [1×1 struct]

c: 30

>> obj.params

ans =

struct with fields:

a: 10

b: 20

>> obj.disp_a

10

[James Lebak]

@Matt J -- You don't see recursion with RedefinesDot because the methods of RedefinesDot don't override accessible properties.

We couldn't do something like that for RedefinesParen because it specifically is designed for scalars masquerading as arrays, and in such a case we have no idea how or where array elements are stored by your class.

[Matt J]

the methods of RedefinesDot don't override accessible properties.

@James Lebak I'm not sure what it means for a method to override properties. Am I wrong to assume that dot-indexing statements revert to built-in indexing methods inside dotReference, dotAssign, and dotListLength, but nowhere else?

[James Lebak]

If you set a breakpoint in dotReference you will see it is never called for the public property obj.params, regardless of whether you issue that statement from inside or outside of your class. We don't call dotReference when the property is an actual property that is accessible in the present context.

See the doc for dotReference, which states that it

"handles index referencing operations that begin with dots and index into fields that are inaccessible or nonexistent, such as obj.PrivateProp or obj.NotAProp{1}(2)"

[James Lebak]

You asked "Am I wrong to assume that dot-indexing statements revert to built-in indexing methods inside dotReference, dotAssign, and dotListLength, but nowhere else?"

The idea of 'reverting to built-in indexing methods' is not something that any of the current modular indexing APIs do.

The behavior of dotReference/dotAssign depends on context. When you have a public property named prop, obj.prop does not call dotReference, either inside or outside the class. When you have a private property named privateProp, it's accessible inside the class and so doesn't call dotReference there. But outside the class, it's not accessible, and obj.privateProp calls dotReference.

[Matt J]

I see. So the invocation rules of modular indexing overloads are uniform. You don't have special rules that say built-in indexing will be invoked instead of modular overloads, but only in specific methods.

[James Lebak]

Correct. That uniformity was a design goal for the modular indexing APIs.

[Matt J]

@James Lebak Is there any reason for MathWorks not to offer a syntax for the builtin() command like the following,

tmp=builtin('parenReference',obj,indexOp);

that will call built-in ()-indexing when the user truly wants it? As it stands now, I don't think there is a solution to the OP's recursion issue without something like this.

[James Lebak]

The OP's request can't be accomplished with RedefinesParen. If you inherit from RedefinesParen, your object is always a scalar, and you have to have a property inside the object (or something else) that is the array. The usual paradigm is then to have parenReference/parenAssign index into that other entity, and then there's no recursion problem (because the property is a different class and presumably uses built-in indexing). Internally we call this the 'scalar masquerading as an array' pattern, and it's the only thing RedefinesParen supports. The reason RedefinesParen doesn't have a concept of 'calling builtin parenReference' is that it's really not needed for the specific cases that RedefinesParen handles -- which do not, unfortunately, include the OP's use case.

Subsref and subsasgn support the 'scalar masquerading as an array' pattern, but they also allow you to override indexing into arrays like the OP is trying to do -- for example, to translate character strings into numerical indices. This allows the object to be a built-in array, and also requires some sort of 'map the indices and call built-in paren-indexing' capability as you are suggesting. When we designed RedefinesParen we decided to separate these use cases. If we provided the 'override indexing into arrays' functionality in a modular indexing mixin, we would provide something like the builtin parenReference/parenAssign calls that you're suggesting, and eventually I do hope to provide this functionality. But I think you're correct that the OP's request requires subsref/subsasgn as things currently stand.

Answer 1

[Matt J]

Classic subsref doesn't seem to difficult in this case. You can't beat the classics!

function varargout = susbref(obj,S)

    if S(1).type~="()"
       [varargout{1:nargout}] = builtin('subsref',obj,S); return
    end

    T=S(2:end);
    S=S(1);

    if isscalar(S.subs) && isscalar(S.subs{1})
        out = obj(S(1).subs{1});
    else
        %    Other special handling and forward indexing
        %          
       out=...
    end
    
    [varargout{1:nargout}] = builtin('subsref',out,T);
    
    
end

[Mike Falcinelli]

I tried this. The problem is that I am using the RedefinesDot mixin and it is not compatible with subsref. The only reason that I am trying to use the RedefinesParen, is because I the RedefinesDot does not allow compound statements.

for example

V.specialDot       % works fine if V is an Abc object

but

V(1).specialDot     % causes error - compound statements not allowed, must use redefines paren.

[Matt J]

All the more reason to discard RedefinesDot as well... The only reason that I can fathom to use the Redefines is if only one mode of indexing is being overloaded. Otherwise, I can't see the advantage over just using subsref.

[Mike Falcinelli]

Okay - I have discarded Redefines Dot also and am using subsref & subsasgn. In this case, they are pretty simple and not execution time critical.

I had moved to the RedefinesDot because previously ( 2 to 3 years ago) I tried the subsref/subsasgn route for another application. I found it cumbersome and slow because I had to implement much of the built-in functionality. I liked the ease and simplicity of the RedefinesDot, and it added almost no overhead cost in terms of execution time for 'buit-in' capability.

[Thomas Michiels]

@Matt J i am astonished by your advice. redefinesParen is a better version of subsref. You only redefine what you want to redefine and everything else keeps working. In general subsref code is so ugly because you need to handle so many cases where you want default behavior.

@Mike Falcinelli what you are looking for is the fact that you are trying to index on yourself again, causing recursion. (This would be the exact same with subsref EDIT: apparently it is not.) Usually your class contains an array and you want to make the indexing operation operate on that array, which is a property of your class/obj, it isn't obj itself.

[Matt J]

@Thomas Michiels

(This would be the exact same with subsref EDIT: apparently it is not.)

If redefinesParen had the same robustness to recursion as subsref, I would have fully embraced it.

[Thomas Michiels]

@Matt J understandable. it is so strange to me that you cannot access the original indexing operator anymore. i have to agree with you in this case

[Matt J]

@Thomas Michiels Stranger still that you can access the original operator when using RedefinesDot, see my comment above.

Answer 2

[Shanmukha Voggu]

Hi Mike,

The recursion limit is reached because the parenReference is calling itself recursively,

The solution is to create a property that holds the given array in the class Abc

properties (Access=private)
    copyOfArray
end

And in the constructor of class Abc assign the given array to the property above

function obj = Abc(arr)
    obj.copyOfArray = arr;
end

Now you can use indexing on the property to get the desired results, Instead of using below statement in parenReference function

varargout{1} = obj(indexOp(1).Indices);

use the following statements

 obj.CopyOfArray = obj.CopyOfArray.(indexOp(1));
 varargout{1} = obj; %if you want output as object array, or use obj.copyOfArray in order to have array alone as output 

Refer this for more information.

[Mike Falcinelli]

Thanks for the answer. This works fine for a contained array. I am using an array of objects and trying to implement the built in capability.

In my example above, I create array of objects,

Varx 1x5 Abc

If I do not use matlab.mixin.indexing.RedefinesParen, and do the following

Vary = Varx(2:3)

I get

Vary 1x2 Abc <- points to members 2 & 3 of Varx

When I use matlab.mixin.indexing.RedefinesParen

function varargout = parenReference(obj,indexOp)
    
end

obj is a 1x5 Abc. I cannot find a way to access any subset of obj without getting the recursion error from calling parenReference.

How would I code parenReference to implement the built-in function?

Thanks,

Mike

[Matt J]

The recursion limit is reached because the parenReference is calling itself recursively

That is puzzling behavior, I have to say. I would have thought the rules for overloading parenReference would mimic those of subsref, namely that only indexing expressions issued outside the classdef invoke the overload. Not following that rule introduces new pitfalls, and seems to ruin a lot of the simplification over subsref that parenReference is intended to achieve.

[Mike Falcinelli]

Yes, I would expect that also. The dotReference simplication is nice. It handles all the builtin cases and just passes on what it does not understand. I thought parenReference might be similar, but I am stumped.

[James Lebak]

You're correctly picking up on an intentional design decision of the API. One of the most confusing things about subsref/subsasgn is that rule you mention, that indexing inside the class always calls built-in indexing. The good thing about that restriction is exactly that it prevents recursion like the one the OP is experiencing. However, from a usability standpoint, developers inside and outside the company repeatedly told us that they often experienced behavior that they didn't expect because of the rule. In particular, if you overload dot, paren, or brace, there's no good way to invoke your own overloaded indexing behavior inside your class, short of an explicit call to subsref/subsasgn. Those calls hurt the performance of your overloaded indexing and should not be necessary in the new system.

We separated the uses here into two cases. One case is the 'class is a scalar masquerading as an array' and as this answer points out, RedefinesParen is set up to handle this. The other case is the 'class is a built-in array and modifies or logs the indices with which it is called.' Currently, to implement that use case, you still need to use subsref and subsasgn. We have plans to deliver additional modular indexing functionality to address the second use case, but it isn't in the initial delivery.

[Matt J]

@James Lebak

developers inside and outside the company repeatedly told us that they often experienced behavior that they didn't expect because of the rule

It's definitely something people new to Matlab OOP don't initially grasp, but I greatly worry that the recursion pitfall will be highly prevalent without it, in both of the use cases that you've described. It makes me very nervous about embracing the new Redefines.

In particular, if you overload dot, paren, or brace, there's no good way to invoke your own overloaded indexing behavior inside your class, short of an explicit call to subsref/subsasgn.

Seems easy enough to outsource the work to a class-related function or external function, if you really do need a workspace where the overloaded indexing needs to be in force. I personally have never had to do that.

[James Lebak]

@Matt J

I want to offer reassurance that in our experience with using the modular indexing APIs internally, we haven't found recursion to be a big problem. The APIs don't currently do everything that subsref and subsasgn can, but I would hate to see you avoid them just because of the potential to encounter this problem.

It's true that it's easy to write code that encounters the recursion, and we did see developers new to the API write this code. In the case that RedefinesParen is primarily designed for, the scalar that presents itself as an array, we also found it to be a relatively easy problem to avoid, and to be easily uncovered by simple testing.

We are currently moving several MathWorks classes from subsref and subsasgn to modular indexing and in our production test system, we haven't encountered failures due to modular indexing classes unexpectedly calling overloaded indexing recursively.