Unit Test: Write parametrized tests where TestParameters depends on ClassSetupParameters?

I’m currently writing some unit tests, which should be run on every property of every class drawn from a list of classes. Currently I’m using loops in my test methods, but I would prefer to use parametric tests (to get a better log in case of failure). It is needless to say that the list of classes is quite long and each class has different properties – So a static definition of the class name or the property names is not really applicable.

I had something like this in my mind. However, this code is obviously not working since the TestParameters need to be Constant:

classdef TestProperitesOfClasses < matlab.unittest.TestCase

      properties
          class
          object
      end

      properties (ClassSetupParameter)
          class_list = {?MyClass1,?MyClass2} % Classes to be tested
      end

      properties (TestParameter)
          propery_list  % Properties to be tested (Default value required!)
      end

      methods (TestClassSetup)
          function create_test_class(test_case,class_list)
              % Define the currently tested class
              test_case.class = class_list;
              test_case.propery_list = properties(class_list.Name); % Does not work!
          end
      end

      methods (TestMethodSetup)
          function create_test_object(test_case)
              % Create an object of the current class
              constructor = str2func(test_case.class.Name);
              test_case.object = constructor();
          end
      end

      methods (Test)
          function test_property(test_case,propery_list)
              % Code to test some property attributes
              property = test_case.object.(propery_list);
              test_case.verifyNotEmpty(property) % Just an example...
          end
      end

end

Does anybody know a solution or a workaround for this problem? Would the Matlab Unittest Framework accept non-constant test parameters in future releases?

[Neeraj Badamikar]

"... Would the Matlab Unittest Framework accept non-constant test parameters in future releases?" It does in MATLAB R2021a.

Starting R2021a, class-based tests in the MATLAB Unit Test Framework can leverage TestParameterDefinition methods to assign parameter values dynamically at the time of suite creation.

Using this feature, your example could be modified as such -

classdef PropertiesTest < matlab.unittest.TestCase
    properties (ClassSetupParameter)
        ClassToTest = {'ClassA', 'ClassB', 'ClassC'};
    end

    properties (TestParameter)
        PropertyToTest
    end
    
    properties
        ObjectToTest
    end

    methods (TestParameterDefinition, Static)
        function PropertyToTest = initializeProperty(ClassToTest)
            PropertyToTest = properties(ClassToTest);
        end
    end

    methods (TestClassSetup)
        function classSetup(testCase,ClassToTest)
            constructor = str2func(ClassToTest);
            testCase.ObjectToTest = constructor();
        end
    end

    methods (Test)
        function testProperty(testCase,PropertyToTest)
            value = testCase.ObjectToTest.(PropertyToTest);
            testCase.verifyNotEmpty(value)
        end
    end
end

A TestParameterDefinition method is static and can be used to dynamically generate values for a parameter property using parameter values that operate at a higher scope than the one being set. Meaning, values for a TestParameter property can be generated using values of ClassSetupParameter and/or MethodSetupParameter properties. Similarly, values for a MethodSetupParameter property can be set using values of a ClassSetupParameter property. You can achieve this by passing the right inputs to the TestParameterDefinition method.

More information on this feature can be found on this page - https://www.mathworks.com/help/matlab/matlab_prog/define-parameters-at-suite-creation-time.html

Answer 1

[David Hruska]

Sorry for the long delay in replying! Were you able to find a workaround?

The only workaround I can think of is to manually create a single set of parameters that combines both the class and property information. I've included a short example below.

I've also submitted an enhancement request for this feature on your behalf.

classdef TestProperitesOfClasses < matlab.unittest.TestCase
      properties
          class
          property
          object
      end
      properties (MethodSetupParameter)
          class_and_property = constructParameters;
      end
      methods (TestMethodSetup)
          function create_test_object(test_case, class_and_property)
              % Define the currently tested class
              test_case.class = class_and_property.Class;
              test_case.property = class_and_property.Property;

                % Create an object of the current class
                constructor = str2func(test_case.class.Name);
                test_case.object = constructor();
            end
        end
        methods (Test)
            function test_property(test_case)
                % Code to test some property attributes
                test_case.verifyNotEmpty(test_case.property) % Just an example...
            end
        end
  end
  function params = constructParameters
  classes = {?MyClass1,?MyClass2}; % Classes to be tested
  params = struct;
  for class_idx = 1:numel(classes)
      thisClass = classes{class_idx};
      thisName = thisClass.Name;
      props = properties(thisName);
      for prop_idx = 1:numel(props)
          thisProp = props{prop_idx};
          id = [thisName, '_', thisProp];
          params.(id).Class = thisClass;
          params.(id).Property = thisProp;
      end
  end
  end