How to unit test several different functions that take the same input arguments efficiently?

This is my first time writing unit tests, so advice from experienced users will be greatly appreciated.

I have several compute functions from which many of them take in the same input arguments, and I would like to write efficient unit tests with minimum code duplication.

I have a following working test class:

%% Test Class Definition
classdef ComputeFunction1Test < matlab.unittest.TestCase
    
%% Test Method Block
    methods (Test)
        % includes unit test functions
        %% Test Function
        function testFunction1FirstOutput(testCase)
            % Actual solution - AB contribution
            load('test_data.mat')  % variables input1, input2, and input3
            [act, ~] = computeFunction1(input1, input2, input3);
            % Verify using expected AB contribution
            load('function1ExpectedOut1.mat');  % variable exp contains expected output
            testCase.verifyEqual(act, exp);
        end
        function testFunction1SecondOutput(testCase)      
            % Actual solution - normalization factor for AB
            load('test_data.mat')
            [~, act] = computeFunction1(input1, input2, input3);
            % Verify using expected normalization factor
            load('function1ExpectedOut2.mat');  % variable exp contains expected output
            testCase.verifyEqual(act, exp);
        end
    end
end

I have several more different functions that take the same input arguments and produce output in the same format as computeFunction1.m, which above test class tests. (say computeFunction2.m, computeFunction3.m, and computeFunction3.m, ...)

I could copy the above test class and change the function name to test other functions that take the same input arguments, but this seems inefficient.

Is there a way to handle this situation more efficiently?

Note that I do have other compute functions that require different input arguments as well that need to be tested too. (say specialComputeFunction1 and specialComputeFunction2.m)

[Jon]

Maybe you could do it in a loop. Put all of the expected results in a single matrix, with a column for each argument

then just evalute the function once returning all of the output arguments and have a loop to compare each argument with appropriate column in expected results matrix. Return vector of pass fail logicals with an element for each output argument. In your example, just two elements, but it could be any number

[Jon]

Sorry, I should have looked more deeply into your question. I hadn't realized that you were using a built in unit test class provided by MATLAB. I would then steer you to looking at the examples and documentation for that. I'm sure there is a standard approach for what you are trying to do as it seems like a typical usecase. I will look into this more myself, would probably be helpful in my work to use this built in testing functionality

Answer 1

[Sean de Wolski]

Look into using TestParameters. You can use an array of function handles as the parameter values to traverse different functions and pass the results as parameters as well. You'll need to run them with ParameterCombination=sequential. Here's a functional example:

classdef tFcnParameter < matlab.unittest.TestCase

    properties (TestParameter)
        fcn = struct('plus',@plus,'minus',@minus)
        result = {5 1}
    end    
 
    methods (Test, ParameterCombination = 'sequential')

        function shouldApplyFcn(testCase, fcn, result)
            r = fcn(3, 2);
            testCase.verifyEqual(r, result);
             
        end
       
    end
 
end

Answer 2

[Steven Lord]

%% Test Class Definition
classdef ComputeFunction1Test < matlab.unittest.TestCase
    
%% Test Method Block
    methods (Test)
        % includes unit test functions
        %% Test Function
        function testFunction1FirstOutput(testCase)
            % Actual solution - AB contribution
            load('test_data.mat')  % variables input1, input2, and input3
            [act, ~] = computeFunction1(input1, input2, input3);
            % Verify using expected AB contribution
            load('function1ExpectedOut1.mat');  % variable exp contains expected output
            testCase.verifyEqual(act, exp);
            
            % *snip rest of test file*

You don't want to do this. The identifier exp already has a meaning in MATLAB, and there's no indication that it should be a variable when MATLAB parses the function. If you must load data in a function I strongly encourage you to call load with an output argument so you don't "poof" variables into the workspace.

Rather than using MAT-files, if the output arguments are "small" consider hard-coding them. Or consider using the definition of the operation that computeFunction1 performs to determine a way to validate the results without hard-coding them. For instance, if I were validating the svd function (accepts A and computes U, S, and V such that U*S*V' effectively equals A) a valid verification could be "is U*S*V' 'close enough' to A?" This avoids having to create hard-coded copies of expected U, S, and V matrices.

[U, S, V] = svd(A);
testCase.verifyEqual(U*S*V', A, 'AbsTol', someTolerance)

You can also have multiple verify* calls in the same method. I've labeled each portion of the test with which of the four phases they implement.

classdef testBounds < matlab.unittest.TestCase
    methods(Test)
        function simpleTest1(testCase)
            % Setup phase
            x = 1:10;
            
            % Exercise phase
            [minValue, maxValue] = bounds(x);
            
            % Verify phase
            testCase.verifyEqual(minValue, 1);
            testCase.verifyEqual(maxValue, 11); % Whoops!
            
            % Teardown phase
            %
            % x, minValue, and maxValue will automatically be destroyed when the function exits
            % so there's no need for explicit teardown for this test.
            %
            % If the test had opened a figure window (for example) this is where you'd want to
            % close that figure window. Actually in that case I would have used addTeardown 
            % right after I created the figure in the Setup phase so it would be closed
            % even if the Exercise or Verify phases threw a hard error. But here is where that
            % teardown added by addTeardown would trigger under normal execution.
        end
    end
end

If you run this, the test failure message will indicate the problem is on the line I've commented "Whoops!"

Be wary of trying to test too much in any individual test method, but in this case testing both outputs from a single function call doesn't strike me as too much.

That being said, if you have a collection of inputs and the corresponding outputs you could write a parameterized test. See the testNumel test method on that page for an example.

Answer 3

[Jon]

Here are a couple of possibilities

You can pass function handles, and then make the function you are testing be one of the arguments to your test program https://www.mathworks.com/help/matlab/matlab_prog/pass-a-function-to-another-function.html

You can also use the MATLAB function eval to evaluate expressions that are written as strings however this seems clumsy, I would prefer using function handles

Also in your example above, why not call the function once, evaluate both output arguments, and then test them rather than calling the function twice, and having all that cut and paste code.

[Louis]

Thank you - I will take a look at the document you referenced.

First output is a list of double whereas the second output is just a single double value, and I wasn't able to find compelling example in the document to evaluate both argument at once.

Could you provide an example for testing both?

I suppose I could just append the second output to the first output and test them together:

testCase.verifyEqual([act1; act2], [exp1; exp2]);

But wouldn't this make it hard to tell where the mismatch is exactly coming from in case this test fails?

[Jon]

Maybe you could do it in a loop. Put all of the expected results in a single matrix, with a column for each argument

then just evalute the function once returning all of the output arguments and have a loop to compare each argument with appropriate column in expected results matrix. Return vector of pass fail logicals with an element for each output argument. In your example, just two elements, but it could be any number

[Jon]

Sorry, I should have looked more deeply into your question. I hadn't realized that you were using a built in unit test class provided by MATLAB. I would then steer you to looking at the examples and documentation for that. I'm sure there is a standard approach for what you are trying to do as it seems like a typical usecase. I will look into this more myself, would probably be helpful in my work to use this built in testing functionality