Write Parameterized Tests for C/C++ Functions Using Polyspace Test xUnit API

You can test a function for several combinations of input values using the Polyspace® Test™ xUnit API. Instead of calling the function under test several times with different combinations of input values, you can define a test data array that enumerates all combinations and call the function under test once with the test data array as argument. The test results show each test with a given input combination, allowing you to identify input combinations that fail the test.

This example shows a simple parameterized test.

Prerequisites

This topic describes test authoring using the Polyspace Test xUnit API. To compile these tests, you are required to know some file paths in advance. For your convenience, you can define environment variables to stand for the file paths, or otherwise include the file paths in your build. For more information, see Set Up C/C++ Testing and Code Profiling Using Self-Managed Builds.

Workflow

You can write a parameterized test following these steps:

  1. Define one or more parameters with the PST_PARAM_WITH_VALUES macro. The macro has this format:

    PST_PARAM_WITH_VALUES (parameterName, parameterType, displayFunction, parameterValues, numValues)
    Where:

    • parameterName is the name of the parameter.

    • parameterType is the data type of the parameter.

      You cannot use array parameters such as int[2] directly for parameterType. Instead, you can define a new type that maps to the array type, for instance:

      typedef int[3] threeElemIntArr;
      And use the new type, in this case threeElemIntArr, for parameterType.

    • displayFunction is a display function that can customize the display of parameter values.

      If you use parameters with fundamental types such as int, enter a display function name of the form pst_format_param_typename, for example, pst_format_param_int. For aggregate types, define and use your own display function. To align with the display function name for fundamental types, name your custom display function using the format pst_format_param_typename.

    • parameterValues is an array that supplies values for the parameter.

    • numValues is the number of the values that the parameter can take.

  2. Add the parameters in the test configuration with the PST_ADD_PARAM macro. For example, for a simple test, the addition of two parameters with names expectedValueParam and actualValueParam can look like this:

    PST_SIMPLE_TEST_CONFIG(NewTestCase) {
     PST_ADD_PARAM(expectedValueParam);
     PST_ADD_PARAM(actualValueParam);
}

  3. When calling the assessment macros, instead of calling a variable, call a parameter with the dereference of the PST_PARAM_PTR pointer-like macro.

    For easier test authoring, before parameterizing a test, first write assessments using regular variables, for example, expectedValue and actualValue in this example:

    PST_VERIFY_EQ_INT(expectedValue, actualValue);
    Once the tests are working, parameterize the assessments using test parameters instead of regular variables, for example, expectedValueParam and actualValueParam in this example:
    PST_VERIFY_EQ_INT(*PST_PARAM_PTR(expectedValueParam), *PST_PARAM_PTR(actualValueParam));

    If the test parameter param has structured type, to access a field, say paramField, use the syntax:

    PST_PARAM_PTR(param)->paramField

Example

Example Files

Copy code from steps 1 and 2 into .c files.

Alternatively, find the files for this tutorial in the folder polyspaceroot\polyspace\examples\doc_pstest\parametrized_tests. Copy these files to a writable location and continue the tutorial. Here, polyspaceroot is the Polyspace installation folder, for example, C:\Program Files\Polyspace\R2026a.

Inspect Function Under Test and Requirements

The function saturate_add_int adds two unsigned integers and saturates their sum at UINT_MAX.

#include <limits.h> /* For UINT_MAX = 4294967295*/

unsigned saturate_add_uint(unsigned x, unsigned y)
{
    if (y > UINT_MAX - x) {
        return UINT_MAX;
    }
    return x + y;
}

Save this function in a file example.c.

Suppose you have to test the function for these values of inputs x and y:

 x=0, x=(UINT_MAX/2 - 1), x=(UINT_MAX/2), x=(UINT_MAX/2 + 1), x=UINT_MAX
 y=0, y=(UINT_MAX/2 - 1), y=(UINT_MAX/2), y=(UINT_MAX/2 + 1), y=UINT_MAX
Suppose the testing requirement is that if the sum of the inputs is less than UINT_MAX, the function must return the sum. Otherwise, the function must return UINT_MAX.

Suppose that you want to follow an exhaustive testing strategy. That is, for each value of x in the list, you want to run the test with all possible values of y from the list.

Write Test

The following test code enumerates all combinations of inputs along with the expected returns from the saturate_add_int function. For greater readability, the code defines each combination of inputs and the associated return value as members of a structure test_param_t. The test_param_t array then enumerates all such combinations.

#include <pstunit.h>

unsigned saturate_add_uint(unsigned x, unsigned y);

struct test_param_t{
    unsigned input1;
    unsigned input2;
    unsigned expected_result;
};

void pst_format_param_test_param_t(char* buff, const pst_size_t param_size, const void* param);

struct test_param_t test_data[] =
/*   input1                   input2               expected_result    */
{{   0,                       0,                     0               },
 {   0,                       UINT_MAX,              UINT_MAX        },
 {   (UINT_MAX/2 - 1),        UINT_MAX/2,            (UINT_MAX - 1)  },
 {   (UINT_MAX/2 - 1),        (UINT_MAX/2 + 1),      UINT_MAX        },
 {   (UINT_MAX/2 - 1),        UINT_MAX,              UINT_MAX        },
 {   UINT_MAX/2,              UINT_MAX,              UINT_MAX        }, 
 {   (UINT_MAX/2 + 1),        (UINT_MAX/2 - 1),      UINT_MAX        },
 {   (UINT_MAX/2 + 1),        UINT_MAX/2,            UINT_MAX        },
 {   UINT_MAX,                UINT_MAX,              UINT_MAX        } 
};
 

PST_PARAM_WITH_VALUES(test_param, struct test_param_t, pst_format_param_test_param_t, 
                test_data, 9);
void pst_format_param_test_param_t(char* buff, const pst_size_t param_size, const void* param) {
    pst_format(buff, param_size, "in1:%u, in2:%u, res:%u",
              (*PST_STATIC_CAST(struct test_param_t*, param)).input1,
              (*PST_STATIC_CAST(struct test_param_t*, param)).input2,
              (*PST_STATIC_CAST(struct test_param_t*, param)).expected_result);
}

PST_SIMPLE_TEST_CONFIG(test_saturate_add_uint) {
     PST_ADD_PARAM(test_param);
}

PST_SIMPLE_TEST_BODY(test_saturate_add_uint) {
    PST_ASSERT_EQ_UINT(saturate_add_uint
                         (
                           (PST_PARAM_PTR(test_param))->input1, 
                           (PST_PARAM_PTR(test_param))->input2
                         ), 
                           (PST_PARAM_PTR(test_param))->expected_result
                      );
}

PST_REGFCN(myRegFcn) {
    PST_ADD_SIMPLE_TEST(test_saturate_add_uint);
}

#ifndef PSTEST_BUILD
int main(int argc, char *argv[]) {
    PST_REGFCN_CALL(myRegFcn);
    return PST_MAIN(argc, argv);
}
#endif

Save this code in a file test.c.

The test consists of these macros from the Polyspace Test xUnit API:

  • PST_PARAM_WITH_VALUES – This macro defines a test parameter, test_param, that has type struct test_param_t, uses the display function pst_format_param_test_param_t, and iterates through the first 9 values in the array test_data.

  • PST_SIMPLE_TEST_CONFIG – This macro defines the test configuration for the test case test_saturate_add_uint.

  • PST_ADD_PARAM – This macro adds the previously defined test parameter test_param to the test configuration for test_saturate_add_uint.

  • PST_SIMPLE_TEST_BODY – This macro defines the test body for test_saturate_add_uint and contains the call to the function under test.

  • PST_ASSERT_EQ_UINT – This macro checks if the first argument is equal to the second. The first argument is the return value of the function under test, saturate_add_uint, with inputs from the test data array, and the second argument contains the corresponding expected result from the same array.

  • PST_PARAM_PTR – This macro allows you to iterate through all values of the parameter test_param.

The remaining macros, PST_REGFCN, PST_ADD_SIMPLE_TEST and PST_REGFCN_CALL, register the test.

This test code also defines a specific display function pst_format_param_test_param_t to display the members of the test parameter structure test_param_t. For more information, see Custom Test Result Display Functions for Complex C/C++ Data Types.

Execute Test and Inspect Results

Compile the files example.c and test.c along with files that ship with Polyspace Test:

gcc example.c test.c <PSTUNIT_SOURCE> -I <PSTUNIT_INCLUDE> -o testrunner
For more information on the file paths <PSTUNIT_SOURCE> and <PSTUNIT_INCLUDE>, see Set Up C/C++ Testing and Code Profiling Using Self-Managed Builds.

Run the test executable:

testrunner.exe
(or ./testrunner in Linux®).

Since the test data contains nine values, you see the results of nine tests. The test data contains some failing values and the results end with this table.

|        |  Total | Passed | Failed | Incomplete
|--------|--------|--------|--------|------------
| Suites |      1 |      0 |      1 |          1
|  Tests |      9 |      6 |      3 |          3
If you look in the output, you can identify which iterations fail the test. For example, iteration 3 with input values of UINT_MAX/2 - 1 and UINT_MAX/2 fails with this message:
| Running         | test       | test_saturate_add_uint(3) with values ps
                   (test_param=in1:2147483646, in2:2147483647)
|     ASSERT FAIL |            |
test.c:45: Assert failed
Expression 'test_condition' evaluated to false
lhs="4294967293" rhs="4294967294"
| FAIL INCOMPLETE | test       | test_saturate_add_uint(3) with values 
                   (test_param=in1:2147483646, in2:2147483647)
Because the value of UINT_MAX is odd, the sum of UINT_MAX/2 - 1 and UINT_MAX/2 is UINT_MAX - 2 (4294967293), which is different from the expected result, UINT_MAX - 1 (4294967294).

Further Exploration

To test the function with a more exhaustive list of inputs, you can use a bigger array of test data. The example, test_exhaustive.c, provides a bigger data set for the test parameter.

See Also

Topics