Calculate Code Coverage of C/C++ Code Using Python API for Polyspace

You can manage Polyspace® Test™ workflows entirely at the command line by using the Python® API for Polyspace. This example demonstrates the workflow for using the API to:

  • Create a project.

  • Add source files and xUnit test files to the project.

  • Build and run tests in the project with code coverage computation enabled.

Prerequisites

Make sure you are using a supported Python version and you are able to import the polyspace.project and polyspace.test Python modules without errors. For more information, see Set Up Python API for Polyspace.

Import Modules

Import the modules polyspace.project and polyspace.test:

import polyspace.project
import polyspace.test
import os
For more information, see Set Up Python API for Polyspace.

Create Project and Add Files

To create a project, use the polyspace.project module.

examples_path = os.path.join(polyspace.__install_path__, "polyspace", 
                            "examples", "pstest", "Getting_Started_Example")
polyspaceProject = polyspace.project.Project("getStarted")
The command creates a polyspace.project.Project object polyspaceProject and a project file getStarted.psprjx in the current folder.

Use the properties of polyspaceProject to add source and xUnit test files to the project.

# Add source code files
polyspaceProject.Code.Files.add(os.path.join(examples_path, "sources", "utils.c"))
polyspaceProject.IncludePaths.add(os.path.join(examples_path, "includes"))

# Add test file
polyspaceProject.Tests.Files.add(os.path.join(examples_path, "tests", "test.c"))

Set Coverage Level in Project Configuration

Set the coverage metric level to MCDC in the active test configuration of the project:

coverageMetricLevel = polyspace.project.CoverageMetricLevel.MCDC
polyspaceProject.ActiveTestConfiguration.CoverageOptions.Level = coverageMetricLevel
The other possible values instead of MCDC are STATEMENT, DECISION, CONDITION_DECISION, or NONE. For an explanation of the coverage metric levels, see Coverage metrics (-cov-metric-level).

Run Tests with Coverage Enabled

Run the tests added to the project with code coverage computation enabled.

res = polyspace.test.run(
      polyspaceProject,
      ProfilingSelection=polyspace.test.ProfilingSelection.COVERAGE
)

See Overview of Coverage Results

Extract the coverage results and view which coverage metric passes a specific threshold, for example, 50%.

# Read coverage results
profilingResults = res.Profiling
coverageResults = profilingResults.Coverage

coverage_metrics = ["decision","condition","mcdc","statement","function call", \
"function", "function exit"]

# Show pass/fail status of coverage results based on coverage percentage
print("Coverage Threshold is 50%")
for metric in coverage_metrics:
  metric_details = coverageResults.getCoverageInfo(metric, "utils.c")
  if metric_details.TotalCount:
    pct = (metric_details.CoveredCount + metric_details.JustifiedCount) / metric_details.TotalCount
    print(f"{metric} coverage is {pct:.2%}\n ", " (FAIL)" if pct < 0.5 else " (PASS)")
  else:
    print(f"{metric} coverage not calculated\n")

If you use the example source and test files, you see an output like this:

decision coverage is 50.00%
   (PASS)
condition coverage is 40.00%
   (FAIL)
mcdc coverage is 10.00%
   (FAIL)
statement coverage is 57.14%
   (PASS)
function call coverage is 100%
   (PASS)
function coverage is 66.67%
   (PASS)
function exit coverage is 42.86%
   (FAIL)

Add Justification for Unreachable Branch in Coverage Results

In the source file utils.c, the function isGreaterThanSpeedLimit contains an unreachable branch because the if condition in this function is always true. Inspect the decision coverage results for this function.

result_isGreaterThanSpeedLimit = coverageResults.getCoverageInfo("decision", "utils.c","isGreaterThanSpeedLimit")
print(result_isGreaterThanSpeedLimit)

Only one of the two decision outcomes of the if statement is covered by the tests.

{
    TotalCount: 2
    CoveredCount: 1
    JustifiedCount: 0
    Details: [<DecisionCoverageDetail>]
}

Define a filter to justify all missing code coverage results in the isGreaterThanSpeedLimit function. Apply this filter to the coverageResults object to obtain a filtered object filteredCoverageResults.

filters = polyspace.test.CoverageFilters()
filters.Rules.createForFunction("isGreaterThanSpeedLimit", "utils.c", Rationale="Unreachable code", Status="justified")
filteredCoverageResults = coverageResults.applyFilters(filters)

Note

Instead of using the createForFunction method, you can also use:

  • The createForFile method to create a rule that applies to an entire file

  • The create method to create a rule that filters a specific decision or a specific decision outcome

For more information on these methods, see the description of the Rules property of the polyspace.test.CoverageFilters class.

Inspect the filtered decision coverage results for the isGreaterThanSpeedLimit function.

filteredResult_isGreaterThanSpeedLimit = filteredCoverageResults.getCoverageInfo("decision", "utils.c","isGreaterThanSpeedLimit")
print(filteredResult_isGreaterThanSpeedLimit)

The previously uncovered decision outcome has now been marked as justified.

{
    TotalCount: 2
    CoveredCount: 1
    JustifiedCount: 1
    Details: [<DecisionCoverageDetail>]
}