padv.builtin.task.MergeTestResults Class

Namespace: padv.builtin.task
Superclasses: padv.Task

Task for generating consolidated test results report and merged coverage reports

Description

The padv.builtin.task.MergeTestResults class provides a task that can generate a consolidated test results report and merged coverage reports using Simulink^® Test™ and Simulink Coverage™. The task can generate the following artifacts for a model:

• a consolidated test results report

• a merged model coverage report for normal mode simulation results

• a merged code coverage report for software-in-the-loop (SIL) mode results

• a merged code coverage report for processor-in-the-loop (PIL) mode results

You can run your tests using the built-in task padv.builtin.task.RunTestsPerTestCase and then generate the reports using the MergeTestResults task. You can add these tasks to your process model by using the method addTask. After you add the tasks to your process model, you can run the tasks from the Process Advisor app or by using the function runprocess.

Alternatively, you can run your tests using the built-in task padv.builtin.task.RunTestsPerModel, but to generate the consolidated test results report and merged coverage report you need to reconfigure the MergeTestResults task.

To view the source code for this built-in task, in the MATLAB^® Command Window, enter:

open padv.builtin.task.MergeTestResults

The padv.builtin.task.MergeTestResults class is a handle class.

Creation

Description

task = padv.builtin.task.MergeTestResults() creates a task for generating a consolidated test results report and merged coverage reports using Simulink Test and Simulink Coverage.

task = padv.builtin.task.MergeTestResults(PredecessorTask=PredecessorTask) specifies that the task can only execute after the specified predecessor task runs.

When you specify a predecessor task, you must also specify a dependsOn relationship between the MergeTestResults task and the predecessor task in your process model.

task = padv.builtin.task.MergeTestResults(Name=Value) sets certain properties using one or more name-value arguments. For example, task = padv.builtin.task.MergeTestResults(Name = "MyTestAndCoverageReportsTask") creates a task with the specified name.

You can use this syntax to set property values for Name, Title, IterationQuery, InputQueries, InputDependencyQuery, or Licenses.

The padv.builtin.task.MergeTestResults class also has other properties, but you cannot set those properties during task creation.

Input Arguments

PredecessorTask — Task that must run before this task

"padv.builtin.task.RunTestsPerTestCase" (default) | string | padv.Task object | array of padv.Task objects

Task that must run before this task, specified as either the:

• Task name, specified as a string.

• Task instance, specified as a padv.Task object or an array of padv.Task objects.

By default, MergeTestResults requires outputs from a task named "padv.builtin.task.RunTestsPerTestCase". If your test task has a different task name, you can use the PredecessorTask argument to provide the task name or instance to the MergeTestResults task.

When you specify a predecessor task, you must also specify in the process model that the MergeTestResults task dependsOn the predecessor task.

Example: "MyRunTestsTask"

Example: [milTask, silTask]

Properties

The MergeTestResults class inherits properties from padv.Task. The properties listed in Specialized Inherited Properties are padv.Task properties that the MergeTestResults task overrides.

The task also has properties for specifying:

• Test Result Options

• Coverage Report Options

Specialized Inherited Properties

Name — Unique identifier for task in process

"padv.builtin.task.MergeTestResults" (default) | string

Unique identifier for task in process, specified as a string.

Example: "MyTestAndCoverageReportsTask"

Data Types: string

Title — Human-readable name that appears in Process Advisor app

"Merge Test Results" (default) | string

Human-readable name that appears in Process Advisor app, specified as a string.

Example: "My Test And Coverage Reports Task"

Data Types: string

DescriptionText — Task description

"This task uses Simulink Test and Simulink Coverage to generate a consolidated test results report and a merged coverage report for a model." (default) | string

Task description, specified as a string.

When you point to a task in Process Advisor and click the information icon, the tooltip shows the task description.

Example: "This task uses Simulink Test and Simulink Coverage to generate a consolidated test results report and a merged coverage report for a model."

Data Types: string

DescriptionCSH — Path to task documentation

path to MergeTestResults documentation (default) | string

Path to task documentation, specified as a string.

When you point to a task in Process Advisor, click the ellipsis (...), and click Help, Process Advisor opens the task documentation.

Example: fullfile(pwd,"taskHelpFiles","myTaskDocumentation.pdf")

Data Types: string

RequiredIterationArtifactType — Artifact type that task can run on

"sl_model_file" (default) | ...

Type of artifact, specified as one or more of the values listed in this table. To specify multiple values, use an array.

Category	Artifact Type	Description
MATLAB	"m_class"	MATLAB class
	"m_file"	MATLAB file
	"m_func"	MATLAB function
	"m_method"	MATLAB class method
	"m_property"	MATLAB class property
Model Advisor	"ma_config_file"	Model Advisor configuration file
	"ma_justification_file"	Model Advisor justification file
Model Finder	"mf_database"	Model Finder database file
Process Advisor	"padv_dep_artifacts"	Related artifacts that current artifact depends on
	"padv_output_file"	Process Advisor output file
Project	"project"	Current project file
Requirements	"mwreq_file"	Requirement file (since R2024b)
	"mwreq_item"	Requirement (since R2024b)
	"sl_req"	Requirement (for R2024a and earlier)
	"sl_req_file"	Requirement file (for R2024a and earlier)
	"sl_req_table"	Requirements Table
Stateflow®	"sf_chart"	Stateflow chart
	"sf_graphical_fcn"	Stateflow graphical function
	"sf_group"	Stateflow group
	"sf_state"	Stateflow state
	"sf_state_transition_chart"	Stateflow state transition chart
	"sf_truth_table"	Stateflow truth table
Simulink	"sl_block_diagram"	Block diagram
	"sl_data_dictionary_file"	Data dictionary file
	"sl_embedded_matlab_fcn"	MATLAB function
	"sl_block_diagram"	Block diagram
	"sl_library_file"	Library file
	"sl_model_file"	Simulink model file
	"sl_protected_model_file"	Protected Simulink model file
	"sl_subsystem"	Subsystem
	"sl_subsystem_file"	Subsystem file
System Composer™	"zc_block_diagram"	System Composer architecture
	"zc_component"	System Composer architecture component
	"zc_file"	System Composer architecture file
Tests	"harness_info_file"	Harness info file
	"sl_harness_block_diagram"	Harness block diagram
	"sl_harness_file"	Test harness file
	"sl_test_case"	Simulink Test case
	"sl_test_case_result"	Simulink Test case result
	"sl_test_file"	Simulink Test file
	"sl_test_iteration"	Simulink Test iteration
	"sl_test_iteration_result"	Simulink Test iteration result
	"sl_test_report_file"	Simulink Test result report
	"sl_test_result_file"	Simulink Test result file
	"sl_test_resultset"	Simulink Test result set
	"sl_test_seq"	Test Sequence
	"sl_test_suite"	Simulink Test suite
	"sl_test_suite_result"	Simulink Test suite result

Example: "sl_model_file"

Example: ["sl_model_file "zc_file"]

IterationQuery — Find artifacts that task iterates over

padv.builtin.query.FindModelsWithTestCases (default) | padv.Query object | name of padv.Query object

Query that finds the artifacts that the task iterates over, specified as a padv.Query object or the name of a padv.Query object. When you specify IterationQuery, the task runs one time for each artifact returned by the query. In the Process Advisor app, the artifacts returned by IterationQuery appear under task title.

For more information about task iterations, see Overview of Process Model.

InputDependencyQuery — Finds artifact dependencies for task inputs

padv.builtin.query.GetDependentArtifacts(IncludeRequirementLinkFiles=true) (default) | padv.Query object | name of padv.Query object

Query that finds artifact dependencies for task inputs, specified as a padv.Query object or the name of a padv.Query object.

The build system runs the query specified by InputDependencyQuery to find the dependencies for the task inputs, since those dependencies can impact if task results are up-to-date. For more information, see Overview of Process Model.

Example: padv.builtin.query.GetDependentArtifacts

Licenses — List of licenses that task requires

"simulink_test" (default) | string

List of licenses that the task requires, specified as a string.

Data Types: string

CISupportOutputsByTask — Type of CI-compatible result files that task generates when run

"JUnit" (default)

Type of CI-compatible result files that the task itself generates when run, specified as either:

• "JUnit" — JUnit-style XML report for task results.

• "" — None. The build system generates a JUnit-style XML report for the task instead.

InputQueries — Inputs to task

[padv.builtin.query.GetIterationArtifact, padv.builtin.query.GetOutputsOfDependentTask(Task="padv.builtin.task.RunTestsPerTestCase")] (default) | padv.Query object | name of padv.Query object | array of padv.Query objects

Inputs to the task, specified as:

• a padv.Query object

• the name of padv.Query object

• an array of padv.Query objects

• an array of names of padv.Query objects

By default, if the task iterates over the project file and does not specify input queries, the task gets the outputs from the predecessor task, PredecessorTask, and uses those as task inputs. If the task iterates over model files, the task uses the built-in query padv.builtin.query.GetIterationArtifact to use the models as task inputs.

OutputDirectory — Location for standard task outputs

fullfile('$DEFAULTOUTPUTDIR$', '$ITERATIONARTIFACT$' , 'test_results') (default) | string

Location for standard task outputs, specified as a string.

The built-in tasks use tokens, like $DEFAULTOUTPUTDIR$, as placeholders for dynamic path resolution during run-time. For more information, see Dynamically Resolve Paths with Tokens.

Data Types: string

Test Result Options

Author — Name of report author

"Process Advisor" (default) | string

Name of the report author, specified as a string.

Example: "My Team Name"

Data Types: string

IncludeComparisonSignalPlots — Include signal comparison plots

false or 0 (default) | true or 1

Include the signal comparison plots defined under baseline criteria, equivalence criteria, or assessments using the verify operator in the test case, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

IncludeCoverageResult — Include coverage metrics collected at test execution

true or 1 (default) | false or 0

Include coverage metrics that are collected at test execution, specified as a numeric or logical 1 (true) or 0 (false).

Example: false

Data Types: logical

IncludeErrorMessages — Include error messages from test case simulations

true or 1 (default) | false or 0

Include error messages from the test case simulations, specified as a numeric or logical 1 (true) or 0 (false).

Example: false

Data Types: logical

IncludeMATLABFigures — Include figures in report

false or 0 (default) | true or 1

Include the figures opened from a callback script, custom criteria, or by the model in the report, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

IncludeMLVersion — Include MATLAB version that ran test cases

true or 1 (default) | false or 0

Include the version of MATLAB used to run the test cases, specified as a numeric or logical 1 (true) or 0 (false).

Example: false

Data Types:

IncludeSimulationMetadata — Include simulation metadata for each test case or iteration

false or 0 (default) | true or 1

Include simulation metadata for each test case or iteration, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

IncludeSimulationSignalPlots — Include simulation output plots of each signal

false or 0 (default) | true or 1

Include the simulation output plots of each signal, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

IncludeTestRequirement — Include test requirement link

true or 1 (default) | false or 0

Include the test requirement link defined under Requirements in the test case, specified as a numeric or logical 1 (true) or 0 (false).

Example: false

Data Types: logical

IncludeTestResults — Include all or subset of test results in report

0 (default) | 1 | 2

Include all or a subset of test results in the report, specified as either:

• 0 — Passed and failed results

• 1 — Only passed results

• 2 — Only failed results

Example: 2

LaunchReport — Open generated report

false or 0 (default) | true or 1

Open the generated report, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

LoadSimulationSignalData — Task loads simulation signal data when loading test results

false or 0 (default) | true or 1

Task loads simulation signal data when loading test results, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

NumPlotColumnsPerPage — Number of columns of plots to include on report pages

1 (default) | 2 | 3 | 4

Number of columns of plots to include on report pages, specified as an integer 1, 2, 3, or 4.

Example: 2

NumPlotRowsPerPage — Number of rows of plots to include on report pages

2 (default) | 1 | 3 | 4

Number of rows of plots to include on report pages, specified as an integer 1, 2, 3, or 4.

Example: 1

ReportFormat — Output format for generated report

"pdf" (default) | "docx" | "zip"

Output format for the generated report, specified as either:

• "pdf" — PDF format

• "docx" — Microsoft^® Word document format

• "zip" — Zipped file

Example: "zip"

ReportPath — Path to generated report

fullfile('$DEFAULTOUTPUTDIR$', '$ITERATIONARTIFACT$' , 'test_results') (default) | string

Path to the generated report, specified as a string.

The built-in tasks use tokens, like $DEFAULTOUTPUTDIR$, as placeholders for dynamic path resolution during run-time. For more information, see Dynamically Resolve Paths with Tokens.

Data Types: string

ReportName — File name for generated report

"$ITERATIONARTIFACT$_Test_Report" (default) | string

File name for the generated report, specified as a string.

The built-in tasks use tokens, like $ITERATIONARTIFACT$, as placeholders for dynamic path resolution during run-time. For more information, see Dynamically Resolve Paths with Tokens.

Data Types: string

ReportTitle — Title of report

"$ITERATIONARTIFACT$ TEST REPORT" (default) | string

Title of the report, specified as a string.

The built-in tasks use tokens, like $ITERATIONARTIFACT$, as placeholders for dynamic path resolution during run-time. For more information, see Dynamically Resolve Paths with Tokens.

Data Types: string

Coverage Report Options

CovAllTestInMdlSumm — Include each test in model summary

false or 0 (default) | true or 1

Include each test in the model summary, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

CovBarGrInMdlSumm — Produce bar graphs in model summary

true or 1 (default) | false or 0

Produce bar graphs in the model summary, specified as a numeric or logical 1 (true) or 0 (false).

Example: false

Data Types: logical

CovComplexInBlkTable — Include cyclomatic complexity numbers in block details

true or 1 (default) | false or 0

Include cyclomatic complexity numbers in block details, specified as a numeric or logical 1 (true) or 0 (false).

Example: false

Data Types: logical

CovComplexInSumm — Include cyclomatic complexity numbers in summary

true or 1 (default) | false or 0

Include cyclomatic complexity numbers in summary, specified as a numeric or logical 1 (true) or 0 (false).

Example: false

Data Types: logical

CovElimFullCov — Exclude fully covered model objects from report

false or 0 (default) | true or 1

Exclude fully covered model objects from report, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

CovElimFullCovDetails — Exclude fully covered model object details from report

true or 1 (default) | false or 0

Exclude fully covered model object details from report, specified as a numeric or logical 1 (true) or 0 (false).

Example: false

Data Types: logical

CovFiltExecMetric — Filter Execution metric from report

false or 0 (default) | true or 1

Filter Execution metric from report, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

CovFiltSFEvent — Filter Stateflow events from report

false or 0 (default) | true or 1

Filter Stateflow events from report, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

CovGenerateWebViewReport — Generate web view report

false or 0 (default) | true or 1

Generate web view report, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

CovHitCntInMdlSumm — Display hit/count ratio in model summary

false or 0 (default) | true or 1

Display hit/count ratio in the model summary, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

CovReportName — Name of generated model coverage report

"$ITERATIONARTIFACT$_ModelCoverage_Report.html" (default) | string

Name of generated model coverage report, specified as a string. The report is an aggregated coverage report for normal simulation mode results.

The built-in tasks use tokens, like $ITERATIONARTIFACT$, as placeholders for dynamic path resolution during run-time. For more information, see Dynamically Resolve Paths with Tokens.

Example: "myModel_NormalModelCoverage_Report.html"

Data Types: string

CovReportNameSIL — Name of generated SIL code coverage report

"$ITERATIONARTIFACT$_SIL_CodeCoverage_Report.html" (default) | string

Name of generated software-in-the-loop (SIL) code coverage report, specified as a string. The report is an aggregated coverage report for SIL mode results.

The built-in tasks use tokens, like $ITERATIONARTIFACT$, as placeholders for dynamic path resolution during run-time. For more information, see Dynamically Resolve Paths with Tokens.

Example: "myModel_SIL_CodeCoverage_Report.html"

Data Types: string

CovReportNamePIL — Name of generated (PIL) code coverage report

"$ITERATIONARTIFACT$_PIL_CodeCoverage_Report.html" (default) | string

Name of generated processor-in-the-loop (PIL) code coverage report, specified as a string. The report is an aggregated coverage report for PIL mode results.

The built-in tasks use tokens, like $ITERATIONARTIFACT$, as placeholders for dynamic path resolution during run-time. For more information, see Dynamically Resolve Paths with Tokens.

Example: "myModel_PIL_CodeCoverage_Report.html"

Data Types: string

CovShowReport — Show coverage report

false or 0 (default) | true or 1

Show coverage report, specified as a numeric or logical 1 (true) or 0 (false).

Example: true

Data Types: logical

CovTwoColorBarGraphs — Use two-color bar graphs

true or 1 (default) | false or 0

Use two-color bar graphs, specified as a numeric or logical 1 (true) or 0 (false).

Example: false

Data Types: logical

Methods

Specialized Public Methods

This class overrides the following inherited methods.

run

Generate a consolidated test results report and merged coverage reports using Simulink Test and Simulink Coverage Note You do not need to manually invoke this method. When you run a task using the Process Advisor app or the runprocess function, the build system automatically invokes the run method for the task. The run method inside this built-in task runs on a task object obj with task input input and returns a task result taskResult. The task result is a padv.TaskResult object that can store the pass, fail, and error information. If you inherit from this built-in task, make sure to use the same method signature inside your custom task. function taskResult = run(obj,input) ... end

dryRun

Dry run the task to validate task inputs and generate representative task outputs without actually running the task. The dryRun method inside this built-in task runs on a task object obj with task input input and returns a task result taskResult. The task result is a padv.TaskResult object that can store the pass, fail, and error information. If you inherit from this class, make sure to use the same method signature inside your custom task. function taskResult = dryRun(obj, input) ... end For more information, see Dry Run Tasks to Test Process Model.

Examples

Add Merge Test Results Task to Process

Add a task that can generate a consolidated test results report and merged coverage reports for models in your project.

Open the process model for your project. If you do not have a process model, open the Process Advisor app to automatically create a process model.

In the process model file, add the GenerateSimulinkWebView task to your process model by using the addTask method.

mergeTestTask = pm.addTask(padv.builtin.task.MergeTestResults);

You can reconfigure the task behavior by using the task properties. For example, to change where the consolidated test results report and merged model coverage report generate:

defaultTestResultPath = fullfile('$DEFAULTOUTPUTDIR$','test_results');
mergeTestTask.ReportPath = defaultTestResultPath;
mergeTestTask.CovReportPath = defaultTestResultPath;

The MergeTestResults task requires outputs from the RunTestsPerTestCase task. Specify this dependency in your process model by using the dependsOn method.

To make sure that you run your tasks using the built-in task RunTestsPerTestCase before you add the MergeTestResults task to the process model, you can use conditional logic in your process model. For example:

includeTestsPerTestCaseTask = true;
includeMergeTestResultsTask = true;

%% Run tests per test case
% Tools required: Simulink Test
if includeTestsPerTestCaseTask
    milTask = pm.addTask(padv.builtin.task.RunTestsPerTestCase);
    % ... Optionally specify task property values
end

%% Merge test results
% Tools required: Simulink Test (and optionally Simulink Coverage)
if includeTestsPerTestCaseTask && includeMergeTestResultsTask
    mergeTestTask = pm.addTask(padv.builtin.task.MergeTestResults);
    % ... Optionally specify task property values
end

% Set task dependencies
if includeTestsPerTestCaseTask && includeMergeTestResultsTask
    mergeTestTask.dependsOn(milTask,WhenStatus=["Pass","Fail"]);
end

This code specifies that the MergeTestResults task runs when the status of the RunTestsPerTestCase task is either a passing or failing result. If you only want to run the MergeTestResults when the RunTestsPerTestCase task passes, you can specify WhenStatus as "Pass" instead. For more information, see dependsOn.

Tips

• Run your tests by using either the built-in task padv.builtin.task.RunTestsPerModel or padv.builtin.task.RunTestsPerTestCase.

• If you are using the pipeline generator, set the StopOnStageFailure property of your pipeline generator options object to false so that the pipeline collects and publishes results, even if earlier stages fail.