Generate Variants of Scenario Created from Recorded Sensor Data

Since R2025a

This example uses:

This example shows how to generate scenario variants for a seed scenario created from recorded sensor data from recorded sensor data, such as GPS data, and actor track data.

In the interest of automotive safety, autonomous vehicles must be rigorously tested in real-world scenarios. Testing vehicles on public roads can be expensive and hazardous. In contrast, a virtual scenario, that represents a digital twin of a real-world scenario provides a safe and cost-effective alternative for testing autonomous vehicles. You can use recorded sensor data to generate virtual scenarios that mimic the real-world driving scenarios in which they were recorded, which you can use to enhance the test coverage of automated driving systems [1]. However, some critical driving test cases are rare, and not recorded as scenario sensor data. In this example, you use a virtual scenario, created from recorded sensor data, as a seed scenario to generate scenario variants. The seed scenario used in this example does not contain any collision event, but each generated variant contains a selected collision event. To simulate the scenarios virtually, this example uses the RoadRunner Scenario simulator.

In this example, you:

Set up RoadRunner to visualize the seed scenario and scenario variants.
Extract lane information associated with each actor trajectory.
Identify potential target actors to use to create cut-in collision events.
Generate scenario variants with cut-in collision events.

Set Up RoadRunner

This example requires the Automated Driving Toolbox™ Test Suite for Euro NCAP® Protocols support package. Check if the support package is installed.

helperCheckSupportPackageInstalled

Specify the RoadRunner scene and scenario filenames as I35w.rrscene and ScenarioFromSensorData.rrscenario, respectively, attached to this example as supporting files. The scene file contains the road and lane information, and the scenario file contains the trajectories of multiple actors, including the trajectory of an ego vehicle. The scene and scenario files have been created from recorded Global Positioning System (GPS) data and actor track data.

Note: You can create custom scene and scenario files from recorded sensor data. For more information on how to generate a RoadRunner scene and scenario, see the Generate RoadRunner Scene Using Processed Camera Data and GPS Data example and the Generate RoadRunner Scenario from Recorded Sensor Data example, respectively.

sceneFileName = "I35w.rrscene";
scenarioFileName = "ScenarioFromSensorData.rrscenario";

Check that no active session of RoadRunner is currently open. If a RoadRunner session is already active, then close it by using the close object function of the roadrunner object.

if exist("rrApp","var")
    close(rrApp)
    clear rrApp
end

Start the RoadRunner application interactively by using the roadrunnerSetup function. The function opens a dialog box in which you specify the project folder and installation folder to use when opening RoadRunner. The function returns a roadrunner object, rrApp, that enables you to perform common tasks in the RoadRunner application, such as opening, closing, and saving scenes and projects.

rrApp = roadrunnerSetup;

Copy the scene and scenario files into the Scenes and Scenarios folders, respectively, of your RoadRunner project.

% Get the paths of the Scenes and Scenarios folders in your RoadRunner project. 
rrProjectPath = rrApp.status.Project.Filename;
rrProjectScenesPath = fullfile(rrProjectPath,"Scenes");
rrProjectScenariosPath = fullfile(rrProjectPath,"Scenarios");

% Get the paths of the scene and scenario files to store in
% the Scenes and Scenarios folders of your RoadRunner project
sceneProjectFilePath = fullfile(rrProjectScenesPath,sceneFileName);
scenarioProjectFilePath = fullfile(rrProjectScenariosPath,scenarioFileName);

% Check if your specified scene and scenario files are already in the 
% Scenes and Scenarios folders of your RoadRunner project
checkSceneFilePath = isfile(sceneProjectFilePath);
checkScenarioFilePath = isfile(scenarioProjectFilePath);

if ~(checkSceneFilePath && checkScenarioFilePath)
    % Copy the specified scene and scenario files into the Scenes and Scenarios folders
    % of your RoadRunner project.
    copyfile(fullfile(pwd,sceneFileName),rrProjectScenesPath,"f")
    copyfile(fullfile(pwd,scenarioFileName),rrProjectScenariosPath,"f")
end

Open the ScenarioFromSensorData scenario in RoadRunner Scenario by using the openScenario function.

openScenario(rrApp,scenarioFileName)

Create a RoadRunner Scenario simulation object by using the createSimulation function.

rrSim = createSimulation(rrApp)

Connection status: 1
Connected to RoadRunner Scenario server on localhost:54322, with client id {a251125e-17a9-4c44-8fb7-43c2b80f3df5}

rrSim = 
  ScenarioSimulation with no properties.

Create a structure to reference the RoadRunner simulation information, and specify the simulation step size as 0.2

Visualize the loaded seed scenario, created from the recorded sensor data, in RoadRunner Scenario.

stepSize = 0.2;
set(rrSim,StepSize=stepSize)

Extract Lane Information Associated with Actor Trajectories

Create a ScenarioDescriptor object for the seed scenario in RoadRunner by using the getScenarioDescriptor function.

rrs.SimulatorInstance = rrApp;
rrs.SimulationInstance = rrSim;
seedScenarioDescriptor = getScenarioDescriptor(rrs);

Obtain a RoadRunner HD Map for the scene used in the seed scenario simulation by using the get object function of the ScenarioSimulation object. This function returns a roadrunnerHDMap object that contains the Lanes property, which represents the lane properties of the roads in the scene.

rrHDMap = get(rrSim,"Map");

Extract the trajectory information of the ego vehicle and other actors from the ScenarioDescriptor object, and associate the trajectories with their corresponding lane information from the RoadRunner HD Map, by using the helperAssociateVehicleTrajectoriesWithLanes helper function.

[egoData,targetData] = helperAssociateVehicleTrajectoriesWithLanes(seedScenarioDescriptor,rrHDMap);

Identify Target Actors to Create Cut-In Collision

To create a cut-in collision event in a scenario with no collision instances, you must identify a potential target actor that can collide with the ego vehicle.

Identify all potential target actors for a cut-in collision opportunity with the ego vehicle by using the helperIdentifyCandidateCollisionTargets helper function. The minOverlapTime input argument of the helper function specifies the minimum duration, in seconds, for which the ego vehicle shares either its left lane boundary or its right lane boundary with a target vehicle.

The helper function returns this information for potential target actors:

ActorID — Actor ID, returned as a string scalar.
Name — Name of the actor, returned as a string scalar.
OverlapTime — Overlap time, returned as a scalar. This value indicates the time for which the ego vehicle shares either its left lane boundary or its right lane boundary with the target actor. Units are in seconds.
OverlapEndTime — Overlap end time, returned as a scalar. This value indicates the time at which the target actor and the ego vehicle stop sharing a lane boundary. Units are in seconds.
Route — Trajectory route, returned as a structure that contains the actor trajectory information.

Note: If the helper function does not return appropriate target actors for your scene and scenario, you can lower the value of the minOverlapTime argument to potentially find other target actors that can collide with the ego actor.

minOverlapTime = 15; % seconds
potentialCollisionTargets = helperIdentifyCandidateCollisionTargets(egoData,targetData,stepSize,minOverlapTime)

potentialCollisionTargets=1×4 struct array with fields:
    ActorID
    Name
    OverlapTime
    OverlapEndTime
    Route

Generate Scenario Variants with Cut-In Collision

Create an array of variationProperties objects by using the helperGenerateCutInVariationParameters helper function. Each object in the array contains the variation properties required to create a cut-in collision event in the scenario. This helper function uses these functions:

generateWaypoints — Generates waypoints to perform lane change or lane drift.
varyActorProperties — Adds variations to actor properties to create a collision event.

variationProperties = helperGenerateCutInVariationParameters(seedScenarioDescriptor,potentialCollisionTargets)

variationProperties=1×4 variationProperties array with properties:
    ActorVariationProperties
    CollisionVariationProperties
    SceneVariationProperties
    TestVariationProperties

Specify an index of the variation properties to use to generate a scenario variant with a collision event.

In this example, the variationProperties array contains 4 variation parameters to create scenario variants, each with a collision event. By default, this example displays the last generated scenario variation property. For this example, you can alternatively specify an integer value in the range [1, 4].

variationIndex = numel(variationProperties); 
variationProps = variationProperties(variationIndex)

variationProps = 
  variationProperties with properties:

        ActorVariationProperties: [1×1 struct]
    CollisionVariationProperties: []
        SceneVariationProperties: [1×1 struct]
         TestVariationProperties: [1×1 struct]

Create a variant ScenarioDescriptor object from your seed scenario descriptor by using the helperGenerateVariantsWithModifiedEntryAndExitTime helper function. The helper function creates a scenario variant descriptor by using the obtained variation parameters, which also modifies the EntryTime and ExitTime values of the target actors to ensure collision in the generated scenario variant.

VariantDescriptor = helperGenerateVariantsWithModifiedEntryAndExitTime(seedScenarioDescriptor,variationProps);

Update the RoadRunner Scenario visualization and the roadrunner object rrApp by using the getScenario function.

getScenario(VariantDescriptor,Simulator="RoadRunner",SimulatorInstance=rrApp);

To view the scenario from the ego vehicle view or chase view, in the RoadRunner Simulation pane, in the Camera section, set Camera View to Follow or Front. Note that the default value of the Actor attribute is VUT, which is the ego vehicle for the scenario in this example.

This figure shows a comparison of a scenario simulation of the seed scenario and of the variant scenario in RoadRunner. Observe that the seed scenario does not contain a collision event, whereas the generated variant contains a cut-in collision event. In the variant scenario, the collision occurs between the gray sedan ego vehicle and the gray pick-up truck target vehicle. Note that the simulations of both the scenarios are clipped to a timestamp range around the collision event of the generated variant to create the figure for visual comparison of the scenarios.

Comparison of scenario simulation of the seed scenario and the variant scenario in RoadRunner.

To stop and close the RoadRunner simulation, uncomment and execute this code.

% close(rrApp)

Summary

In this example, you have explored how to generate scenario variants with collision from a seed scenario created using recorded sensor data such as GPS data and actor track data. You can also use an actor track list generated from camera or lidar data to generate scenarios. For more information, see the Extract Vehicle Track List from Recorded Camera Data for Scenario Generation and Extract Vehicle Track List from Recorded Lidar Data for Scenario Generation examples.

References

[1] The MathWorks. "Automotive Research Association of India Enables Virtual Testing of ADAS Application with Real-World Simulation Scenarios." MathWorks Technical Articles. The MathWorks, 2023. https://www.mathworks.com/company/technical-articles/automotive-research-association-of-india-enables-virtual-testing-of-adas-application-with-real-world-simulation-scenarios.html.