Main Content

Change Parameters and Coverage Goals of Scoreboard in UVM Testbench

This example shows how to control the parameters of the response checker in a UVM testbench generated from Simulink®. Such parameterization helps the design verification engineer reuse the scoreboard under different testing scenarios. Simulink parameters to the checker will result in a configuration object whose values can be set in a derived test class (randomized or not) or directly via a command line plusarg.

For a description of the design and the background on generating a UVM testbench, see the example Generate Parameterized UVM Testbench from Simulink.

Generate UVM Testbench

First, open the project and model.

openProject('pulsedetector_proj');
model = 'pulsedetector_tb';
open_system(model);

To generate the default UVM testbench for this example, execute:

design     = [model '/PulseDetector'];
sequence   = [model '/GenPulse'];
scoreboard = [model '/CheckDetection'];
predictor  = [model '/PulseDetectorRef'];
driver     = [model '/InputDriver'];
monitor    = [model '/OutputMonitor'];
uvmbuild(design, sequence, scoreboard, Predictor=predictor, Driver=driver, Monitor=monitor);
### Starting DPI subsystem generation for UVM test bench
### Starting build procedure for model: PulseDetector
### Starting SystemVerilog DPI Component Generation
### Generating DPI H Wrapper PulseDetector_dpi.h
### Generating DPI C Wrapper PulseDetector_dpi.c
### Generating UVM module package PulseDetector_dpi_pkg.sv
### Generating SystemVerilog module PulseDetector_dpi.sv
### Generating makefiles for: PulseDetector_dpi
### Invoking make to build the DPI Shared Library
### Successful completion of build procedure for model: PulseDetector

Build Summary

Top model targets built:

Model          Action                        Rebuild Reason                                    
===============================================================================================
PulseDetector  Code generated and compiled.  Code generation information file does not exist.  

1 of 1 models built (0 models already up to date)
Build duration: 0h 0m 58.21s
### Starting build procedure for model: GenPulse
### Starting SystemVerilog DPI Component Generation
### Generating DPI H Wrapper GenPulse_dpi.h
### Generating DPI C Wrapper GenPulse_dpi.c
### Generating UVM module package GenPulse_dpi_pkg.sv
### Generating SystemVerilog module GenPulse_dpi.sv
### Generating makefiles for: GenPulse_dpi
### Invoking make to build the DPI Shared Library
### Successful completion of build procedure for model: GenPulse

Build Summary

Top model targets built:

Model     Action                        Rebuild Reason                                    
==========================================================================================
GenPulse  Code generated and compiled.  Code generation information file does not exist.  

1 of 1 models built (0 models already up to date)
Build duration: 0h 0m 31.474s
### Starting build procedure for model: InputDriver
### Starting SystemVerilog DPI Component Generation
### Generating DPI H Wrapper InputDriver_dpi.h
### Generating DPI C Wrapper InputDriver_dpi.c
### Generating UVM module package InputDriver_dpi_pkg.sv
### Generating SystemVerilog module InputDriver_dpi.sv
### Generating makefiles for: InputDriver_dpi
### Invoking make to build the DPI Shared Library
### Successful completion of build procedure for model: InputDriver

Build Summary

Top model targets built:

Model        Action                        Rebuild Reason                                    
=============================================================================================
InputDriver  Code generated and compiled.  Code generation information file does not exist.  

1 of 1 models built (0 models already up to date)
Build duration: 0h 0m 22.829s
### Starting build procedure for model: OutputMonitor
### Starting SystemVerilog DPI Component Generation
### Generating DPI H Wrapper OutputMonitor_dpi.h
### Generating DPI C Wrapper OutputMonitor_dpi.c
### Generating UVM module package OutputMonitor_dpi_pkg.sv
### Generating SystemVerilog module OutputMonitor_dpi.sv
### Generating makefiles for: OutputMonitor_dpi
### Invoking make to build the DPI Shared Library
### Successful completion of build procedure for model: OutputMonitor

Build Summary

Top model targets built:

Model          Action                        Rebuild Reason                                    
===============================================================================================
OutputMonitor  Code generated and compiled.  Code generation information file does not exist.  

1 of 1 models built (0 models already up to date)
Build duration: 0h 0m 22.874s
### Starting build procedure for model: CheckDetection
### Starting SystemVerilog DPI Component Generation
### Generating DPI H Wrapper CheckDetection_dpi.h
### Generating DPI C Wrapper CheckDetection_dpi.c
### Generating UVM module package CheckDetection_dpi_pkg.sv
### Generating SystemVerilog module CheckDetection_dpi.sv
### Generating makefiles for: CheckDetection_dpi
### Invoking make to build the DPI Shared Library
### Successful completion of build procedure for model: CheckDetection

Build Summary

Top model targets built:

Model           Action                        Rebuild Reason                                    
================================================================================================
CheckDetection  Code generated and compiled.  Code generation information file does not exist.  

1 of 1 models built (0 models already up to date)
Build duration: 0h 0m 39.196s
### Starting build procedure for model: PulseDetectorRef
### Starting SystemVerilog DPI Component Generation
### Generating DPI H Wrapper PulseDetectorRef_dpi.h
### Generating DPI C Wrapper PulseDetectorRef_dpi.c
### Generating UVM module package PulseDetectorRef_dpi_pkg.sv
### Generating SystemVerilog module PulseDetectorRef_dpi.sv
### Generating makefiles for: PulseDetectorRef_dpi
### Invoking make to build the DPI Shared Library
### Successful completion of build procedure for model: PulseDetectorRef

Build Summary

Top model targets built:

Model             Action                        Rebuild Reason                                    
==================================================================================================
PulseDetectorRef  Code generated and compiled.  Code generation information file does not exist.  

1 of 1 models built (0 models already up to date)
Build duration: 0h 0m 23.232s
### Starting UVM test bench generation for model: pulsedetector_tb
### Generating UVM transaction object ./uvm_build/pulsedetector_tb_uvm_testbench/scoreboard/mw_PulseDetector_scoreboard_trans.sv
### Generating UVM interface ./uvm_build/pulsedetector_tb_uvm_testbench/uvm_artifacts/mw_PulseDetector_if.sv
### Generating UVM sequence ./uvm_build/pulsedetector_tb_uvm_testbench/sequence/mw_PulseDetector_sequence.sv
### Generating UVM sequencer ./uvm_build/pulsedetector_tb_uvm_testbench/sequence/mw_PulseDetector_sequencer.sv
### Generating UVM sequence transaction ./uvm_build/pulsedetector_tb_uvm_testbench/sequence/mw_PulseDetector_sequence_trans.sv
### Generating UVM driver ./uvm_build/pulsedetector_tb_uvm_testbench/driver/mw_PulseDetector_driver.sv
### Generating UVM monitor ./uvm_build/pulsedetector_tb_uvm_testbench/monitor/mw_PulseDetector_monitor.sv
### Generating UVM input monitor ./uvm_build/pulsedetector_tb_uvm_testbench/uvm_artifacts/mw_PulseDetector_monitor_input.sv
### Generating UVM Reference Model ./uvm_build/pulsedetector_tb_uvm_testbench/predictor/mw_PulseDetector_predictor.sv
### Generating UVM transaction object ./uvm_build/pulsedetector_tb_uvm_testbench/predictor/mw_PulseDetector_predictor_trans.sv
### Generating UVM agent ./uvm_build/pulsedetector_tb_uvm_testbench/uvm_artifacts/mw_PulseDetector_agent.sv
### Generating UVM scoreboard ./uvm_build/pulsedetector_tb_uvm_testbench/scoreboard/mw_PulseDetector_scoreboard.sv
### Generating UVM scoreboard configuration object ./uvm_build/pulsedetector_tb_uvm_testbench/scoreboard/mw_PulseDetector_scoreboard_cfg_obj.sv
### Generating UVM environment ./uvm_build/pulsedetector_tb_uvm_testbench/uvm_artifacts/mw_PulseDetector_environment.sv
### Generating UVM test ./uvm_build/pulsedetector_tb_uvm_testbench/uvm_artifacts/mw_PulseDetector_test.sv
### Generating UVM top ./uvm_build/pulsedetector_tb_uvm_testbench/top/mw_PulseDetector_top.sv
### Generating UVM test package ./uvm_build/pulsedetector_tb_uvm_testbench/top/pulsedetector_tb_pkg.sv
### Generating UVM test bench simulation script for Mentor Graphics QuestaSim/Modelsim ./uvm_build/pulsedetector_tb_uvm_testbench/top/run_tb_mq.do
### Generating UVM test bench simulation script for Cadence Xcelium ./uvm_build/pulsedetector_tb_uvm_testbench/top/run_tb_xcelium.sh
### Generating UVM test bench simulation script for Synopsys VCS ./uvm_build/pulsedetector_tb_uvm_testbench/top/run_tb_vcs.sh

Scoreboard Parameterization

In the model, the response checking is parameterized by an error threshold variable used in an assertion checking block. It is designated as a parameter to retain in the SystemVerilog model by using a Simulink.Parameter element, pErrorThreshold. In that specification, it is given a default value of 5.0 percent, which reflects the tolerance of error difference between the golden reference and the actual DUT. (In this testbench, the discrepancy is due to double-precision versus fixed-point implementations of the detection logic.)

scoreboard block mask showing that the upper bound parameter is set to pErrorThreshold.

In the generated UVM code, the parameter is placed in a configuration object, mw_PulseDetector_scoreboard_cfg_obj.

See mw_PulseDetector_scoreboard_cfg_obj.sv.

ScoreboardCfgObj.png

The mw_PulseDetector_scoreboard_cfg_obj configuration object is instantiated in the build phase of the test, mw_PulseDetector_test:

See mw_PulseDetector_test.sv.

ScoreboardTestBuildPhase.png

Set the mw_PulseDetector_scoreboard_cfg_obj configuration object via the uvm_config_db database in the scoreboard, mw_PulseDetector_scoreboard, in its start_of_simulation_phase function.

See mw_PulseDetector_scoreboard.sv.

ScoreboardStartSimPhase.png

Updating the Error Threshold Using a Plus Argument

The test plan originally called for a tolerance of 5.0 percent. If later, the design specification was tightened to 1.0 percent to reflect a more safety critical requirement, this can be done using the command line. The support scripts use an environment variable to affect the SystemVerilog command line plus argument.

The provided script includes a MathWorks-specific set up for the environment to run Questa®. Adjust the commands to match your installation or choose to use a different simulator.

current_simulator = 'Questa';
setup_questa();

Next, clear the environment variables that influence the UVM simulation.

setenv EXTRA_UVM_SIM_ARGS
setenv EXTRA_UVM_COMP_ARGS
setenv UVM_TOP_MODULE

Then, choose a simulator and simulate the UVM testbench using plusarg overrides.

cd uvm_build/pulsedetector_tb_uvm_testbench/top

% Simulate with an SNR of 2.0 and an error threshold of 1.0 percent.
setenv EXTRA_UVM_SIM_ARGS '+SNR_default_inp_val=10000000 +RTWStructParam_pErrorThreshold=1.0'
switch current_simulator
    case 'Questa',     ! vsim -c -do run_tb_mq.do
    case 'Questa_gui', ! vsim -do run_tb_mq.do
    case 'Xcelium',    ! ./run_tb_xcelium.sh
    case 'VCS',        ! ./run_tb_vcs.sh
end
Reading pref.tcl

# 2022.2

# do run_tb_mq.do
# +SNR_default_inp_val=10000000 +RTWStructParam_pErrorThreshold=1.0
# mw_PulseDetector_top.sv
# QuestaSim-64 vlog 2022.2 Compiler 2022.04 Apr 25 2022
# Start time: 17:37:23 on Jun 14,2023
# vlog -timescale 1ns/1ns ../DPI_dut/PulseDetector_dpi_pkg.sv ../sequence/GenPulse_dpi_pkg.sv ../scoreboard/CheckDetection_dpi_pkg.sv ../driver/InputDriver_dpi_pkg.sv ../monitor/OutputMonitor_dpi_pkg.sv ../predictor/PulseDetectorRef_dpi_pkg.sv pulsedetector_tb_pkg.sv mw_PulseDetector_top.sv "+define+MG_SIM" 
# -- Compiling package PulseDetector_dpi_pkg
# -- Compiling package GenPulse_dpi_pkg
# -- Compiling package CheckDetection_dpi_pkg
# -- Compiling package InputDriver_dpi_pkg
# -- Compiling package OutputMonitor_dpi_pkg
# -- Compiling package PulseDetectorRef_dpi_pkg
# ** Note: (vlog-2286) pulsedetector_tb_pkg.sv(5): Using implicit +incdir+/mathworks/hub/3rdparty/R2023a/8709182/share/Questasim/Lin/uvm-1.1d/../verilog_src/uvm-1.1d/src from import uvm_pkg
# -- Compiling package pulsedetector_tb_pkg
# -- Importing package mtiUvm.uvm_pkg (uvm-1.1d Built-in)
# -- Importing package GenPulse_dpi_pkg
# -- Importing package CheckDetection_dpi_pkg
# -- Importing package PulseDetectorRef_dpi_pkg
# -- Importing package InputDriver_dpi_pkg
# -- Importing package OutputMonitor_dpi_pkg
# ** Warning: ** while parsing file included at mw_PulseDetector_top.sv(5)
# ** at ../DPI_dut/PulseDetector_dpi.sv(15): (vlog-13314) Defaulting port 'coeff_in' kind to 'var' rather than 'wire' due to default compile option setting of -svinputport=relaxed.
# -- Compiling package mw_PulseDetector_top_sv_unit
# -- Importing package PulseDetector_dpi_pkg
# -- Importing package pulsedetector_tb_pkg
# -- Compiling module PulseDetector_dpi
# ** Warning: ../DPI_dut/PulseDetector_dpi.sv(15): (vlog-13314) Defaulting port 'coeff_in' kind to 'var' rather than 'wire' due to default compile option setting of -svinputport=relaxed.
# -- Compiling interface mw_PulseDetector_if
# -- Compiling module mw_PulseDetector_top
# 
# Top level modules:
# 	mw_PulseDetector_top
# End time: 17:37:24 on Jun 14,2023, Elapsed time: 0:00:01
# Errors: 0, Warnings: 2
# vsim "+SNR_default_inp_val=10000000" "+RTWStructParam_pErrorThreshold=1.0" -L work -voptargs="+acc" -sv_lib ../DPI_dut/PulseDetector -sv_lib ../sequence/GenPulse -sv_lib ../scoreboard/CheckDetection -sv_lib ../driver/InputDriver -sv_lib ../monitor/OutputMonitor -sv_lib ../predictor/PulseDetectorRef "+UVM_TESTNAME=mw_PulseDetector_test" mw_PulseDetector_top "+define+MG_SIM" 
# Start time: 17:37:24 on Jun 14,2023
# ** Note: (vsim-3812) Design is being optimized...
# ** Warning: (vopt-10587) Some optimizations are turned off because the +acc switch is in effect. This will cause your simulation to run slowly. Please use -access/-debug to maintain needed visibility.
# ** Warning: ../DPI_dut/PulseDetector_dpi.sv(15): (vopt-13314) Defaulting port 'coeff_in' kind to 'var' rather than 'wire' due to default compile option setting of -svinputport=relaxed.
# ** Note: (vsim-12126) Error and warning message counts have been restored: Errors=0, Warnings=2.
# //  Questa Sim-64
# //  Version 2022.2 linux_x86_64 Apr 25 2022
# //
# //  Copyright 1991-2022 Mentor Graphics Corporation
# //  All Rights Reserved.
# //
# //  QuestaSim and its associated documentation contain trade
# //  secrets and commercial or financial information that are the property of
# //  Mentor Graphics Corporation and are privileged, confidential,
# //  and exempt from disclosure under the Freedom of Information Act,
# //  5 U.S.C. Section 552. Furthermore, this information
# //  is prohibited from disclosure under the Trade Secrets Act,
# //  18 U.S.C. Section 1905.
# //
# Loading sv_std.std
# Loading work.mw_PulseDetector_if(fast)
# Loading work.OutputMonitor_dpi_pkg(fast)
# Loading work.InputDriver_dpi_pkg(fast)
# Loading work.PulseDetectorRef_dpi_pkg(fast)
# Loading work.CheckDetection_dpi_pkg(fast)
# Loading work.GenPulse_dpi_pkg(fast)
# Loading mtiUvm.uvm_pkg(fast)
# Loading work.pulsedetector_tb_pkg(fast)
# Loading work.PulseDetector_dpi_pkg(fast)
# Loading work.mw_PulseDetector_top_sv_unit(fast)
# Loading mtiUvm.questa_uvm_pkg(fast)
# Loading work.mw_PulseDetector_top(fast)
# Loading work.mw_PulseDetector_if(fast__2)
# Loading work.PulseDetector_dpi(fast)
# Loading ./../DPI_dut/PulseDetector.so
# Loading ./../sequence/GenPulse.so
# Loading ./../scoreboard/CheckDetection.so
# Loading ./../driver/InputDriver.so
# Loading ./../monitor/OutputMonitor.so
# Loading ./../predictor/PulseDetectorRef.so
# Loading /mathworks/hub/3rdparty/R2023a/8709182/share/Questasim/Lin/uvm-1.1d/linux_x86_64/uvm_dpi.so
# ----------------------------------------------------------------
# UVM-1.1d
# (C) 2007-2013 Mentor Graphics Corporation
# (C) 2007-2013 Cadence Design Systems, Inc.
# (C) 2006-2013 Synopsys, Inc.
# (C) 2011-2013 Cypress Semiconductor Corp.
# ----------------------------------------------------------------
# 
#   ***********       IMPORTANT RELEASE NOTES         ************
# 
#   You are using a version of the UVM library that has been compiled
#   with `UVM_NO_DEPRECATED undefined.
#   See http://www.eda.org/svdb/view.php?id=3313 for more details.
# 
#   You are using a version of the UVM library that has been compiled
#   with `UVM_OBJECT_MUST_HAVE_CONSTRUCTOR undefined.
#   See http://www.eda.org/svdb/view.php?id=3770 for more details.
# 
#       (Specify +UVM_NO_RELNOTES to turn off this notice)
# 
# UVM_INFO verilog_src/questa_uvm_pkg-1.2/src/questa_uvm_pkg.sv(277) @ 0: reporter [Questa UVM] QUESTA_UVM-1.2.3
# UVM_INFO verilog_src/questa_uvm_pkg-1.2/src/questa_uvm_pkg.sv(278) @ 0: reporter [Questa UVM]  questa_uvm::init(+struct)
# UVM_INFO @ 0: reporter [RNTST] Running test mw_PulseDetector_test...
# ** Info: Gathering coverage for           2 Simulink verify() calls.
#    Time: 0 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.initVerifyInfo File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 251
# 
# [FrameNum=  0] No peak found in Ref or Impl.
# 
# [FrameNum=  1] PREDICTED: Peak location=2163.000000, mag-squared=0.280 using global max
# 
# [FrameNum=  1] ACTUAL   : Peak location=2170.000000, mag-squared=0.285 using global max
# 
# [FrameNum=  1] DIFF     : Peak location=7, mag-squared=0.004 (1.551%)
# ** Error: pulsedetector_tb:744:
#    Time: 50020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 50020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum=  2] PREDICTED: Peak location=2163.000000, mag-squared=0.200 using global max
# 
# [FrameNum=  2] ACTUAL   : Peak location=2170.000000, mag-squared=0.194 using global max
# 
# [FrameNum=  2] DIFF     : Peak location=7, mag-squared=0.006 (2.881%)
# ** Error: pulsedetector_tb:744:
#    Time: 100020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 100020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum=  3] PREDICTED: Peak location=2163.000000, mag-squared=0.224 using global max
# 
# [FrameNum=  3] ACTUAL   : Peak location=2170.000000, mag-squared=0.234 using global max
# 
# [FrameNum=  3] DIFF     : Peak location=7, mag-squared=0.010 (4.623%)
# ** Error: pulsedetector_tb:744:
#    Time: 150020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 150020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum=  4] PREDICTED: Peak location=2163.000000, mag-squared=0.200 using global max
# 
# [FrameNum=  4] ACTUAL   : Peak location=2170.000000, mag-squared=0.209 using global max
# 
# [FrameNum=  4] DIFF     : Peak location=7, mag-squared=0.009 (4.346%)
# ** Error: pulsedetector_tb:744:
#    Time: 200020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 200020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum=  5] PREDICTED: Peak location=2163.000000, mag-squared=0.255 using global max
# 
# [FrameNum=  5] ACTUAL   : Peak location=2170.000000, mag-squared=0.257 using global max
# 
# [FrameNum=  5] DIFF     : Peak location=7, mag-squared=0.002 (0.735%)
# 
# [FrameNum=  6] PREDICTED: Peak location=2163.000000, mag-squared=0.241 using global max
# 
# [FrameNum=  6] ACTUAL   : Peak location=2170.000000, mag-squared=0.250 using global max
# 
# [FrameNum=  6] DIFF     : Peak location=7, mag-squared=0.009 (3.660%)
# ** Error: pulsedetector_tb:744:
#    Time: 300020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 300020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum=  7] PREDICTED: Peak location=2163.000000, mag-squared=0.241 using global max
# 
# [FrameNum=  7] ACTUAL   : Peak location=2170.000000, mag-squared=0.243 using global max
# 
# [FrameNum=  7] DIFF     : Peak location=7, mag-squared=0.002 (0.790%)
# 
# [FrameNum=  8] PREDICTED: Peak location=2163.000000, mag-squared=0.225 using global max
# 
# [FrameNum=  8] ACTUAL   : Peak location=2170.000000, mag-squared=0.231 using global max
# 
# [FrameNum=  8] DIFF     : Peak location=7, mag-squared=0.007 (3.076%)
# ** Error: pulsedetector_tb:744:
#    Time: 400020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 400020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum=  9] PREDICTED: Peak location=2163.000000, mag-squared=0.239 using global max
# 
# [FrameNum=  9] ACTUAL   : Peak location=2170.000000, mag-squared=0.254 using global max
# 
# [FrameNum=  9] DIFF     : Peak location=7, mag-squared=0.015 (6.083%)
# ** Error: pulsedetector_tb:744:
#    Time: 450020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 450020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum= 10] PREDICTED: Peak location=2163.000000, mag-squared=0.225 using global max
# 
# [FrameNum= 10] ACTUAL   : Peak location=2170.000000, mag-squared=0.230 using global max
# 
# [FrameNum= 10] DIFF     : Peak location=7, mag-squared=0.005 (2.242%)
# ** Error: pulsedetector_tb:744:
#    Time: 500020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 500020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum= 11] PREDICTED: Peak location=2163.000000, mag-squared=0.207 using global max
# 
# [FrameNum= 11] ACTUAL   : Peak location=2170.000000, mag-squared=0.221 using global max
# 
# [FrameNum= 11] DIFF     : Peak location=7, mag-squared=0.014 (6.697%)
# ** Error: pulsedetector_tb:744:
#    Time: 550020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 550020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum= 12] PREDICTED: Peak location=2163.000000, mag-squared=0.265 using global max
# 
# [FrameNum= 12] ACTUAL   : Peak location=2170.000000, mag-squared=0.260 using global max
# 
# [FrameNum= 12] DIFF     : Peak location=7, mag-squared=0.005 (2.014%)
# ** Error: pulsedetector_tb:744:
#    Time: 600020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 600020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# UVM_INFO verilog_src/uvm-1.1d/src/base/uvm_objection.svh(1267) @ 650000: reporter [TEST_DONE] 'run' phase is ready to proceed to the 'extract' phase
# ** Info: Instance coverage for verify 'pulsedetector_tb:757', coverpoint 'pass_cp': metric=100.00, at_least=   1 (   COVERED)
#    Time: 650 us  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.reportVerifyCoverage File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 369
# ** Info: Instance coverage for verify 'pulsedetector_tb:744', coverpoint 'pass_cp': metric=100.00, at_least=   1 (   COVERED)
#    Time: 650 us  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.reportVerifyCoverage File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 369
# ** Info: Overall coverage for CheckDetection_dpi_verify_calls: metric=100.00 (   COVERED)
#    Time: 650 us  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.reportVerifyCoverage File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 376
# 
# --- UVM Report Summary ---
# 
# ** Report counts by severity
# UVM_INFO :    4
# UVM_WARNING :    0
# UVM_ERROR :    0
# UVM_FATAL :    0
# ** Report counts by id
# [Questa UVM]     2
# [RNTST]     1
# [TEST_DONE]     1
# ** Note: $finish    : /mathworks/hub/3rdparty/R2023a/8709182/share/Questasim/Lin/linux_x86_64/../verilog_src/uvm-1.1d/src/base/uvm_root.svh(430)
#    Time: 650 us  Iteration: 61  Instance: /mw_PulseDetector_top
# End time: 17:37:46 on Jun 14,2023, Elapsed time: 0:00:22
# Errors: 20, Warnings: 2
cd ../../..

Observe that with an SNR of 2.0 the design violates the new 1.0 percent error threshold the majority of the time.

Change Coverage Goal for the Error Threshold Using a Plus Argument

NOTE: The generation and collection of cover groups in the UVM testbench requires Simulink Test™.

You can specify the number of detected frames that must meet the error threshold by setting a coverage goal. Test Assessment and Test Sequence blocks with verify() calls, Assertion blocks, and Simulink Model Verification blocks all generate coverage information. For more information, see the Generate SystemVerilog Assertions and Functional Coverage and the example Generating Functional Coverage in SystemVerilog from Simulink Test verify Calls

In this model, the Check Static Upper Bound block in the Checker subsystem has a coverage ID of pulsedetector_tb:744 in the generated UVM testbench. You can set alternative SNR values, error threshold values, and coverage goals via plus args in the EXTRA_UVM_SIM_ARGS. For example:

  • Express that all 13 frames must pass the error threshold. With an SNR of 2.0 and a threshold of 5 percent we often see one or two frames fail the threshold, so this coverage goal will likely not be met.

setenv EXTRA_UVM_SIM_ARGS '+SNR_default_inp_val=10000000 +RTWStructParam_pErrorThreshold=5.0 +pulsedetector_tb:744=13'
  • Raise the error threshold to eliminate borderline predictor/actual comparisons. This threshold allows the coverage goal to be met.

setenv EXTRA_UVM_SIM_ARGS '+SNR_default_inp_val=10000000 +RTWStructParam_pErrorThreshold=10.0 +pulsedetector_tb:744=13'
  • Keep the 5 percent error threshold and set a lower success rate for the coverage goal.

setenv EXTRA_UVM_SIM_ARGS '+SNR_default_inp_val=10000000 +RTWStructParam_pErrorThreshold=5.0 +pulsedetector_tb:744=8'

Randomize Error Threshold Using Default Configuration Object in Test

You can generate and set values directly from a test since the scoreboard configuration object is a member. Here, we randomize a threshold in the range [4.00, 8.00] percent. The threshold is of type real and, therefore, cannot be randomized directly.

See mw_PulseDetector_SCRPRM_param_overrides.sv.

ScoreboardRndDefaultVal.png

To run the UVM simulations using these new classes, you can use the original scripts with extra arguments given through environment variables.

cd uvm_build/pulsedetector_tb_uvm_testbench/top
setenv EXTRA_UVM_COMP_ARGS '-f ../../../overrides_SCRPRM/extra_comp_args.f'
setenv EXTRA_UVM_SIM_ARGS '+SNR_default_inp_val=10000000 +UVM_TESTNAME=mw_PulseDetector_test_SCRPRM'
switch current_simulator
    case 'Questa',     ! vsim -c -do run_tb_mq.do
    case 'Questa_gui', ! vsim -do run_tb_mq.do
    case 'Xcelium',    ! ./run_tb_xcelium.sh
    case 'VCS',        ! ./run_tb_vcs.sh
end
Reading pref.tcl

# 2022.2

# do run_tb_mq.do
# -f ../../../overrides_SCRPRM/extra_comp_args.f
# +SNR_default_inp_val=10000000 +UVM_TESTNAME=mw_PulseDetector_test_SCRPRM
# mw_PulseDetector_top.sv
# ** Warning: (vlib-34) Library already exists at "work".
# Errors: 0, Warnings: 1
# QuestaSim-64 vlog 2022.2 Compiler 2022.04 Apr 25 2022
# Start time: 17:37:47 on Jun 14,2023
# vlog -timescale 1ns/1ns ../DPI_dut/PulseDetector_dpi_pkg.sv ../sequence/GenPulse_dpi_pkg.sv ../scoreboard/CheckDetection_dpi_pkg.sv ../driver/InputDriver_dpi_pkg.sv ../monitor/OutputMonitor_dpi_pkg.sv ../predictor/PulseDetectorRef_dpi_pkg.sv pulsedetector_tb_pkg.sv -f ../../../overrides_SCRPRM/extra_comp_args.f mw_PulseDetector_top.sv "+define+MG_SIM" 
# -- Compiling package PulseDetector_dpi_pkg
# -- Compiling package GenPulse_dpi_pkg
# -- Compiling package CheckDetection_dpi_pkg
# -- Compiling package InputDriver_dpi_pkg
# -- Compiling package OutputMonitor_dpi_pkg
# -- Compiling package PulseDetectorRef_dpi_pkg
# ** Note: (vlog-2286) pulsedetector_tb_pkg.sv(5): Using implicit +incdir+/mathworks/hub/3rdparty/R2023a/8709182/share/Questasim/Lin/uvm-1.1d/../verilog_src/uvm-1.1d/src from import uvm_pkg
# -- Compiling package pulsedetector_tb_pkg
# -- Importing package mtiUvm.uvm_pkg (uvm-1.1d Built-in)
# -- Importing package GenPulse_dpi_pkg
# -- Importing package CheckDetection_dpi_pkg
# -- Importing package PulseDetectorRef_dpi_pkg
# -- Importing package InputDriver_dpi_pkg
# -- Importing package OutputMonitor_dpi_pkg
# -- Compiling package pulsedetector_tb_extension_pkg
# -- Importing package pulsedetector_tb_pkg
# ** Warning: ../../../overrides_SCRPRM/./mw_PulseDetector_SCRPRM_param_overrides.sv(76): (vlog-2240) Treating stand-alone use of function 'randomize' as an implicit VOID cast.
# ** Warning: ** while parsing file included at mw_PulseDetector_top.sv(5)
# ** at ../DPI_dut/PulseDetector_dpi.sv(15): (vlog-13314) Defaulting port 'coeff_in' kind to 'var' rather than 'wire' due to default compile option setting of -svinputport=relaxed.
# ** Warning: ../DPI_dut/PulseDetector_dpi.sv(6): (vlog-13233) Design unit "mw_PulseDetector_top_sv_unit" already exists and will be overwritten. Design unit compiled with different set of options.
# -- Compiling package mw_PulseDetector_top_sv_unit
# -- Importing package PulseDetector_dpi_pkg
# -- Importing package pulsedetector_tb_extension_pkg
# -- Compiling module PulseDetector_dpi
# ** Warning: ../DPI_dut/PulseDetector_dpi.sv(15): (vlog-13314) Defaulting port 'coeff_in' kind to 'var' rather than 'wire' due to default compile option setting of -svinputport=relaxed.
# -- Compiling interface mw_PulseDetector_if
# -- Compiling module mw_PulseDetector_top
# 
# Top level modules:
# 	mw_PulseDetector_top
# End time: 17:37:47 on Jun 14,2023, Elapsed time: 0:00:00
# Errors: 0, Warnings: 4
# vsim "+SNR_default_inp_val=10000000" "+UVM_TESTNAME=mw_PulseDetector_test_SCRPRM" -L work -voptargs="+acc" -sv_lib ../DPI_dut/PulseDetector -sv_lib ../sequence/GenPulse -sv_lib ../scoreboard/CheckDetection -sv_lib ../driver/InputDriver -sv_lib ../monitor/OutputMonitor -sv_lib ../predictor/PulseDetectorRef "+UVM_TESTNAME=mw_PulseDetector_test" mw_PulseDetector_top "+define+MG_SIM" 
# Start time: 17:37:47 on Jun 14,2023
# ** Note: (vsim-3813) Design is being optimized due to module recompilation...
# ** Warning: (vopt-10587) Some optimizations are turned off because the +acc switch is in effect. This will cause your simulation to run slowly. Please use -access/-debug to maintain needed visibility.
# ** Warning: ../../../overrides_SCRPRM/./mw_PulseDetector_SCRPRM_param_overrides.sv(76): (vopt-2240) Treating stand-alone use of function 'randomize' as an implicit VOID cast.
# ** Warning: ../DPI_dut/PulseDetector_dpi.sv(15): (vopt-13314) Defaulting port 'coeff_in' kind to 'var' rather than 'wire' due to default compile option setting of -svinputport=relaxed.
# ** Note: (vsim-12126) Error and warning message counts have been restored: Errors=0, Warnings=3.
# //  Questa Sim-64
# //  Version 2022.2 linux_x86_64 Apr 25 2022
# //
# //  Copyright 1991-2022 Mentor Graphics Corporation
# //  All Rights Reserved.
# //
# //  QuestaSim and its associated documentation contain trade
# //  secrets and commercial or financial information that are the property of
# //  Mentor Graphics Corporation and are privileged, confidential,
# //  and exempt from disclosure under the Freedom of Information Act,
# //  5 U.S.C. Section 552. Furthermore, this information
# //  is prohibited from disclosure under the Trade Secrets Act,
# //  18 U.S.C. Section 1905.
# //
# Loading sv_std.std
# Loading work.mw_PulseDetector_if(fast)
# Loading work.OutputMonitor_dpi_pkg(fast)
# Loading work.InputDriver_dpi_pkg(fast)
# Loading work.PulseDetectorRef_dpi_pkg(fast)
# Loading work.CheckDetection_dpi_pkg(fast)
# Loading work.GenPulse_dpi_pkg(fast)
# Loading mtiUvm.uvm_pkg(fast)
# Loading work.pulsedetector_tb_pkg(fast)
# Loading work.pulsedetector_tb_extension_pkg(fast)
# Loading work.PulseDetector_dpi_pkg(fast)
# Loading work.mw_PulseDetector_top_sv_unit(fast)
# Loading mtiUvm.questa_uvm_pkg(fast)
# Loading work.mw_PulseDetector_top(fast)
# Loading work.mw_PulseDetector_if(fast__2)
# Loading work.PulseDetector_dpi(fast)
# Loading ./../DPI_dut/PulseDetector.so
# Loading ./../sequence/GenPulse.so
# Loading ./../scoreboard/CheckDetection.so
# Loading ./../driver/InputDriver.so
# Loading ./../monitor/OutputMonitor.so
# Loading ./../predictor/PulseDetectorRef.so
# Loading /mathworks/hub/3rdparty/R2023a/8709182/share/Questasim/Lin/uvm-1.1d/linux_x86_64/uvm_dpi.so
# ----------------------------------------------------------------
# UVM-1.1d
# (C) 2007-2013 Mentor Graphics Corporation
# (C) 2007-2013 Cadence Design Systems, Inc.
# (C) 2006-2013 Synopsys, Inc.
# (C) 2011-2013 Cypress Semiconductor Corp.
# ----------------------------------------------------------------
# 
#   ***********       IMPORTANT RELEASE NOTES         ************
# 
#   You are using a version of the UVM library that has been compiled
#   with `UVM_NO_DEPRECATED undefined.
#   See http://www.eda.org/svdb/view.php?id=3313 for more details.
# 
#   You are using a version of the UVM library that has been compiled
#   with `UVM_OBJECT_MUST_HAVE_CONSTRUCTOR undefined.
#   See http://www.eda.org/svdb/view.php?id=3770 for more details.
# 
#       (Specify +UVM_NO_RELNOTES to turn off this notice)
# 
# UVM_INFO verilog_src/questa_uvm_pkg-1.2/src/questa_uvm_pkg.sv(277) @ 0: reporter [Questa UVM] QUESTA_UVM-1.2.3
# UVM_INFO verilog_src/questa_uvm_pkg-1.2/src/questa_uvm_pkg.sv(278) @ 0: reporter [Questa UVM]  questa_uvm::init(+struct)
# UVM_WARNING @ 0: reporter [MULTTST] Multiple (2) +UVM_TESTNAME arguments provided on the command line.  'mw_PulseDetector_test_SCRPRM' will be used.  Provided list: mw_PulseDetector_test_SCRPRM, mw_PulseDetector_test.
# UVM_INFO @ 0: reporter [RNTST] Running test mw_PulseDetector_test_SCRPRM...
# UVM_INFO ../../../overrides_SCRPRM/./mw_PulseDetector_SCRPRM_param_overrides.sv(28) @ 0: uvm_test_top [mw_PulseDetector_test_SCRPRM] Setting error threshold to 5.200 percent
# 
# ** Info: Gathering coverage for           2 Simulink verify() calls.
#    Time: 0 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.initVerifyInfo File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 251
# 
# [FrameNum=  0] No peak found in Ref or Impl.
# 
# [FrameNum=  1] PREDICTED: Peak location=2163.000000, mag-squared=0.280 using global max
# 
# [FrameNum=  1] ACTUAL   : Peak location=2170.000000, mag-squared=0.285 using global max
# 
# [FrameNum=  1] DIFF     : Peak location=7, mag-squared=0.004 (1.551%)
# 
# [FrameNum=  2] PREDICTED: Peak location=2163.000000, mag-squared=0.200 using global max
# 
# [FrameNum=  2] ACTUAL   : Peak location=2170.000000, mag-squared=0.194 using global max
# 
# [FrameNum=  2] DIFF     : Peak location=7, mag-squared=0.006 (2.881%)
# 
# [FrameNum=  3] PREDICTED: Peak location=2163.000000, mag-squared=0.224 using global max
# 
# [FrameNum=  3] ACTUAL   : Peak location=2170.000000, mag-squared=0.234 using global max
# 
# [FrameNum=  3] DIFF     : Peak location=7, mag-squared=0.010 (4.623%)
# 
# [FrameNum=  4] PREDICTED: Peak location=2163.000000, mag-squared=0.200 using global max
# 
# [FrameNum=  4] ACTUAL   : Peak location=2170.000000, mag-squared=0.209 using global max
# 
# [FrameNum=  4] DIFF     : Peak location=7, mag-squared=0.009 (4.346%)
# 
# [FrameNum=  5] PREDICTED: Peak location=2163.000000, mag-squared=0.255 using global max
# 
# [FrameNum=  5] ACTUAL   : Peak location=2170.000000, mag-squared=0.257 using global max
# 
# [FrameNum=  5] DIFF     : Peak location=7, mag-squared=0.002 (0.735%)
# 
# [FrameNum=  6] PREDICTED: Peak location=2163.000000, mag-squared=0.241 using global max
# 
# [FrameNum=  6] ACTUAL   : Peak location=2170.000000, mag-squared=0.250 using global max
# 
# [FrameNum=  6] DIFF     : Peak location=7, mag-squared=0.009 (3.660%)
# 
# [FrameNum=  7] PREDICTED: Peak location=2163.000000, mag-squared=0.241 using global max
# 
# [FrameNum=  7] ACTUAL   : Peak location=2170.000000, mag-squared=0.243 using global max
# 
# [FrameNum=  7] DIFF     : Peak location=7, mag-squared=0.002 (0.790%)
# 
# [FrameNum=  8] PREDICTED: Peak location=2163.000000, mag-squared=0.225 using global max
# 
# [FrameNum=  8] ACTUAL   : Peak location=2170.000000, mag-squared=0.231 using global max
# 
# [FrameNum=  8] DIFF     : Peak location=7, mag-squared=0.007 (3.076%)
# 
# [FrameNum=  9] PREDICTED: Peak location=2163.000000, mag-squared=0.239 using global max
# 
# [FrameNum=  9] ACTUAL   : Peak location=2170.000000, mag-squared=0.254 using global max
# 
# [FrameNum=  9] DIFF     : Peak location=7, mag-squared=0.015 (6.083%)
# ** Error: pulsedetector_tb:744:
#    Time: 450020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 450020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum= 10] PREDICTED: Peak location=2163.000000, mag-squared=0.225 using global max
# 
# [FrameNum= 10] ACTUAL   : Peak location=2170.000000, mag-squared=0.230 using global max
# 
# [FrameNum= 10] DIFF     : Peak location=7, mag-squared=0.005 (2.242%)
# 
# [FrameNum= 11] PREDICTED: Peak location=2163.000000, mag-squared=0.207 using global max
# 
# [FrameNum= 11] ACTUAL   : Peak location=2170.000000, mag-squared=0.221 using global max
# 
# [FrameNum= 11] DIFF     : Peak location=7, mag-squared=0.014 (6.697%)
# ** Error: pulsedetector_tb:744:
#    Time: 550020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 550020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum= 12] PREDICTED: Peak location=2163.000000, mag-squared=0.265 using global max
# 
# [FrameNum= 12] ACTUAL   : Peak location=2170.000000, mag-squared=0.260 using global max
# 
# [FrameNum= 12] DIFF     : Peak location=7, mag-squared=0.005 (2.014%)
# UVM_INFO verilog_src/uvm-1.1d/src/base/uvm_objection.svh(1267) @ 650000: reporter [TEST_DONE] 'run' phase is ready to proceed to the 'extract' phase
# ** Info: Instance coverage for verify 'pulsedetector_tb:757', coverpoint 'pass_cp': metric=100.00, at_least=   1 (   COVERED)
#    Time: 650 us  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.reportVerifyCoverage File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 369
# ** Info: Instance coverage for verify 'pulsedetector_tb:744', coverpoint 'pass_cp': metric=100.00, at_least=   1 (   COVERED)
#    Time: 650 us  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.reportVerifyCoverage File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 369
# ** Info: Overall coverage for CheckDetection_dpi_verify_calls: metric=100.00 (   COVERED)
#    Time: 650 us  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.reportVerifyCoverage File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 376
# 
# --- UVM Report Summary ---
# 
# ** Report counts by severity
# UVM_INFO :    5
# UVM_WARNING :    1
# UVM_ERROR :    0
# UVM_FATAL :    0
# ** Report counts by id
# [MULTTST]     1
# [Questa UVM]     2
# [RNTST]     1
# [TEST_DONE]     1
# [mw_PulseDetector_test_SCRPRM]     1
# ** Note: $finish    : /mathworks/hub/3rdparty/R2023a/8709182/share/Questasim/Lin/linux_x86_64/../verilog_src/uvm-1.1d/src/base/uvm_root.svh(430)
#    Time: 650 us  Iteration: 61  Instance: /mw_PulseDetector_top
# End time: 17:38:06 on Jun 14,2023, Elapsed time: 0:00:19
# Errors: 4, Warnings: 3
cd ../../..

The number of violations will depend on the generated randomized error threshold.

Randomize Error Threshold Using Derived Configuration Object

You can also create more sophisticated randomization of the configuration settings by creating a derived scoreboard configuration object. In this example, the test plan requires that the threshold fall in different ranges that reflect different operating environments of the detector.

See mw_PulseDetector_SCRPRM_param_overrides.sv.

To achieve this goal, the derived configuration object has the follow extra behaviors:

  • Add a threshold_bucket member declared as randc.

  • Use that threshold_bucket member to imply threshold ranges.

  • Print out the randomized values for inspection during simulation.

ScoreboardRndCfgObj.png

A new test is created, which tells the UVM factory to use the new configuration object and to randomize its settings during the build phase.

ScoreboardRndCfgObjTest.png

To run the UVM simulations using these new classes, you can use the original scripts with extra arguments given through environment variables.

cd uvm_build/pulsedetector_tb_uvm_testbench/top
setenv EXTRA_UVM_COMP_ARGS '-f ../../../overrides_SCRPRM/extra_comp_args.f'
setenv EXTRA_UVM_SIM_ARGS '+SNR_default_inp_val=10000000 +UVM_TESTNAME=mw_PulseDetector_test_SCRPRM2'
switch current_simulator
    case 'Questa',     ! vsim -c -do run_tb_mq.do
    case 'Questa_gui', ! vsim -do run_tb_mq.do
    case 'Xcelium',    ! ./run_tb_xcelium.sh
    case 'VCS',        ! ./run_tb_vcs.sh
end
Reading pref.tcl

# 2022.2

# do run_tb_mq.do
# -f ../../../overrides_SCRPRM/extra_comp_args.f
# +SNR_default_inp_val=10000000 +UVM_TESTNAME=mw_PulseDetector_test_SCRPRM2
# mw_PulseDetector_top.sv
# ** Warning: (vlib-34) Library already exists at "work".
# Errors: 0, Warnings: 1
# QuestaSim-64 vlog 2022.2 Compiler 2022.04 Apr 25 2022
# Start time: 17:38:07 on Jun 14,2023
# vlog -timescale 1ns/1ns ../DPI_dut/PulseDetector_dpi_pkg.sv ../sequence/GenPulse_dpi_pkg.sv ../scoreboard/CheckDetection_dpi_pkg.sv ../driver/InputDriver_dpi_pkg.sv ../monitor/OutputMonitor_dpi_pkg.sv ../predictor/PulseDetectorRef_dpi_pkg.sv pulsedetector_tb_pkg.sv -f ../../../overrides_SCRPRM/extra_comp_args.f mw_PulseDetector_top.sv "+define+MG_SIM" 
# -- Compiling package PulseDetector_dpi_pkg
# -- Compiling package GenPulse_dpi_pkg
# -- Compiling package CheckDetection_dpi_pkg
# -- Compiling package InputDriver_dpi_pkg
# -- Compiling package OutputMonitor_dpi_pkg
# -- Compiling package PulseDetectorRef_dpi_pkg
# ** Note: (vlog-2286) pulsedetector_tb_pkg.sv(5): Using implicit +incdir+/mathworks/hub/3rdparty/R2023a/8709182/share/Questasim/Lin/uvm-1.1d/../verilog_src/uvm-1.1d/src from import uvm_pkg
# -- Compiling package pulsedetector_tb_pkg
# -- Importing package mtiUvm.uvm_pkg (uvm-1.1d Built-in)
# -- Importing package GenPulse_dpi_pkg
# -- Importing package CheckDetection_dpi_pkg
# -- Importing package PulseDetectorRef_dpi_pkg
# -- Importing package InputDriver_dpi_pkg
# -- Importing package OutputMonitor_dpi_pkg
# -- Compiling package pulsedetector_tb_extension_pkg
# -- Importing package pulsedetector_tb_pkg
# ** Warning: ../../../overrides_SCRPRM/./mw_PulseDetector_SCRPRM_param_overrides.sv(76): (vlog-2240) Treating stand-alone use of function 'randomize' as an implicit VOID cast.
# ** Warning: ** while parsing file included at mw_PulseDetector_top.sv(5)
# ** at ../DPI_dut/PulseDetector_dpi.sv(15): (vlog-13314) Defaulting port 'coeff_in' kind to 'var' rather than 'wire' due to default compile option setting of -svinputport=relaxed.
# -- Compiling package mw_PulseDetector_top_sv_unit
# -- Importing package PulseDetector_dpi_pkg
# -- Importing package pulsedetector_tb_extension_pkg
# -- Compiling module PulseDetector_dpi
# ** Warning: ../DPI_dut/PulseDetector_dpi.sv(15): (vlog-13314) Defaulting port 'coeff_in' kind to 'var' rather than 'wire' due to default compile option setting of -svinputport=relaxed.
# -- Compiling interface mw_PulseDetector_if
# -- Compiling module mw_PulseDetector_top
# 
# Top level modules:
# 	mw_PulseDetector_top
# End time: 17:38:08 on Jun 14,2023, Elapsed time: 0:00:01
# Errors: 0, Warnings: 3
# vsim "+SNR_default_inp_val=10000000" "+UVM_TESTNAME=mw_PulseDetector_test_SCRPRM2" -L work -voptargs="+acc" -sv_lib ../DPI_dut/PulseDetector -sv_lib ../sequence/GenPulse -sv_lib ../scoreboard/CheckDetection -sv_lib ../driver/InputDriver -sv_lib ../monitor/OutputMonitor -sv_lib ../predictor/PulseDetectorRef "+UVM_TESTNAME=mw_PulseDetector_test" mw_PulseDetector_top "+define+MG_SIM" 
# Start time: 17:38:08 on Jun 14,2023
# ** Note: (vsim-8009) Loading existing optimized design _opt
# //  Questa Sim-64
# //  Version 2022.2 linux_x86_64 Apr 25 2022
# //
# //  Copyright 1991-2022 Mentor Graphics Corporation
# //  All Rights Reserved.
# //
# //  QuestaSim and its associated documentation contain trade
# //  secrets and commercial or financial information that are the property of
# //  Mentor Graphics Corporation and are privileged, confidential,
# //  and exempt from disclosure under the Freedom of Information Act,
# //  5 U.S.C. Section 552. Furthermore, this information
# //  is prohibited from disclosure under the Trade Secrets Act,
# //  18 U.S.C. Section 1905.
# //
# Loading sv_std.std
# Loading work.mw_PulseDetector_if(fast)
# Loading work.OutputMonitor_dpi_pkg(fast)
# Loading work.InputDriver_dpi_pkg(fast)
# Loading work.PulseDetectorRef_dpi_pkg(fast)
# Loading work.CheckDetection_dpi_pkg(fast)
# Loading work.GenPulse_dpi_pkg(fast)
# Loading mtiUvm.uvm_pkg(fast)
# Loading work.pulsedetector_tb_pkg(fast)
# Loading work.pulsedetector_tb_extension_pkg(fast)
# Loading work.PulseDetector_dpi_pkg(fast)
# Loading work.mw_PulseDetector_top_sv_unit(fast)
# Loading mtiUvm.questa_uvm_pkg(fast)
# Loading work.mw_PulseDetector_top(fast)
# Loading work.mw_PulseDetector_if(fast__2)
# Loading work.PulseDetector_dpi(fast)
# Loading ./../DPI_dut/PulseDetector.so
# Loading ./../sequence/GenPulse.so
# Loading ./../scoreboard/CheckDetection.so
# Loading ./../driver/InputDriver.so
# Loading ./../monitor/OutputMonitor.so
# Loading ./../predictor/PulseDetectorRef.so
# Loading /mathworks/hub/3rdparty/R2023a/8709182/share/Questasim/Lin/uvm-1.1d/linux_x86_64/uvm_dpi.so
# ----------------------------------------------------------------
# UVM-1.1d
# (C) 2007-2013 Mentor Graphics Corporation
# (C) 2007-2013 Cadence Design Systems, Inc.
# (C) 2006-2013 Synopsys, Inc.
# (C) 2011-2013 Cypress Semiconductor Corp.
# ----------------------------------------------------------------
# 
#   ***********       IMPORTANT RELEASE NOTES         ************
# 
#   You are using a version of the UVM library that has been compiled
#   with `UVM_NO_DEPRECATED undefined.
#   See http://www.eda.org/svdb/view.php?id=3313 for more details.
# 
#   You are using a version of the UVM library that has been compiled
#   with `UVM_OBJECT_MUST_HAVE_CONSTRUCTOR undefined.
#   See http://www.eda.org/svdb/view.php?id=3770 for more details.
# 
#       (Specify +UVM_NO_RELNOTES to turn off this notice)
# 
# UVM_INFO verilog_src/questa_uvm_pkg-1.2/src/questa_uvm_pkg.sv(277) @ 0: reporter [Questa UVM] QUESTA_UVM-1.2.3
# UVM_INFO verilog_src/questa_uvm_pkg-1.2/src/questa_uvm_pkg.sv(278) @ 0: reporter [Questa UVM]  questa_uvm::init(+struct)
# UVM_WARNING @ 0: reporter [MULTTST] Multiple (2) +UVM_TESTNAME arguments provided on the command line.  'mw_PulseDetector_test_SCRPRM2' will be used.  Provided list: mw_PulseDetector_test_SCRPRM2, mw_PulseDetector_test.
# UVM_INFO @ 0: reporter [RNTST] Running test mw_PulseDetector_test_SCRPRM2...
# UVM_INFO ../../../overrides_SCRPRM/./mw_PulseDetector_SCRPRM_param_overrides.sv(57) @ 0: reporter [mw_PulseDetector_scoreboard_cfg_CRT] Setting error threshold to 5.16 percent
# 
# ** Info: Gathering coverage for           2 Simulink verify() calls.
#    Time: 0 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.initVerifyInfo File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 251
# 
# [FrameNum=  0] No peak found in Ref or Impl.
# 
# [FrameNum=  1] PREDICTED: Peak location=2163.000000, mag-squared=0.280 using global max
# 
# [FrameNum=  1] ACTUAL   : Peak location=2170.000000, mag-squared=0.285 using global max
# 
# [FrameNum=  1] DIFF     : Peak location=7, mag-squared=0.004 (1.551%)
# 
# [FrameNum=  2] PREDICTED: Peak location=2163.000000, mag-squared=0.200 using global max
# 
# [FrameNum=  2] ACTUAL   : Peak location=2170.000000, mag-squared=0.194 using global max
# 
# [FrameNum=  2] DIFF     : Peak location=7, mag-squared=0.006 (2.881%)
# 
# [FrameNum=  3] PREDICTED: Peak location=2163.000000, mag-squared=0.224 using global max
# 
# [FrameNum=  3] ACTUAL   : Peak location=2170.000000, mag-squared=0.234 using global max
# 
# [FrameNum=  3] DIFF     : Peak location=7, mag-squared=0.010 (4.623%)
# 
# [FrameNum=  4] PREDICTED: Peak location=2163.000000, mag-squared=0.200 using global max
# 
# [FrameNum=  4] ACTUAL   : Peak location=2170.000000, mag-squared=0.209 using global max
# 
# [FrameNum=  4] DIFF     : Peak location=7, mag-squared=0.009 (4.346%)
# 
# [FrameNum=  5] PREDICTED: Peak location=2163.000000, mag-squared=0.255 using global max
# 
# [FrameNum=  5] ACTUAL   : Peak location=2170.000000, mag-squared=0.257 using global max
# 
# [FrameNum=  5] DIFF     : Peak location=7, mag-squared=0.002 (0.735%)
# 
# [FrameNum=  6] PREDICTED: Peak location=2163.000000, mag-squared=0.241 using global max
# 
# [FrameNum=  6] ACTUAL   : Peak location=2170.000000, mag-squared=0.250 using global max
# 
# [FrameNum=  6] DIFF     : Peak location=7, mag-squared=0.009 (3.660%)
# 
# [FrameNum=  7] PREDICTED: Peak location=2163.000000, mag-squared=0.241 using global max
# 
# [FrameNum=  7] ACTUAL   : Peak location=2170.000000, mag-squared=0.243 using global max
# 
# [FrameNum=  7] DIFF     : Peak location=7, mag-squared=0.002 (0.790%)
# 
# [FrameNum=  8] PREDICTED: Peak location=2163.000000, mag-squared=0.225 using global max
# 
# [FrameNum=  8] ACTUAL   : Peak location=2170.000000, mag-squared=0.231 using global max
# 
# [FrameNum=  8] DIFF     : Peak location=7, mag-squared=0.007 (3.076%)
# 
# [FrameNum=  9] PREDICTED: Peak location=2163.000000, mag-squared=0.239 using global max
# 
# [FrameNum=  9] ACTUAL   : Peak location=2170.000000, mag-squared=0.254 using global max
# 
# [FrameNum=  9] DIFF     : Peak location=7, mag-squared=0.015 (6.083%)
# ** Error: pulsedetector_tb:744:
#    Time: 450020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 450020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum= 10] PREDICTED: Peak location=2163.000000, mag-squared=0.225 using global max
# 
# [FrameNum= 10] ACTUAL   : Peak location=2170.000000, mag-squared=0.230 using global max
# 
# [FrameNum= 10] DIFF     : Peak location=7, mag-squared=0.005 (2.242%)
# 
# [FrameNum= 11] PREDICTED: Peak location=2163.000000, mag-squared=0.207 using global max
# 
# [FrameNum= 11] ACTUAL   : Peak location=2170.000000, mag-squared=0.221 using global max
# 
# [FrameNum= 11] DIFF     : Peak location=7, mag-squared=0.014 (6.697%)
# ** Error: pulsedetector_tb:744:
#    Time: 550020 ns  Scope: pulsedetector_tb_pkg.mw_PulseDetector_scoreboard.run_phase File: ../scoreboard/mw_PulseDetector_scoreboard.sv Line: 84
# ** Error: pulsedetector_tb:744: At step 'inferred verify call' verify id 'CheckDetection/CheckDetection/Check Static 
# Upper Bound' Failed
#    Time: 550020 ns  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.checkVerifyStatus File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 347
# 
# [FrameNum= 12] PREDICTED: Peak location=2163.000000, mag-squared=0.265 using global max
# 
# [FrameNum= 12] ACTUAL   : Peak location=2170.000000, mag-squared=0.260 using global max
# 
# [FrameNum= 12] DIFF     : Peak location=7, mag-squared=0.005 (2.014%)
# UVM_INFO verilog_src/uvm-1.1d/src/base/uvm_objection.svh(1267) @ 650000: reporter [TEST_DONE] 'run' phase is ready to proceed to the 'extract' phase
# ** Info: Instance coverage for verify 'pulsedetector_tb:757', coverpoint 'pass_cp': metric=100.00, at_least=   1 (   COVERED)
#    Time: 650 us  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.reportVerifyCoverage File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 369
# ** Info: Instance coverage for verify 'pulsedetector_tb:744', coverpoint 'pass_cp': metric=100.00, at_least=   1 (   COVERED)
#    Time: 650 us  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.reportVerifyCoverage File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 369
# ** Info: Overall coverage for CheckDetection_dpi_verify_calls: metric=100.00 (   COVERED)
#    Time: 650 us  Scope: CheckDetection_dpi_pkg.VerifyInterfaceT.reportVerifyCoverage File: ../scoreboard/CheckDetection_dpi_pkg.sv Line: 376
# 
# --- UVM Report Summary ---
# 
# ** Report counts by severity
# UVM_INFO :    5
# UVM_WARNING :    1
# UVM_ERROR :    0
# UVM_FATAL :    0
# ** Report counts by id
# [MULTTST]     1
# [Questa UVM]     2
# [RNTST]     1
# [TEST_DONE]     1
# [mw_PulseDetector_scoreboard_cfg_CRT]     1
# ** Note: $finish    : /mathworks/hub/3rdparty/R2023a/8709182/share/Questasim/Lin/linux_x86_64/../verilog_src/uvm-1.1d/src/base/uvm_root.svh(430)
#    Time: 650 us  Iteration: 61  Instance: /mw_PulseDetector_top
# End time: 17:38:26 on Jun 14,2023, Elapsed time: 0:00:18
# Errors: 4, Warnings: 0
cd ../../..

The number of violations will depend on the generated randomized error threshold.

Conclusions and Next Steps

In a full UVM environment, whenever randomization is introduced, it is usually essential to include coverage and to ensure different parts of the environment are aware of which random values were generated. In these examples, randomized values are simply printed out. Adding coverage for generated error thresholds and analyzing the actual signal errors over the course of many simulation runs is an exercise left to the reader.

See Also

uvmbuild

Related Topics

Generate Parameterized UVM Testbench from Simulink

Replace Behavioral DUT with RTL DUT in UVM Testbench