coder.TensorRTConfig

Parameters to configure deep learning code generation with the NVIDIA TensorRT library

Description

The coder.TensorRTConfig object contains NVIDIA^® high performance deep learning inference optimizer and run-time library (TensorRT) specific parameters. codegen uses those parameters for generating CUDA^® code for deep neural networks.

To use a coder.TensorRTConfig object for code generation, assign it to the DeepLearningConfig property of a coder.gpuConfig object that you pass to codegen.

Creation

Create a TensorRT configuration object by using the coder.DeepLearningConfig function with target library set as 'tensorrt'.

Properties

DataType — Inference computation precision

'fp32' (default) | 'fp16' | 'int8'

Specify the precision of the inference computations in supported layers. For more information, see Data type (TensorRT).

DataPath — Image dataset location

'' (default) | character vector | string scalar

Location of the image dataset used during recalibration. This option is applicable only when DataType is set to 'int8'. For more information, see Calibration data path.

NumCalibrationBatches — Number of calibration batches

50 (default) | positive integer

Numeric value specifying the number of batches for int8 calibration. This option is applicable only when DataType is set to 'int8'. For more information, see Number of calibration batches.

TargetLib — Target library name

'tensorrt' (default) | character vector

A read-only value that specifies the name of the target library.

Examples

Specify Configuration Parameters for MEX Function Generation for the ResNet-50 Network

Create an entry-point function resnet_predict that uses the imagePretrainedNetwork function to load the dlnetwork object that contains the ResNet-50 network. For more information, see Code Generation for dlarray

function out = resnet_predict(in)

dlIn = dlarray(in, 'SSCB');
persistent dlnet;
if isempty(dlnet)
    dlnet = imagePretrainedNetwork('resnet50');
end

dlOut = predict(dlnet, dlIn);
out = extractdata(dlOut);

Create a coder.gpuConfig configuration object for MEX code generation.

cfg = coder.gpuConfig('mex');

Set the target language to C++.

cfg.TargetLang = 'C++';

Create a coder.TensorRTConfig deep learning configuration object. Assign it to the DeepLearningConfig property of the cfg configuration object.

cfg.DeepLearningConfig = coder.DeepLearningConfig('tensorrt');

Use the -config option of the codegen function to pass the cfg configuration object. The codegen function must determine the size, class, and complexity of MATLAB^® function inputs. Use the -args option to specify the size of the input to the entry-point function.

codegen -args {ones(224,224,3,'single')} -config cfg resnet_predict;

The codegen command places all the generated files in the codegen folder. The folder contains the CUDA code for the entry-point function resnet_predict.cu, header and source files containing the C++ class definitions for the convoluted neural network (CNN), weight, and bias files.

Version History

Introduced in R2018b

R2025a: The NVIDIA TensorRT library not installed by default in MATLAB

GPU Coder no longer pre-installs the NVIDIA TensorRT^TM library with MATLAB for generating MEX functions or accelerating Simulink^® simulations on a GPU. GPU Coder throws an error if the TensorRT library is not found in MATLAB. You must install the TensorRT library by using gpucoder.installTensorRT.