Main Content

Deploy and Run Sobel Edge Detection with I/O on NVIDIA Jetson Nano

This example uses:

Open Live Script

This example shows you how to deploy Sobel edge detection application that uses a Raspberry Pi Camera Module V2 and displays the edge detected output on the NVIDIA® Jetson™ Nano Hardware. The Sobel Edge Detection on NVIDIA Jetson Nano Using Raspberry Pi Camera Module V2 example showed how to capture image frames from the Raspberry Pi Camera Module V2 on an NVIDIA Jetson Nano hardware and process them in the MATLAB® environment. This example shows how to generate code for accessing I/O peripherals (camera and display) and perform processing on the NVIDIA Jetson Nano hardware.

Prerequisites

Target Board Requirements

  • NVIDIA Jetson Nano embedded platform.

  • NVIDIA CUDA toolkit and libraries installed on the board.

  • V4L2 and SDL (v1.2) libraries on the board.

  • GStreamer libraries on the board.

  • Raspberry Pi Camera Module V2 connected to the CSI host port of the target.

  • Ethernet crossover cable to connect the target board and host PC (if you cannot connect the target board to a local network).

  • Environment variables on the target for the compilers and libraries. For more information, see Install and Setup Prerequisites for NVIDIA Boards.

  • The profiling workflow of this example depends on the profiling tools from NVIDIA that accesses GPU performance counters. From CUDA toolkit v10.1, NVIDIA restricts access to performance counters to only admin users. To enable GPU performance counters to be used by all users, see the instructions provided in Permission issue with Performance Counters (NVIDIA).

Development Host Requirements

Connect to NVIDIA Jetson Nano

The support package uses an SSH connection over TCP/IP to execute commands while building and running the generated CUDA code on the Jetson Nano platforms. Connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. For information on how to set up and configure your board, see NVIDIA documentation.

Create Jetson Object

To communicate with the NVIDIA hardware, create a live hardware connection object by using the jetson function.

hwobj = jetson('jetson-nano-name','ubuntu','ubuntu');

When connecting to the target board for the first time,you must provide the host name or IP address, user name, and password of the target board. On subsequent connections, you do not need to supply the address, user name, and password. The hardware object reuses these settings from the most recent successful connection to an NVIDIA board.

By default, this example reuses the settings from the most recent successful connection to a NVIDIA Jetson board. 

hwobj = jetson;
Checking for CUDA availability on the Target...
Checking for 'nvcc' in the target system path...
Checking for cuDNN library availability on the Target...
Checking for TensorRT library availability on the Target...
Checking for prerequisite libraries is complete.
Gathering hardware details...
Checking for third-party library availability on the Target...
Gathering hardware details is complete.
 Board name              : NVIDIA Jetson Nano Developer Kit
 CUDA Version            : 10.2
 cuDNN Version           : 8.2
 TensorRT Version        : 8.2
 GStreamer Version       : 1.14.5
 V4L2 Version            : 1.14.2-1
 SDL Version             : 1.2
 OpenCV Version          : 4.1.1
 Available Webcams       : Logitech Webcam C925e
 Available GPUs          : NVIDIA Tegra X1
 Available Digital Pins  : 7  11  12  13  15  16  18  19  21  22  23  24  26  29  31  32  33  35  36  37  38  40

During the hardware live object creation, the support package performs hardware and software checks, installs MATLAB IO server on the target board, and gathers information on peripheral devices connected to the target. This information is displayed in the Command Window. In case of a connection failure, a diagnostics error message is reported at the MATLAB command line. If the connection has failed, the most likely cause is incorrect IP address or host name.

List Available Cameras

Run the getCameraList function of the hwobj object to find the available cameras. If this function outputs an empty table, then try re-connecting the camera and execute the function again.

camlist = getCameraList(hwobj);
          Camera Name          Video Device     Available Resolutions    Pixel Formats
    _______________________    _____________    _____________________    _____________

    "Logitech Webcam C925e"    "/dev/video0"    "(View resolutions)"      "YUYV,MJPG"

The getCameraList function lists the optimum resolutions supported by the camera sensor. At these resolutions, the image acquisition pipeline works efficiently. Based on the requirements of your algorithm, you can pick any supported resolution.

This example uses the first camera from the list and the first supported resolution. 

camName = table2array(camlist(1,"Camera Name"));
camResolution = [1280 720];

Verify GPU Environment on Target Board

To verify that the compilers and libraries necessary for running this example are set up correctly, use the coder.checkGpuInstall (GPU Coder) function.

envCfg = coder.gpuEnvConfig('jetson');
envCfg.BasicCodegen = 1;
envCfg.Quiet = 1;
envCfg.HardwareObject = hwobj;
coder.checkGpuInstall(envCfg);

Prepare Sobel Edge Detection Application for Deployment

The sobelEdgeDetection.m entry-point function implements an algorithm to capture live images from a camera connected to the hardware board, apply edge detection algorithm, and display the result on a monitor connected to the hardware board. The algorithm consists of a 3-by-3 Sobel operator that is applied to the image in horizontal and vertical directions, and then threshold against a constant value.

type sobelEdgeDetection.m
function sobelEdgeDetection(cameraName,resolution) %#codegen
%SOBELEDGEDETECTION() Entry-point function for Sobel edge detection
%   This function is the entry-point function that supports examples in
%   MATLAB Coder Support Package for NVIDIA Jetson and NVIDIA DRIVE 
%   Platforms that use Sobel algorithms for edge detection.

%   Copyright 2020-2023 The MathWorks, Inc.

hwobj = jetson;
camObj = camera(hwobj,cameraName,resolution);
dispObj = imageDisplay(hwobj);

% Sobel kernel
kern = [1 2 1; 0 0 0; -1 -2 -1];

% Main loop
for k = 1:1000
    % Capture the image from the camera on hardware.
    img = snapshot(camObj);
    
    % Finding horizontal and vertical gradients.
    h = conv2(img(:,:,2),kern,'same');
    v = conv2(img(:,:,2),kern','same');
    
    % Finding magnitude of the gradients.
    e = sqrt(h.*h + v.*v);
    
    % Threshold the edges
    edgeImg = uint8((e > 100) * 240);
    
    % Display image.
    image(dispObj,edgeImg');
end

end

Generate CUDA Code for the Jetson Target Using GPU Coder

To generate a CUDA executable that you can deploy on to an NVIDIA target, create a GPU code configuration object for generating an executable.

cfg = coder.gpuConfig('exe');

To create a configuration object for the Jetson platform and assign it to the Hardware property of the code configuration object cfg, use the coder.hardware function.

cfg.Hardware = coder.hardware('NVIDIA Jetson');

To specify the folder for performing remote build process on the target board, use the BuildDir property. If the specified build folder does not exist on the target board, then the software creates a folder with the given name. If no value is assigned to cfg.Hardware.BuildDir, the remote build process occurs in the last specified build folder. If there is no stored build folder value, the build process takes place in the home folder.

cfg.Hardware.BuildDir = '~/remoteBuildDir';

Set the GenerateExampleMain property to generate an example C++ main file and compile it. This example does not require modifications to the generated main files. 

cfg.GenerateExampleMain = 'GenerateCodeAndCompile';

To generate CUDA code, use the codegen function and pass the GPU code configuration and the input specifications for the sobelEdgeDetection.m entry-point function. After the code generation takes place on the host, the generated files are copied over and built on the target board.

inputArgs = {coder.Constant(camName),coder.Constant(camResolution)};
codegen('-config ',cfg,'-args',inputArgs,'sobelEdgeDetection','-report');
Code generation successful: View report

Run Sobel Edge Detection on Target Board

To run the generated executable on the target board, use the runApplication function.

Set the appropriate display environment.

hwobj.setDisplayEnvironment('0.0');

Run the application on target. 

pid = runApplication(hwobj,'sobelEdgeDetection');
### Launching the executable on the target...
Executable launched successfully with process ID 28340.
Displaying the simple runtime log for the executable...

Note: For the complete log, run the following command in the MATLAB command window:
system(hwobj,'cat /home/ubuntu/remoteBuildDir/MATLAB_ws/R2023b/home/lnarasim/Documents/MATLAB/ExampleManager/lnarasim.Bdoc23b.j2336540/nvidia-ex10094443/sobelEdgeDetection.log')

A window opens on the target hardware display showing the Sobel edge detection output of the live camera feed.

MicrosoftTeams-image.png

Stop the Application

To kill the sobelEdgeDetection application on the board, use the killApplication function.

killApplication(hwobj,'sobelEdgeDetection');