Get Started with Deep Learning FPGA Deployment on Xilinx ZCU102 SoC

This example uses:

Deep Learning HDL Toolbox Deep Learning HDL Toolbox
Deep Learning HDL Toolbox Support Package for Xilinx FPGA and SoC Devices Deep Learning HDL Toolbox Support Package for Xilinx FPGA and SoC Devices
Deep Learning Toolbox Deep Learning Toolbox

This example shows how to create, compile, and deploy a dlhdl.Workflow object that has a handwritten character detection network as the network object by using the Deep Learning HDL Toolbox™ Support Package for Xilinx FPGA and SoC. Use MATLAB® to retrieve the prediction results from the target device.

Prerequisites

Xilinx ZCU102 SoC development kit.
Deep Learning HDL Toolbox™
Deep Learning HDL Toolbox™ Support Package for Xilinx FPGA and SoC
Deep Learning Toolbox™

Load the Pretrained Network

To load the pretrained network, that has been trained on the Modified National Institute Standards of Technology (MNIST) database[1], enter:

net = getDigitsNetwork;

View the layers of the pretrained network by using the deepNetworkDesigner app.

deepNetworkDesigner(net)

Create Target Object

Create a target object that has a custom name for your target device and an interface to connect your target device to the host computer. Interface options are JTAG and Ethernet.

hTarget = dlhdl.Target('Xilinx','Interface','Ethernet')

hTarget = 
  TargetEthernet with properties:

    Interface: Ethernet
    IPAddress: '192.168.1.101'
     Username: 'root'
       Vendor: 'Xilinx'

Create WorkFlow Object

Create an object of the dlhdl.Workflow class. Specify the network and the bitstream name during the object creation. Specify saved pretrained MNIST neural network, snet, as the network. Make sure that the bitstream name matches the data type and the FPGA board that you are targeting. In this example, the target FPGA board is the Xilinx ZCU102 SOC board and the bitstream uses a single data type.

hW = dlhdl.Workflow('network', net, 'Bitstream', 'zcu102_single','Target',hTarget)

hW = 
  Workflow with properties:

      Network: [1×1 dlnetwork]
    Bitstream: [1×1 dnnfpga.bitstream.Bitstream]
       Target: [1×1 dnnfpga.hardware.TargetEthernet]

Compile the MNIST Network

To compile the MNIST network, run the compile function of the dlhdl.Workflow object.

dn = hW.compile;

### Compiling network for Deep Learning FPGA prototyping ...
### Targeting FPGA bitstream zcu102_single.
### An output layer called 'Output1_softmax' of type 'nnet.cnn.layer.RegressionOutputLayer' has been added to the provided network. This layer performs no operation during prediction and thus does not affect the output of the network.
### Optimizing network: Fused 'nnet.cnn.layer.BatchNormalizationLayer' into 'nnet.cnn.layer.Convolution2DLayer'
### Notice: The layer 'imageinput' of type 'ImageInputLayer' is split into an image input layer 'imageinput' and an addition layer 'imageinput_norm' for normalization on hardware.
### The network includes the following layers:
     1   'imageinput'        Image Input         28×28×1 images with 'zerocenter' normalization                (SW Layer)
     2   'conv_1'            2-D Convolution     8 3×3×1 convolutions with stride [1  1] and padding 'same'    (HW Layer)
     3   'relu_1'            ReLU                ReLU                                                          (HW Layer)
     4   'maxpool_1'         2-D Max Pooling     2×2 max pooling with stride [2  2] and padding [0  0  0  0]   (HW Layer)
     5   'conv_2'            2-D Convolution     16 3×3×8 convolutions with stride [1  1] and padding 'same'   (HW Layer)
     6   'relu_2'            ReLU                ReLU                                                          (HW Layer)
     7   'maxpool_2'         2-D Max Pooling     2×2 max pooling with stride [2  2] and padding [0  0  0  0]   (HW Layer)
     8   'conv_3'            2-D Convolution     32 3×3×16 convolutions with stride [1  1] and padding 'same'  (HW Layer)
     9   'relu_3'            ReLU                ReLU                                                          (HW Layer)
    10   'fc'                Fully Connected     10 fully connected layer                                      (HW Layer)
    11   'softmax'           Softmax             softmax                                                       (SW Layer)
    12   'Output1_softmax'   Regression Output   mean-squared-error                                            (SW Layer)
                                                                                                             
### Notice: The layer 'softmax' with type 'nnet.cnn.layer.SoftmaxLayer' is implemented in software.
### Notice: The layer 'Output1_softmax' with type 'nnet.cnn.layer.RegressionOutputLayer' is implemented in software.
### Compiling layer group: conv_1>>maxpool_2 ...
### Compiling layer group: conv_1>>maxpool_2 ... complete.
### Compiling layer group: conv_3>>relu_3 ...
### Compiling layer group: conv_3>>relu_3 ... complete.
### Compiling layer group: fc ...
### Compiling layer group: fc ... complete.

### Allocating external memory buffers:

          offset_name          offset_address     allocated_space 
    _______________________    ______________    _________________

    "InputDataOffset"           "0x00000000"     "368.0 kB"       
    "OutputResultOffset"        "0x0005c000"     "4.0 kB"         
    "SchedulerDataOffset"       "0x0005d000"     "220.0 kB"       
    "SystemBufferOffset"        "0x00094000"     "76.0 kB"        
    "InstructionDataOffset"     "0x000a7000"     "28.0 kB"        
    "ConvWeightDataOffset"      "0x000ae000"     "28.0 kB"        
    "FCWeightDataOffset"        "0x000b5000"     "76.0 kB"        
    "EndOffset"                 "0x000c8000"     "Total: 800.0 kB"

### Network compilation complete.

Program Bitstream onto FPGA and Download Network Weights

To deploy the network on the Xilinx ZCU102 SoC hardware, run the deploy function of the dlhdl.Workflow object. This function uses the output of the compile function to program the FPGA board by using the programming file. It also downloads the network weights and biases. The deploy function starts programming the FPGA device, displays progress messages, and the time it takes to deploy the network.

hW.deploy

### Programming FPGA Bitstream using Ethernet...
### Attempting to connect to the hardware board at 192.168.1.101...
### Connection successful
### Programming FPGA device on Xilinx SoC hardware board at 192.168.1.101...
### Attempting to connect to the hardware board at 192.168.1.101...
### Connection successful
### Copying FPGA programming files to SD card...
### Setting FPGA bitstream and devicetree for boot...
# Copying Bitstream zcu102_single.bit to /mnt/hdlcoder_rd
# Set Bitstream to hdlcoder_rd/zcu102_single.bit
# Copying Devicetree devicetree_dlhdl.dtb to /mnt/hdlcoder_rd
# Set Devicetree to hdlcoder_rd/devicetree_dlhdl.dtb
# Set up boot for Reference Design: 'AXI-Stream DDR Memory Access : 3-AXIM'
### Programming done. The system will now reboot for persistent changes to take effect.
### Rebooting Xilinx SoC at 192.168.1.101...
### Reboot may take several seconds...
### Attempting to connect to the hardware board at 192.168.1.101...
### Connection successful
### Programming the FPGA bitstream has been completed successfully.
### Loading weights to Conv Processor.
### Conv Weights loaded. Current time is 19-Jun-2024 09:56:28
### Loading weights to FC Processor.
### FC Weights loaded. Current time is 19-Jun-2024 09:56:28

Run Prediction for Example Image

To load the example image, execute the predict function of the dlhdl.Workflow object, and then display the FPGA result, enter:

inputImg = imread('five_28x28.pgm');
inputImg = dlarray(single(inputImg), 'SSCB');

Run prediction with the profile 'on' to see the latency and throughput results.

[prediction, speed] = hW.predict(inputImg,'Profile','on');

### Finished writing input activations.
### Running single input activation.


              Deep Learning Processor Profiler Performance Results

                   LastFrameLatency(cycles)   LastFrameLatency(seconds)       FramesNum      Total Latency     Frames/s
                         -------------             -------------              ---------        ---------       ---------
Network                      41735                  0.00019                       1              42578           5167.0
    imageinput_norm           5613                  0.00003 
    conv_1                    6751                  0.00003 
    maxpool_1                 4786                  0.00002 
    conv_2                    4877                  0.00002 
    maxpool_2                 3066                  0.00001 
    conv_3                    7189                  0.00003 
    fc                        9421                  0.00004 
 * The clock frequency of the DL processor is: 220MHz

[val, idx] = max(prediction);
fprintf('The prediction result is %d\n', idx-1);

The prediction result is 5

Bibliography

LeCun, Y., C. Cortes, and C. J. C. Burges. "The MNIST Database of Handwritten Digits." https://yann.lecun.com/exdb/mnist/.