- https://www.mathworks.com/help/vision/ref/detectharrisfeatures.html
- https://www.mathworks.com/help/vision/ref/detectmineigenfeatures.html
"detectHarrisFeatures" much slower when deploying to Jetson Nano GPU (cuda) than CPU
3 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
Good evening, i am developing a basic vision tracker with MALTAB/Simulink to deploy on Jetson Nano board.
The code works fine when targeting the CPU: "Generate GPU code" checkbox in the Code Generation Pane of Configuration Parameters is left unchecked. Frame rate is pretty high on my Jetson Nano and corner points are acquired very fast every time. However, if i check the afore mentioned checkbox in order to generate cuda, the tracking stage executes flawlessly but the application hangs when it has to compute new corners and it takes several seconds (almost a minute). Investigating a little bit, it seems like that for some reasons, neither "detectHarrisFeatures" nor "detectMinEigenFeatures" work well for code generation with cuda. The image size is 360 rows x 640 cols.
The parameters in the pane "GPU code" under Code Generation in Configuration Parameters are left at their default values except for Compute capability which is set to 5.3 according to NVIDIA documentation for Jetson Nano.
The code is as follows (and it works perfectly also when executing the simulink model on my laptop running Windows 10 either with or without GPU acceleration).
function pointsout = KLTtraker(frameGS, nMinPoints, BlockSize, NumPyramidLevels) %#codegen
persistent pointTracker
persistent numPts
persistent oldPoints
persistent newTrack
% persistent bboxPoints
%% Initialize persistent variables
% Create the face detector object.
if isempty(numPts)
numPts = 0;
end
if isempty(oldPoints)
oldPoints = single([0 0]);
end
if isempty(newTrack)
newTrack = true;
end
% Create the point tracker object.
if isempty(pointTracker)
pointTracker = vision.PointTracker('NumPyramidLevels',NumPyramidLevels,'BlockSize',BlockSize);
initialize(pointTracker, single([10 10]), frameGS);
end
%% Detection and Tracking
% Find corner points inside the detected region.
if newTrack
% assert(all(size(ROI) == [1,4]));
% coder.varsize('xyPoints',[100,2]);
% pointsy = rand(100,1)*360;
% pointsy(pointsy<1) = 1;
% pointsx = rand(100,1)*640;
% pointsx(pointsx<1) = 1;
% xyPoints = single([pointsx pointsy]);
points = detectHarrisFeatures(frameGS);
points = points.selectStrongest(100);
% Re-initialize the point tracker.
xyPoints = points.Location;
assert(size(xyPoints,1) <500);
numPts = size(xyPoints,1);
% coder.varsize('xyPoints',[500,2]);
% extract points from ROI
if numPts < nMinPoints %insufficient number of points extracted from ROI
% quit tracking and return do detect phase
newTrack = true;
pointTracker.reset();
pointsout = single([0 0]);
return
else
newTrack = false;
% initialize Points tracker
if ~pointTracker.isLocked
initialize(pointTracker, xyPoints, frameGS);
end
step(pointTracker, frameGS);
setPoints(pointTracker,xyPoints);
%Save a copy of the points
oldPoints = xyPoints;
end
end
% Tracking mode.
[xyPoints, isFound] = step(pointTracker, frameGS);
assert(size(xyPoints, 1) < 101);
visiblePoints = xyPoints(isFound, :);
oldInliers = oldPoints(isFound, :);
numPts = size(visiblePoints, 1);
if numPts >= nMinPoints
% I don't really care about knowing the transformation here: i just
% want to reject outliers.
[~, ~, visiblePoints] = estimateGeometricTransform(...
oldInliers, visiblePoints, 'similarity', 'MaxDistance', 4);
% there aren't sufficient inliers: quit tracking
if visiblePoints < nMinPoints
newTrack = true;
pointTracker.reset();
pointsout = single([0 0]);
return
% all conditions for successful tracking are met: continue and output
% points
else
pointsout = visiblePoints;
oldPoints = visiblePoints;
setPoints(pointTracker, oldPoints);
end
% KLT tracker has found an insufficient number of correspondances: quit
% tracking
else
newTrack = true;
pointTracker.reset();
pointsout = single([0 0]);
return
end
end
Any help is kindly appreciated.
Thanks in advance,
Marco.
0 Commenti
Risposte (1)
Milan Bansal
il 31 Mar 2024
Hi marco fiorio
I understand that you are developing a basic vision tracker using MATLAB/Simulink to deploy on a Jetson Nano board. Your application performs well when targeting the CPU, but you encounter significant delays when generating CUDA code, specifically during the computation of new corners using "detectHarrisFeatures" or "detectMinEigenFeatures".
As per the documentation of "detectHarrisFeatures" and "detectMinEigenFeatures", these fucntions do not support GPU code generation. MATLAB might allow the code generation process to complete, but it will automatically fall back to executing certain operations on the CPU if those operations are not supported on the GPU. This means that the code will not fully benefit from GPU acceleration for those specific operations, leading to performance bottlenecks.
The below solutions may help to resolve this issue.
Try converting the "frameGS" to GPU Array before passing into "detectHarrisFeatures" function.
frameGS = gpuArray(frameGS);
points = detectHarrisFeatures(frameGS);
Alternatively, write a custom Harris feature detector function which will support GPU Code generation. Please refer to the below code snippet to implement basic Harris feature detection.
function corners = customHarrisFeaturesGPU(frameGS)
% Ensure the input is a gpuArray for GPU computation
frameGS = gpuArray(frameGS);
% Parameters
k = 0.04; % Sensitivity factor
threshold = 1e-6; % Threshold
sigma = 1; % Standard deviation
% 1: Gradients of the image
[Gx, Gy] = imgradientxy(frameGS, 'Sobel');
% 2: Compute elements of the Harris matrix M for each pixel
Ix2 = imgaussfilt(Gx.^2, sigma);
Iy2 = imgaussfilt(Gy.^2, sigma);
Ixy = imgaussfilt(Gx.*Gy, sigma);
% 3: Harris response for each pixel
R = (Ix2 .* Iy2 - Ixy.^2) - k * (Ix2 + Iy2).^2;
% 4: Thresholding to find potential corner points
R(R < threshold) = 0;
% 5: Non-maximum suppression
corners = imregionalmax(R);
% Extract corner positions
[y, x] = find(corners);
corners = [x, y];
end
Please refer to the following documentation links to learn more about "detectHarrisFeatures" and "detectMinEigenFeatures" functions.
Hope it helps!
0 Commenti
Vedere anche
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!