What is the fastest way to display frames from a camera in MATLAB?

I am running the latest version of Matlab (2023b) and I have access to all of the latest verions of the toolboxes. The computer I am using for this has a i5-12600KF and RTX 3060Ti. The monitor I am using runs at 60fps.

I have a 1 megapixel camera connected by a videoinput object that is running at 120fps. I have two goals (1) to display on a UI the most recent raw frame at 30fps and (2) to process/save each frame with a simple algorithm such as the mean intensity of the image. Note: On this computer timeit says it takes mean(I,[1,2]) <2ms per frame.

I do not need to save the frames after they are processed. I just need to save one scalar value per image to memory. I also do not need to display all of the frames in this case it would be great if I could display every fourth frame. I would rather drop frames for the display and show the most recent frame so that there is minmal lag.

I have been testing serval diffrent approches. With my best aproach I am averaging 10fps(see below). However, using Python I can easily get 30fps. Is 10fps just the best I can do with MATLAB on my setup?

function testFPS()
    clc; clear;
    
    ax = axes();
    latestImage = zeros(1000,1000,'uint8'); % This will be updated by processFrames
    imageH = image(ax, latestImage);
    
    n_frames = 1e5;
    data = zeros(0, 1); % Thi will be updated by processFrames
    
    viObj = videoinput('gige', 1, 'Mono8');
    viObj.FramesPerTrigger = 120/30; % This is set to give the display 4/120=30fps
    viObj.TriggerRepeat = Inf;
    viObj.TriggerFcn = @(obj,~) processFrames(obj);
    start(viObj)
    
    % Seperate timer to run UI updates with BusyMode='drop' to keep only the latest
    % frames displayed.
    tmr = timer();
    tmr.BusyMode = 'drop'; 
    tmr.ExecutionMode = 'fixedRate';
    tmr.Period = 1/30;
    tmr.TasksToExecute = Inf;
    tmr.TimerFcn = @(~,~) updateUI(imageH);
    start(tmr)
    
    function processFrames(obj)
        frames = getdata(obj, 4);
        latestImage = frames(:,:,4);
        data(end+1:end+4) = mean(frames, [1,2]); % I do not have a fixed time for how long this recording could be so no pre-alocating data's size.
    end
    
    function updateUI(imageH)
        imageH.CData = latestImage;
        drawnow;
    end
end

Answer 1

[Ayush]

Hi James,

From what I can gather from the code snippet provided and your query, it seems that you want to display the frames from your camera in a faster way specifically greater than 10 fps. Here are some potential areas in your code to optimize its performance which may lead to a better fps capture:

- Pre-allocation of the data (frame) array can help in increasing efficiency. Since you don't know the exact number of frames, you can preallocate a large array and tracking the index of the frames would help avoid the overhead of dynamic resizing. Here is an example code for your reference:

% Preallocate a large array with an estimation of the maximum number of frames
estimatedMaxFrames = 1e6; % Adjust this number based on the expected maximum
data = zeros(estimatedMaxFrames, 1);
currentIndex = 0; % Keep track of the current index in the data array

- You can avoid the overhead in the “processFrames” function in your code by removing some unnecessary execution such as copying the data or using "getdata" to fetch 4 frames instead of the latest frame.Here is an example code for your reference:

function processFrames(obj)
    frame = getdata(obj, 1);
    currentIndex = currentIndex + 1;
    data(currentIndex) = mean(frame, [1,2]);
    latestImage = frame; % Update the latestImage directly
end

- The timer object can introduce additional overheads that may pass by unnoticed. Instead use a while loop with a pause functionality. Here is an example code for your reference:

% Remove the timer code and replace it with a while loop in the main function
while islogging(viObj)
    pause(1/30); % Pause for the display frame rate
    updateUI(imageH);
end

- Additionally, you can use MATLAB’s built-in profiler to identify bottlenecks in your code. Please refer to the below documentationn and code snippet to know more about the "profile" function:

https://www.mathworks.com/help/matlab/ref/profile.html

profile on; % Start profiling

% ... Your code ...

profile off; % End profiling
profview; % View the profiling report

Hope this helps!