How can I convert a ROS Image topic into an matrix that I can process in Simulink?

I am trying to find a simple way of converting images received from ROS into matrix form that can then be processed in Simulink. On Matlab side there are the writeImage and readImage functions that can be used to convert ROS Image to matlab image but I am not sure what should be the correct way of doing this on Simulink side.

One option would be to get the raw data from Image topic and reorganize it into matrix form, but that processing is always dependent on the image size and format. Is there any simple way of doing this?

Answer 1

[Swarooph]

You are correct that readImage and writeImage functions are for MATLAB use only since they work on ROS topic objects. However in Simulink we work with buses. The straight forward way that I could think of to deal with this is to unpack the image on your own in a MATLAB function block. So you could have a model like this:

And inside the MATLAB function block, you could write the following function:

function img = fcn(msg)
%#codegen

%Get msg details
imgData = msg.Data;
imgH = msg.Height;
imgW = msg.Width;

%Create blank image output with required size
img = zeros(msg.Width,msg.Height,3);

%Reshape image data
imgR = reshape(imgData(1:3:end),imgW,imgH)';
imgG = reshape(imgData(2:3:end),imgW,imgH)';
imgB = reshape(imgData(3:3:end),imgW,imgH)';
img(:,:,1) = imgR;
img(:,:,2) = imgG;
img(:,:,3) = imgB;

Note that this example assumes image is in the RGB color space.

[hendriyawan achmad]

I tried to run your code and got this error notification, "Data 'image' is inferred as a variable size matrix, while its properties in the Model Explorer specify its size as inherited or fixed. Please check the 'Variable Size' check box and specify the upper bounds in the size field." am I doing wrong?

[Michele Mondini]

How about greyscale images? Did you ever try to do the same with mono8 encoding?

[Antti]

We had to use fixed size arrays instead of msg.Width. If someone knows how to make this work dynamically, I would like to know ;)

imgW = 1280;%msg.Width;
imgH = 720;%msg.Width;

The greyscale image work the same way but instead of parsing the RGB, you can use the whole array as it is. Depending on the data types, you might also need to converts the data to doubles.

img = zeros(imgH,imgW);
%Reshape image data
img = double(reshape(msg.Data(1:end),imgW,imgH)');