Hello everyone, I need help. I am working on a robot arm project using MATLAB and a webcam to detect colors. Do you have any lessons or tutorials for me?

Hello everyone, I’m working on a robot arm project using MATLAB and Arduino. I use my phone as a webcam to detect colors (like red or green). If the system detects a red object, the robot arm moves and picks it up, then puts it in the red storage area.

Now, I want to improve the project. Instead of the robot always picking red automatically, I want to click on an object in the camera video (GUI in MATLAB) to choose which color to pick — for example, if I click on a red object, it will pick up red. If I click on green, it will pick up green.

I find it hard to research or find tutorials. Can someone help me or guide me with examples or lessons?

Answer 1

[Image Analyst]

Sounds like you have a very controlled environment with known and fixed colors. What you could do with the image is to first mask the image to find all the red objects, and find all the green objects. You can get masks for those colors using the Color Thresholder on the Apps tab of the tool ribbon. Export one function for red, then find green and export the green function. Remember to change their names since they both can't use the same function name which the Export button exports.

Once you have the masks you might have to clean up the mask with functions like bwareafilt or bwareaopen to specify the size range you expect the colored regions to be in.

[Image Analyst]

See attached video where I track a green marker.

[Chan]

Thank you, sir. I was busy with my exam and project, so I forgot to reply. Sorry

[Chan]

Ohh thank you I will try to test it

[Chan]

I made it with webcam how can I do

[Image Analyst]

So you made the webcam image acquisition part already? Sounds like it because you said "I made it with webcam". So if you're capturing the image with your webcam you just need to do the color segmentation on the captured image, using code like I gave you.

Once you have the location of the colored parts you need to talk to your robot arm to move to the desired location to manipulate the part. I don't know how to interface MATLAB to your robot. I suggest you contact your robot manufacturer.

You might also look at the Computer Vision Toolbox which has camera calibration capability so that will turn coordinates in an image (row, column) into actual physical locations in your laboratory. See https://www.mathworks.com/help/vision/examples.html?category=camera-calibration

[Chan]

Yes, it means I’m using MATLAB connected to my phone as a webcam. The webcam acts like a live video feed, and I’m using it for color detection. I know how to control my robot, but I want to learn more about how to detect RGB colors using a webcam in real time — something like what OpenCV does.

[Chan]

I’m new to studying MATLAB. Please guide me. I made this project because my teacher told me to use only a camera connected to a phone. So, I used software called Iriun Webcam to connect my phone to MATLAB. Here my Code with Iriun Webcam :

cam = webcam('Iriun Webcam'); preview(cam)

[Image Analyst]

Uh, it's going a lot more than that. Plus you can't direct a robot to pick up something from a handheld cell phone camera because the objects are moving all over the place in your image. Anyway, try getsnapshot

Also see the attached demmo that works with my Logitech webcam.

[Chan]

Ohh, thank you! I'm going to study now. Once I finish my code, I'll send it to you to use with mine. Thanks for your help! 😁

[Chan]

Here is my code. I just modified your code to work with my webcam. But how can I detect red and blue colors? Please check my code. Thank you, sir!

[Image Analyst]

You forgot to attach any images so I don't know what "red" and "Blue" look like in your situation. Please attach two or three image that have red and blue in them.

It doesn't look like you used the Color Thresholder and exported functions like I recommended.

I'll be gone until later this evening.

[Chan]

I'm not using an image file; I connected it to a webcam.

[Image Analyst]

Yes but the webcam snaps an image. You can then call imwrite right after you snap it to save it to a file and upload it here. Otherwise, I gave you a demo where I track a green marker pen from a webcam so you can modify that.

[Chan]

Thank you sir if i have any problem can i ask you more

Answer 2

[Jacob Mathew]

Hi Chan,

Given that you already have a model to detect objects based on color, you can modify the code to return the various objects of color as a list. You can either select the object from the list of detected objects but if you want to use a GUI, then you can follow the below MATLAB Answers post where the author gets the coordinates from the clicked image:

https://www.mathworks.com/matlabcentral/answers/265820-saving-mouse-click-coordinates-on-image-in-gui

You can then check whether the clicked coordinates is within the bounding box of the objects and if so, the corresponding object information can be passed onto the robotic arm to be picked up.

[Chan]

Thank you, sir. I was busy with my exam and project, so I forgot to reply. Sorry

[Chan]

It can detect the color when I right-click while moving