Need guidance on validating my results on training for Object detection on MATLAB and next steps

I have trained a detector model using Resnet50 layers for multiple object detection for single images. The Recall and F1 score are very low, i don't know what these mean and don't know if my model is a good model. I have tested with augmented images of varying illumination, which was not part of my testData, the detector did identify some classes if not all with accurate bounding boxes. I need clarification and next steps to better my model, if is not, but i dont want to go through the whole training since it took me a significant amount of time. I need arguments of justification i can include as this is for my thesis research project.

Attached is the MATHLAB code

Answer 1

[Shantanu Dixit]

Hi Evangeline,

I understand you’ve trained a ResNet50-based object detector, and while the precision is perfect (1.0 for all classes), the recall and F1 scores are quite low. Precision refers to the proportion of correct predictions among all predictions for a class. A precision of 1 means all detected objects for that class were correct (no false positives). Recall refers to the proportion of actual objects detected. A low recall (e.g., 0.1 for "axion") means only 10% of true objects were detected (high false negatives).

This means that the model is extremely cautious when it detects an object, it’s almost always correct, but it’s missing a lot of actual objects in the images (hence the low recall).

The fact that it works reasonably well on augmented images (like those with varying lighting) is a good sign, it shows the model has some generalization ability. But the low recall suggests it might need a few tweaks to catch more of those missed objects without starting from scratch.

Some next steps that can prove to be useful can be:

• Fine-tune with more varied examples: Since the model struggles with recall, it might help to add more training examples (or augmentations) for the classes it’s missing most (like "axion" with recall <0.1). Instead of retraining the whole model you can unfreeze the last few layers and train a bit longer on a more balanced dataset.
• Adjust the detection threshold, the model seems to be making predictions when it’s 100% sure. Lowering the confidence threshold slightly might help it detect more objects (though it might result in a few more false positives in exchange).

The model seems to be highly precise (precision = 1) which can be great if the task involves avoiding false alarms. Improving recall seems like a logical next step without fully retraining the model. Also the generalization ability seems as it is able to detect instances in the augmented images.

You can also refer to the webinar by MathWorks on Applied Machine Learning for more information on metrics for ML models: https://www.mathworks.com/videos/series/applied-machine-learning.html

Hope this helps!