Mask RCNN custom data training. Problem with JSON to mat conversion

Hi,

I will be really very grateful if somebody can help me here. I am stuck for more than 6 months :(

So, I am trying to train mask RCNN using custom data set. I am following the MATLAB Mask RCNN example here: https://www.mathworks.com/help/vision/ug/example-InstanceSegmentationUsingMaskRCNNDeepLearningExample.html#PerformInstanceSegmentationUsingMaskRCNNExample-1

I have used image labeler to annotate the images and then used this exportgtruthtoJSON program https://www.mathworks.com/help/vision/ug/export-ground-truth-object-to-custom-and-coco-json-files.html.

However, when I am using this as an input for the Mask RCNN, 'Extract the COCO annotations to MAT files using the unpackAnnotations helper function is not working'. The command window shows the processing is done. but in reatlity the 'annotations_unpacked' folder remains empty.

Please help, if you know how to solve this issue.

Thank you!

Answer 1

[Debraj Maji]

Hi @Sam

I see that you have labelled data from the COCO dataset as your ground truth using the Image Labeler and exported it to JSON file. You are then using it as an input to a masked RCNN network for training/inference.

As per the documentation, this is a limitation of the current Iimage Labeller as the JSON file exported by the “exportGroundTruthToJSON” helper function cannot be imported back into MATLAB as a ground truth object.

Possible solutions include:

• Using a separate pre-trained network to annotate the images and export the annotations to JSON format.
• Using the COCO API for MATLAB as found here: coco API-MATLAB

You may go through the following MathWorks documentation link to learn more about exporting ground truth objects to custom and COCO JSON files:

Export Ground Truth Object to Custom and COCO JSON Files.

[Connor Seavers]

Hi,

I have run into the same issue as Sam recently.

I have labeled my own data in the imageLabeler app and then followed the instructions from the "Export Ground Truth Object to Custom and COCO JSON Files" page to create a .json file with the proper COCO format. As far as I can tell, it matches the format of the JSON file provided with the Mask RCNN example. However, whenever I use the new JSON file as the input for Mask RCNN, it still fails to unpack the annotations into separate MAT files like it says it will.

Any help would be greatly appreciated.

Thanks.

Answer 2

[Birju Patel]

You do not need convert ground truth exported from Image Labeler into the COCO format in order to train Mask R-CNN. The way the example is authored has caused this confusion.

I've attached a function (exampleInstanceSegementationTrainingData) to convert groundTruth into a datastore that can be used for training Mask R-CNN as well as SOLO v2.

Here is an example of how to use the attached function:

% Load groundTruth (attached)

gt = load("gTruth.mat");

imds = imageDatastore({'visionteam.jpg','visionteam1.jpg'});

src = groundTruthDataSource(imds);

gTruth = groundTruth(src,gt.labelDef,gt.labelData);

% Create training datastores from ground truth. This function converts

% polygon data into masks and writes the resulting masks to a folder named

% masksAndBoxes. You can use this datastore directly to train Mask R-CNN.

ds = exampleInstanceSegmentationTrainingData(gTruth,OutputFolder="masksAndBoxes");

preview(ds)

We'll look to add similar functionality to Computer Vision Toolbox in a future release.