Error when trying to perform TransferLearning with Yolov4 Object Detector: The network predicted invalid bounding boxes during training. Try reducing the learning rate.

Hi all, I am trying to train an tiny-yolo-v4 object detector network to detect new objects with TransferLearning.

I am using SGDM optimizer as an training option, and its default learning rate is 0.01. However, when I start to train the detector with Learning Rate = 0.01, following error occurs and training is stopped:

Error using images.dltrain.internal.ParallelTrainer/fit
Error detected on workers 1 2.

Error in images.dltrain.internal.dltrain (line 102)
net = fit(networkTrainer);

Error in trainYOLOv4ObjectDetector (line 188)
[trainedDetector,infoTrain] = images.dltrain.internal.dltrain(mbq,detector,options,lossFcn,metrics,validationPatienceMetric,'ExperimentMonitor',params.ExperimentMonitor);

Error in TrainYoloV4v2 (line 110)
trainedDetector = trainYOLOv4ObjectDetector(augmentedTrainingData,detector,options)

Caused by:
    Error using trainYOLOv4ObjectDetector>iGetMaxIOUPredictedWithGroundTruth
    The network predicted invalid bounding boxes during training. Try reducing the learning rate

Any ideas on why this happens? When I decrease the learning Rate to 0.001, training goes fine but it takes a long time to the alghoritm to converge and get a low Training/Validation loss. As default learning Rate for SGDM is 0.01, I would like to know why this value is not working.

I am attaching my code for reference.

initialDetector = yolov4ObjectDetector("tiny-yolov4-coco")
net = initialDetector.Network

data = load("LEDMatrixData7.mat");
trainingData = data.LEDMatrixData7;

dataDir = fullfile("C:\Users\Fortu\OneDrive\Área de Trabalho\OCC\OCC_codes\pretrained-yolo-v4-main");
trainingData.imageFilename = fullfile(dataDir,trainingData.imageFilename);

% Split the dataset into training, validation, and test sets. 
% Select 80% of the data for training, 10% for validation, 
% and the rest for testing the trained detector.

trainingData{:,2} = trainingData{:,2}{1};

rng("default");
shuffledIndices = randperm(height(trainingData));
idx = floor(0.8 * length(shuffledIndices));

trainingIdx = 1:idx;
trainingDataTbl = trainingData(shuffledIndices(trainingIdx),:);

validationIdx = idx+1: idx + 1 + floor(0.1 * length(shuffledIndices));
validationDataTbl = trainingData(shuffledIndices(validationIdx),:);

testIdx = validationIdx(end)+1 : length(shuffledIndices);
testDataTbl = trainingData(shuffledIndices(testIdx),:);

% Use imageDatastore and boxLabelDatastore to create datastores for 
% loading the image and label data during training and evaluation.

imdsTrain = imageDatastore(trainingDataTbl{:,"imageFilename"});
bldsTrain = boxLabelDatastore(trainingDataTbl(:,"LEDMatrix"));

imdsValidation = imageDatastore(validationDataTbl{:,"imageFilename"});
bldsValidation = boxLabelDatastore(validationDataTbl(:,"LEDMatrix"));

imdsTest = imageDatastore(testDataTbl{:,"imageFilename"});
bldsTest = boxLabelDatastore(testDataTbl(:,"LEDMatrix"));

trainingLEDData = combine(imdsTrain,bldsTrain);
validationLEDData = combine(imdsValidation,bldsValidation);
testLEDData = combine(imdsTest,bldsTest);

%%
inputSize = [416 416 3];

trainingDataForEstimation = transform(trainingLEDData,@(data)preprocessData(data,inputSize));

numAnchors = 6; 
[anchors, meanIoU] = estimateAnchorBoxes(trainingDataForEstimation,numAnchors);
area = anchors(:,1).*anchors(:,2);
[~,idx] = sort(area,"descend");
anchors = anchors(idx,:);
anchorBoxes = {anchors(1:3,:);anchors(4:6,:)};

classes = {'LEDMatrix'};
numClasses = size(classes, 1);
numPredictorsPerAnchor = 5 + numClasses;

% Modify the layegraph to train with new set of classes.
lgraph = layerGraph(net);

yoloModule1 = convolution2dLayer(1,length(anchorBoxes{1})*numPredictorsPerAnchor,'Name','yoloconv1');
yoloModule2 = convolution2dLayer(1,length(anchorBoxes{2})*numPredictorsPerAnchor,'Name','yoloconv2');

lgraph = replaceLayer(lgraph,'conv_31',yoloModule1);
lgraph = replaceLayer(lgraph,'conv_38',yoloModule2);

dlnet = dlnetwork(lgraph);

detector = yolov4ObjectDetector(dlnet,classes,anchorBoxes,InputSize=inputSize)

%%

augmentedTrainingData = transform(trainingLEDData,@augmentData);

    options = trainingOptions("sgdm", ...
    InitialLearnRate=0.01,...
    MiniBatchSize=16,...
    MaxEpochs=10,...
    BatchNormalizationStatistics="moving",...
    ResetInputNormalization=false,...
    L2Regularization= 0.0005,...
    Shuffle="every-epoch",...
    ExecutionEnvironment="parallel",...
    VerboseFrequency=1,...
    ValidationData=validationLEDData,...
    ValidationFrequency=1,...
    Plots="training-progress");
    
trainedDetector = trainYOLOv4ObjectDetector(augmentedTrainingData,detector,options)

%%

detectionResults = detect(trainedDetector,testLEDData);

[ap,recall,precision] = evaluateDetectionPrecision(detectionResults,testLEDData);

figure
plot(recall,precision)
xlabel("Recall")
ylabel("Precision")
grid on
title(sprintf("Average Precision = %.2f",ap))

%%

%-------------------------------------------------------------------------------------------------%
                                %Supporting functions
%-------------------------------------------------------------------------------------------------%

function data = preprocessData(data,targetSize)
for num = 1:size(data,1)
    I = data{num,1};
    imgSize = size(I);
    bboxes = data{num,2};
    I = im2single(imresize(I,targetSize(1:2)));
    scale = targetSize(1:2)./imgSize(1:2);
    bboxes = bboxresize(bboxes,scale);
    data(num,1:2) = {I,bboxes};
end
end

%Helper function for performing data augmentation.
function data = augmentData(A)
% Apply random horizontal flipping, and random X/Y scaling. Boxes that get
% scaled outside the bounds are clipped if the overlap is above 0.25. Also,
% jitter image color.

data = cell(size(A));
for ii = 1:size(A,1)
    I = A{ii,1};
    bboxes = A{ii,2};
    labels = A{ii,3};
    sz = size(I);

    if numel(sz) == 3 && sz(3) == 3
        I = jitterColorHSV(I,...
            contrast=0.0,...
            Hue=0.1,...
            Saturation=0.2,...
            Brightness=0.2);
    end
    
    % Randomly flip image.
    tform = randomAffine2d(XReflection=true,Scale=[1 1.1]);
    rout = affineOutputView(sz,tform,BoundsStyle="centerOutput");
    I = imwarp(I,tform,OutputView=rout);
    
    % Apply same transform to boxes.
    [bboxes,indices] = bboxwarp(bboxes,tform,rout,OverlapThreshold=0.25);
    labels = labels(indices);
    
    % Return original data only when all boxes are removed by warping.
    if isempty(indices)
        data(ii,:) = A(ii,:);
    else
        data(ii,:) = {I,bboxes,labels};
    end
end
end

Answer 1

[Nihal Reddy]

I understand you are getting an error when executing "trainYOLOv4ObjectDetector()" function and want to know the reason behind it.

The error message- 'The network predicted invalid bounding boxes during training. Try reducing the learning rate' is triggered when either predicted width or predicted height is less than 1. This scenario is observed when the network is not training well especially due to high learning rate. In such cases, further training will not fetch good results hence "trainYOLOv4ObjectDetector()" function errors out and suggests to lower the learning rate.

Answer 2

[xinru]

I'm having the same issue when learning rate has been changed to 0.001.I've tryed to open “Parallel pools",now the program is running normally.