weight balancing in pretraining networks.

Dear Deep Learning Team,

Hello, i have a question about training process.

During i work on transfer learning (using pretrained model),

My data has not BALANCED, how to fixing or weight balancing "the gradient value" of mini-batch depends on ratio of my data?

for examples,

now, i have 3 class; A, B, and C // ratio of class is 5:5:1

i try to give weights to gradient

:: In "TrainNetwork" function > trainer.train > computeGradientsForTraning

:: in that function , i found "gradient" variable

> (end-1) cell has 2048x3 array

> (end / the last) cell has 3 value which maybe means average gradients

in that point, i want to give weight to gradient for 3rd class training,

most of "Ypred" predicted A or B, mooooooreeeeeeeee than C

What can i do in this situation?

Thanks for help

Answer 1

[Maria Duarte Rosa]

Once can define a custom weighted classfication layer:

https://www.mathworks.com/help/deeplearning/ug/create-custom-weighted-cross-entropy-classification-layer.html

Please see here for an example using this layer:

https://www.mathworks.com/help/deeplearning/examples/deep-learning-speech-recognition.html

[Raza Ali]

is it applicable to images as well? becasue i am trying weighted classfication layer for images but its giving error.

can you specify how to use it for image data (what shold be the class weights for 100 images with the size 256 x 256 by 3)