CNN multi image classification with 4 channel

Hi guys,

My cnn should receive 4 images that represent the features of the same image.

Each image represents the vertical, horizontal, oblique details and the low pass filtered image. So I started with the original image, then extracted the image details and saved them.

From the saved data I created 4 imagaDatastore and joined them with the combine function. So now I have to pass the datastore I create to my cnn, then I create the vector of the labels and add it to the database.

Now my problem arises: the cnn does not read the input and displays the Invalid training data error. For classification networks, responses must be categorical.

I have read a lot of documentation about it but I can't solve the problem. I did not understand how to input my combinedatastore to cnn

This is a part of my code:

I use the code for create categorical vector labels from https://it.mathworks.com/matlabcentral/answers/586949-multi-input-imagedatastore

%create the 4 imagedatastore that represent the feature of the image
LowPass_imds= imageDatastore('Low_Pass','IncludeSubfolders',true,'LabelSource','foldernames');
Vertical_imds= imageDatastore('Vertical','IncludeSubfolders',true,'LabelSource','foldernames');
Horizontal_imds= imageDatastore('Horizontal','IncludeSubfolders',true,'LabelSource','foldernames');
Diagonal_imds= imageDatastore('Diagonal','IncludeSubfolders',true,'LabelSource','foldernames');

%extract the label, in this example i use only one class
labels = LowPass_imds.Labels;
% %C = categorical for create categorical vector label from
writematrix(labels,'labelsOk.txt');
labelStore = tabularTextDatastore('labels.txt','TextscanFormats','%C',"ReadVariableNames",false);
labelStoreCell = transform(labelStore,@setcat_and_table_to_cell);

cds = combine(LowPass_imds,Vertical_imds,Horizontal_imds,Diagonal_imds, labelStoreCell);
cds.readall %for show that all is ok

%this is only an example, infact i use only one class
layers =[
    imageInputLayer([128 128 5]) 
    
    convolution2dLayer(3,8, 'Padding','same')
    batchNormalizationLayer 
    reluLayer 
    
    maxPooling2dLayer(2, 'Stride',2)
    
    convolution2dLayer(3,16, 'Padding','same')
    batchNormalizationLayer
    reluLayer
    
    maxPooling2dLayer(2, 'Stride',2)
    
    convolution2dLayer(3,32, 'Padding','same')
    batchNormalizationLayer
    
    maxPooling2dLayer(2, 'Stride',2)
    
    fullyConnectedLayer
    softmaxLayer
    classificationLayer];

options = trainingOptions('sgdm', ...
    'InitialLearnRate',0.01, ...
    'MaxEpochs',4, ...
    'Shuffle','every-epoch', ...
    'ValidationData',cds  , ...
    'ValidationFrequency',30, ...
    'Verbose',false, ...
    'Plots','training-progress');

net= trainNetwork(cds , layers, options); 

function [dataout] = setcat_and_table_to_cell(datain)
validcats = string(0:1); % define valid labels for categorical array
datain.(1) = setcats(datain.(1),validcats);
dataout = table2cell(datain);
end

Answer 1

[Shashank Gupta]

Hi,

I think the problem is in forming the datastore. I have one more way to combine the datastore. Please refer to following piece of code for the same.

imds1 = imageDatastore('imds1 location', ...); % First datastore
imds2 = imageDatastore('imds2 location', ...); % Second datastore
imds3 = imageDatastore('imds3 location', ...); % Third datastore

% Combine datastore.
imds_combine = imageDatastore({'imds1 location', 'imds2 location', 'imds3 location'},...
    IncludeSubfolders',true,'LabelSource');

The intent is to combine the location of all of the datastore and make sure rest of the argument remain same.

Now just try using this datastore to train the network.

net= trainNetwork(cds , layers, options); 

I hope this helps. Do reach out if you stuck anywhere. Although this should work.

Cheers.

[Nidhi Sawant]

@Shashank GuptaMatlab 2020b has released multi label classification example as below.

https://in.mathworks.com/help/deeplearning/ug/multilabel-text-classification-using-deep-learning.html

However when the task is to peform multi-label classification on huge image dataset, then there is a need to create image datastore.

The problem is we cannot create a imagedatastore with multi-label classification. Could you please enlighten on this? Thanks.