Azzera filtri
Azzera filtri

How to modify my MATLAB code to train a neural network with images from different directions?

72 visualizzazioni (ultimi 30 giorni)
I am working on a project to estimate the weight of pears using their RGB and Depth images in MATLAB. Initially, I used only front-facing images for training, but now I want to improve the accuracy by including images taken from approximately 90 degrees to the left or right of the pear, in addition to the Depth images.
I tried modifying the code to accommodate these new images, but I'm facing some issues.
I would like to change the following points.
・The pear image uses the front, an image moved approximately 90 degrees to the left or right from the front, and a Depth image.
・I want to put all my data in a datastore.
Currently, the code listed is the code that I have modified, but in this state an error occurs and it is impossible to predict.Please lend me your strength.
cd 'RGB front file path'
folder_name = pwd;
XImageTrainFront = imageDatastore(folder_name);
cd 'RGB side file path'
folder_name = pwd;
XImageTrainSide = imageDatastore(folder_name);
cd 'Depth file path'
folder_name = pwd;
XDispTrain = imageDatastore(folder_name);
YTrain = [ data ]; % weight labels
YTrain = arrayDatastore(YTrain);
% Combine datasets
ds = combine(XImageTrainFront, XImageTrainSide, XDispTrain, YTrain);
dsrand = shuffle(ds);
dsVali = partition(dsrand, 10, 1);
dsTest = partition(dsrand, 10, 2);
ds1=partition(dsrand,10,3);
ds2=partition(dsrand,10,4);
ds3=partition(dsrand,10,5);
ds4=partition(dsrand,10,6);
ds5=partition(dsrand,10,7);
ds6=partition(dsrand,10,8);
ds7=partition(dsrand,10,9);
ds8=partition(dsrand,10,10);
dsTrain=combine(ds1,ds2,ds3,ds4,ds5,ds6,ds7,ds8,ReadOrder="sequential");
YTest = readall(dsTest);
YTest = YTest(:,4);
YTest = cell2mat(YTest);
cd 'path to googlenet300400_multiple.mat'
load googlenet300400_multiple.mat
miniBatchSize = 16;
options = trainingOptions('sgdm', ...
'MiniBatchSize', miniBatchSize, ...
'MaxEpochs', 300, ...
'InitialLearnRate', 1e-7, ...
'LearnRateSchedule', 'piecewise', ...
'LearnRateDropFactor', 0.1, ...
'LearnRateDropPeriod', 30, ...
'Shuffle', 'every-epoch', ...
'ValidationData', dsVali, ...
'ValidationFrequency', 50, ...
'Plots', 'training-progress', ...
'Verbose', true);
net = trainNetwork(dsTrain, lgraph_2, options);
YPredicted = predict(net, dsTest);
predictionError = YTest - YPredicted;
squares = predictionError.^2;
rmse = sqrt(mean(squares));
figure
scatter(YPredicted, YTest, '+');
xlabel("Predicted Value");
ylabel("True Value");
hold on;
plot([100 550], [100 550], 'r--');
Error message that occurred during modification.
Please respond to the following error.
Error using: trainNetwork (line 191)
Invalid training data. The output size of the last layer ([1 1 1]) is the response size ([1 1 360000])
does not match.
Error: newcord2 (line 93)
net = trainNetwork(dsTrain, lgraph_2, options);
Error using: nnet.cnn.LayerGraph/replaceLayer (line 300)
Layer 'finalLayerName' does not exist.
Error: newcord2 (line 85)
lgraph_2 = replaceLayer(lgraph_2, 'finalLayerName', finalLayer);
Edit after comment
Contents of googlenet300400_multiple.mat(lgraph_2)
Properties of <a href="matlab:helpPopup('nnet.cnn.LayerGraph')" style="font-weight:bold">LayerGraph</a>:
InputNames: {'imageinput' 'imageinput_1'}
OutputNames: {'regressionoutput'}
Layers: [284×1 nnet.cnn.layer.Layer]
Connections: [337×2 table]
We will also send you a test image.
  2 Commenti
Umar
Umar il 13 Ago 2024 alle 4:29

Hi @Masaya ,

Let me address your query regarding, “I tried modifying the code to accommodate these new images, but I'm facing some issues. I would like to change the following points.The pear image uses the front, an image moved approximately 90 degrees to the left or right from the front, and a Depth image.I want to put all my data in a datastore.Currently, the code listed is the code that I have modified, but in this state an error occurs and it is impossible to predict.Please lend me your strength.”

Please see my response to your comments below.

The error messages indicate two main problems in your code.

Error#1: The output size of the last layer in your neural network does not match the response size (the expected dimensions of your labels).

Error#2: It seems that you're trying to replace a layer that does not exist in your `LayerGraph`.

To resolve the output size mismatch, you need to make sure that the last layer of your neural network corresponds correctly to the shape of your target variable (`YTrain`). Check what shape your `YTrain` has after you create it:

% make sure this is a column vector if you expect a single output per sample
    YTrain = [data];
    YTrain = arrayDatastore(YTrain);

If `YTrain` is a matrix of dimensions `[N x 1]`, then the last layer of your network should also have an output size of `[1]` (for regression tasks). Also, if you're using a layer like `fullyConnectedLayer`, ensure it matches:

    finalLayer = fullyConnectedLayer(1, 'Name', 'finalLayerName');

Now, addressing issue about replacing layers, if you do not know what layers are present, use:

% Load or create your LayerGraph
lgraph_2 = layerGraph('your_model_name');
disp(lgraph_2.Layers); % Display all layers in the graph

If `finalLayerName` does not exist, replace it with the actual name from your `lgraph_2.Layers` list.

I did notice that your approach to combining datastores looks good, but make sure that all datastores have matching dimensions. When combining:

ds = combine(XImageTrainFront, XImageTrainSide, XDispTrain, YTrain);

Verify that each image datastore corresponds correctly to its labels.

Also, your training options seem well-defined; however, be cautious with learning rates and epochs. Sometimes starting with a higher learning rate can yield better convergence early on.

I will also make following suggestions, consider augmenting your data further to improve robustness (e.g., rotations, flips). Make sure that your validation dataset is representative of potential real-world scenarios to avoid overfitting. After predicting (`YPredicted`), consider using metrics like Mean Absolute Error (MAE) alongside RMSE for a comprehensive evaluation.

By making sure that your output layer matches the dimensions of your training labels and verifying that all layers exist as intended in your model graph, you should be able to resolve the errors you're facing. Adjusting these elements will improve your model's performance and allow for successful training with both front and side images as well as Depth data. If further issues arise, providing specific outputs or additional context will help diagnose any lingering problems effectively.

Matt J
Matt J il 13 Ago 2024 alle 4:49
Please attach things that would allow us to test the code, like lgraph_2 and a subset of dsTrain.

Accedi per commentare.

Risposte (0)

Prodotti


Release

R2024a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by