How to use transfer learning on U-net created in Matlab?
12 views (last 30 days)
Show older comments
Hridayi on 20 Feb 2020
Commented: sab kara on 9 Aug 2020
I'm interested in using a U-net that I have trained in Matlab on one set of data and then applying transfer learning on the last couple layers with another set of data (as shown in https://www.mathworks.com/help/deeplearning/gs/get-started-with-transfer-learning.html )
How would this process work with a network created using U-net layers in Matlab? Also, is the U-net layer in Matlab already pre-trained using some other set of data when it's used in the script?
I'm attempting to change the final convolutional layer using this code:
Net2 = SimNet1; %renaming U-Net
layers = Net2.Layers
layers(56) = convolution2dLayer([1 1], 2,'NumChannels',64); %Should I specify weights and bias?
% Directories of new training images (not used in pre-trained network)
segDir = fullfile('images location');
USDir = fullfile('images location'); % Labels
imds = imageDatastore(USDir); %DataStore of input training images - ultrasound images
classNames = ["bone","background"]; %labels
labelIDs = [1 0];
pxds = pixelLabelDatastore(segDir,classNames,labelIDs); %Has Ground Truth pixel data on training images - Seg images are pixel labeled (only have 255/0)
options = trainingOptions('adam','InitialLearnRate', 3e-4, ...
ds = pixelLabelImageDatastore(imds,pxds) %returns a datastore based on input image data(imds - US images)
%and pxds (required network output - segmentations)
TLNet1 = trainNetwork(ds,layers,options)
But I am recieving an error that says:
Shouldn't these layers already be connected from the pre-trained network?
Daniel Vieira on 21 Feb 2020
what you made works only for sequential models, where each layer is necessarily connected to the next in the same order that they are stacked in "Layers". The uNet is not a sequential model, some layers have multiple outputs, some have multiple inputs, a "early" layer may be connedted to a "late" layer. When you just pick the layers and pass it to trainNetwork, you are ignoring the connections.
Instead of passing the layers, you should pass the lgraph object. And before passing, you need to change it with the methods addLayers, removeLayers, connectLayers, disconnectLayers. You mustn't leave any layer input or output unassigned in the Connections property of lgraph.
Find more on Image Data Workflows in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!