Azzera filtri
Azzera filtri

Why would the file size of a deep learning gradient become much bigger after saving as a .mat file?

1 visualizzazione (ultimi 30 giorni)
Hi,
I have a variable gradients which is the gradient of a deep learning model. From the code "whos gradients" you can see that it only requires 5742 bytes (i.e. 5.7 kB) to store. However, when I tried to save it as a .mat file, the file size becomes 13320098 bytes (i.e. 13.3 mB. Please refer to the code "file_size=file.bytes" ), which is more than 2000 times larger. May I ask the reason why, and how can I save the file with a size similar to 5.7kB?
Thanks!
My code:
%% Define Network Architecture
layers = [
imageInputLayer([1 1 100],'Normalization','none','Name','in')
transposedConv2dLayer([4 4],8*64,'Name','tconv1')
batchNormalizationLayer('Name','bn1')
reluLayer('Name','relu1')
transposedConv2dLayer([4 4],4*64,'Stride',2,'Cropping',1,'Name','tconv2')
batchNormalizationLayer('Name','bn2')
reluLayer('Name','relu2')
transposedConv2dLayer([4 4],2*64,'Stride',2,'Cropping',1,'Name','tconv3')
batchNormalizationLayer('Name','bn3')
reluLayer('Name','relu3')
transposedConv2dLayer([4 4],64,'Stride',2,'Cropping',1,'Name','tconv4')
batchNormalizationLayer('Name','bn4')
reluLayer('Name','relu4')
transposedConv2dLayer([4 4],1,'Stride',2,'Cropping',1,'Name','tconv5')
tanhLayer('Name','tanh')];
MyLGraph = layerGraph(layers);
myDLnet = dlnetwork(MyLGraph);
[dlZ, Y]=get_dlZ_Y();
gradients = dlfeval(@modelGradients, myDLnet, dlZ, Y);
whos gradients
save("gradients.mat","gradients");
file=dir("gradients.mat");
file_size=file.bytes
function [gradients] = modelGradients(myModel, modelInput, CorrectLabels)
CorrectLabels_transpose=transpose(CorrectLabels);
[modelOutput,state] = forward(myModel,modelInput);
modelOutput_mean=reshape(mean(mean(modelOutput)),1,100);
loss = -sum(sum(CorrectLabels_transpose.*log(sigmoid(modelOutput_mean/100))));
gradients = dlgradient(loss, myModel.Learnables);
end
function [dlZ, Y]=get_dlZ_Y()
rng(123); % seed
Z = randn(1,1,100,100,'single');
Y = randn(1,100,'single');
% Convert mini-batch of data to dlarray specify the dimension labels
% 'SSCB' (spatial, spatial, channel, batch).
dlZ = dlarray(Z, 'SSCB');
executionEnvironment="auto";
% If training on a GPU, then convert data to gpuArray.
if (executionEnvironment == "auto" && canUseGPU) || executionEnvironment == "gpu"
dlZ = gpuArray(dlZ);
end
end

Risposta accettata

Joss Knight
Joss Knight il 3 Dic 2019
The difference is that whos is unable to account for the fact that the data is all stored on the GPU, and is only showing CPU memory. Add the following
gradients = dlupdate(@gather, gradients);
and you will see parity between the numbers.
The answer is - you can't save 3.6 million 32-bit numbers in 5.7 kilobytes, no matter what magic you employ!

Più risposte (0)

Categorie

Scopri di più su Deep Learning Toolbox in Help Center e File Exchange

Prodotti


Release

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by