Azzera filtri
Azzera filtri

How can I implement Layernormalization layer to generate C code?

5 visualizzazioni (ultimi 30 giorni)
I have a trained neural network with a layernormalization Layer and I want to generate the C code of it, but this layer has no support for code generation. Is there any way to implement it as a custom layer or is there another solution?
This is the architecture of the neural network:
layers = [ ...
featureInputLayer(4, "Name", "myFeatureInputLayer", 'Normalization','rescale-symmetric')
fullyConnectedLayer(16, "Name", "myFullyConnectedLayer1","WeightsInitializer","glorot")
tanhLayer("Name", "myTanhLayer")
fullyConnectedLayer(8, "Name", "myFullyConnectedLayer4","WeightsInitializer","he")
fullyConnectedLayer(2, "Name", "myFullyConnectedLayer6","WeightsInitializer","he")
This is the entry-point function to generate de code:
function out = DLfunction(in) %#codegen
% A persistent object mynet is used to load the series network object.
% At the first call to this function, the persistent object is constructed and
% setup. When the function is called subsequent times, the same object is reused
% to call predict on inputs, thus avoiding reconstructing and reloading the
% network object.
persistent mynet;
if isempty(mynet)
mynet = coder.loadDeepLearningNetwork('NNfinal.mat');
% pass in input
out = predict(mynet,in,'MiniBatchSize',4);
NNfinal.mat is the trained neural network
And this is the code I used it, following an example:
dlconfig = coder.DeepLearningConfig(TargetLibrary='none');
cfg = coder.config('lib');
cfg.DeepLearningConfig = dlconfig;
myInput=[4 3 4 4];
codegen -config cfg DLfunction -args {myInput} -report

Risposte (2)

Oguz Kaan Hancioglu
Oguz Kaan Hancioglu il 30 Mar 2023
Have you ever change the codegen target language?
cfg = coder.config('lib');
cfg.TargetLang = 'C';
cfg.DeepLearningConfig = coder.DeepLearningConfig(TargetLibrary = 'none');
  4 Commenti
Oguz Kaan Hancioglu
Oguz Kaan Hancioglu il 30 Mar 2023
Yes, you are right. normalization layer doesn't supported by the matlab coder. The help doesn't explain anything about the normalization layer. You need to build your architecture without normalization layer to generate C code.

Accedi per commentare.

Sergio Matiz Romero
Sergio Matiz Romero il 9 Mag 2023
Modificato: Sergio Matiz Romero il 9 Mag 2023
A potential workaround is to use groupNormalizationLayer in 'all-channels' mode instead to layerNormalizationLayer, since they are equivalent. You can find more information on the groupNormalizationLayer modes in the following link:
Below is an example that compares the code generation output of a network using groupNormalizationLayer in 'all-channels' mode with that of an equivalent network using layerNormalizationLayer. For the network using layerNormalizationLayer, we do not generate code (since it is not supported), but we run the entry point function in MATLAB instead. The output values match when comparing them.
cfg = coder.config('mex');
cfg.DeepLearningConfig = coder.DeepLearningConfig('none');
inputSize = [8 8 16];
% Network with layer normalization
layers = [imageInputLayer(inputSize, 'Name', 'input')
layerNormalizationLayer('Name', 'norm')
dlnet = dlnetwork(layers);
save('dlnetLayerNorm.mat', 'dlnet');
% Network with group normalization in mode 'all-channels'
layers = [imageInputLayer(inputSize, 'Name', 'input')
groupNormalizationLayer('all-channels', 'Name', 'norm')
dlnet = dlnetwork(layers);
save('dlnetGroupNorm.mat', 'dlnet');
in = dlarray(randi(255, inputSize, 'single'), 'SSCB');
codegen mPredict -config cfg -args {in, coder.Constant('dlnetGroupNorm.mat')}
outMat = mPredict(in, 'dlnetLayerNorm.mat');
outCode = mPredict_mex(in, 'dlnetGroupNorm.mat');
diff = max(abs(outMat(:) - outCode(:)));
The entry point function is
function out = mPredict(in, matfile)
net = coder.loadDeepLearningNetwork(coder.const(matfile));
out = net.predict(in);
Please let me know if this workaound unblocks your workflow or if you have additional questions


Scopri di più su Deep Learning Code Generation Fundamentals in Help Center e File Exchange




Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by