initialize

Initialize learnable and state parameters of neural network

Since R2021a

Syntax

netUpdated = initialize(net)

netUpdated = initialize(net,X1,...,XN)

Description

Tip

Most dlnetwork objects are initialized by default. You only need to manually initialize a dlnetwork if it is uninitialized. You can check if a network is initialized using the Initialized property of the dlnetwork object.

netUpdated = initialize(net) initializes any unset learnable parameters and state values of net based on the input sizes defined by the network input layers. Any learnable or state parameters that already contain values remain unchanged.

A network with unset, empty values for learnable and state parameters is uninitialized. You must initialize an uninitialized dlnetwork before you can use it. By default, dlnetwork objects are constructed with initial parameters and do not need initializing.

netUpdated = initialize(net,X1,...,XN) initializes any unset learnable parameters and state values of net based on the example network inputs or network data layout objects X1,...,XN. Use this syntax when the network has inputs that are not connected to an input layer.

Examples

Initialize dlnetwork Containing Input Layer

Define a simple image classification network as a layer array.

layers = [
    imageInputLayer([28 28 1],Normalization="none")
    convolution2dLayer(5,20)
    batchNormalizationLayer
    reluLayer
    fullyConnectedLayer(10)
    softmaxLayer];

Convert the layer graph to a dlnetwork object. Create an uninitialized dlnetwork object by setting the Initialize option to false.

net = dlnetwork(layers,Initialize=false);

View the learnable parameters of the network. Because the network is not initialized, the values are empty.

net.Learnables

ans=6×3 table
       Layer       Parameter       Value    
    ___________    _________    ____________

    "conv"         "Weights"    {0×0 double}
    "conv"         "Bias"       {0×0 double}
    "batchnorm"    "Offset"     {0×0 double}
    "batchnorm"    "Scale"      {0×0 double}
    "fc"           "Weights"    {0×0 double}
    "fc"           "Bias"       {0×0 double}

Initialize the learnable parameters of the network using the initialize function.

net = initialize(net);

View the learnable parameters of the network. Because the network is now initialized, the values are nonempty with sizes inferred using the size of the input layer.

net.Learnables

ans=6×3 table
       Layer       Parameter           Value       
    ___________    _________    ___________________

    "conv"         "Weights"    { 5×5×1×20 dlarray}
    "conv"         "Bias"       { 1×1×20   dlarray}
    "batchnorm"    "Offset"     { 1×1×20   dlarray}
    "batchnorm"    "Scale"      { 1×1×20   dlarray}
    "fc"           "Weights"    {10×11520  dlarray}
    "fc"           "Bias"       {10×1      dlarray}

Initialize dlnetwork Not Containing Input Layer

Define a multi-input image classification network.

numFilters = 24;

net = dlnetwork;

layersBranch1 = [
    convolution2dLayer(3,6*numFilters,Padding="same",Stride=2)
    groupNormalizationLayer("all-channels")
    reluLayer
    convolution2dLayer(3,numFilters,Padding="same")
    groupNormalizationLayer("channel-wise")
    additionLayer(2,Name="add")
    reluLayer
    fullyConnectedLayer(10)
    softmaxLayer];

layersBranch2 = [
    convolution2dLayer(1,numFilters,Name="conv_branch")
    groupNormalizationLayer("all-channels",Name="groupnorm_branch")];

net = addLayers(net, layersBranch1);
net = addLayers(net,layersBranch2);
net = connectLayers(net,"groupnorm_branch","add/in2"); 

Visualize the layers in a plot.

figure
plot(net)

View the learnable parameters of the network. Because the network is not initialized, the values are empty.

net.Learnables

ans=14×3 table
          Layer           Parameter       Value    
    __________________    _________    ____________

    "conv_1"              "Weights"    {0×0 double}
    "conv_1"              "Bias"       {0×0 double}
    "groupnorm_1"         "Offset"     {0×0 double}
    "groupnorm_1"         "Scale"      {0×0 double}
    "conv_2"              "Weights"    {0×0 double}
    "conv_2"              "Bias"       {0×0 double}
    "groupnorm_2"         "Offset"     {0×0 double}
    "groupnorm_2"         "Scale"      {0×0 double}
    "fc"                  "Weights"    {0×0 double}
    "fc"                  "Bias"       {0×0 double}
    "conv_branch"         "Weights"    {0×0 double}
    "conv_branch"         "Bias"       {0×0 double}
    "groupnorm_branch"    "Offset"     {0×0 double}
    "groupnorm_branch"    "Scale"      {0×0 double}

View the names of the network inputs.

net.InputNames

ans = 1×2 cell
    {'conv_1'}    {'conv_branch'}

Create random dlarray objects representing inputs to the network. Use an example input of size 64-by-64 with 3 channels for the main branch of the network. Use an input of size 64-by-64 with 18 channels for the second branch.

inputSize = [64 64 3];
inputSizeBranch = [32 32 18];

X1 = dlarray(rand(inputSize),"SSCB");
X2 = dlarray(rand(inputSizeBranch),"SSCB");

Initialize the learnable parameters of the network using the initialize function and specify the example inputs. Specify the inputs with order corresponding to the InputNames property of the network.

net = initialize(net,X1,X2);

View the learnable parameters of the network. Because the network is now initialized, the values are nonempty with sizes inferred using the size of the input data.

net.Learnables

ans=14×3 table
          Layer           Parameter            Value        
    __________________    _________    _____________________

    "conv_1"              "Weights"    { 3×3×3×144  dlarray}
    "conv_1"              "Bias"       { 1×1×144    dlarray}
    "groupnorm_1"         "Offset"     { 1×1×144    dlarray}
    "groupnorm_1"         "Scale"      { 1×1×144    dlarray}
    "conv_2"              "Weights"    { 3×3×144×24 dlarray}
    "conv_2"              "Bias"       { 1×1×24     dlarray}
    "groupnorm_2"         "Offset"     { 1×1×24     dlarray}
    "groupnorm_2"         "Scale"      { 1×1×24     dlarray}
    "conv_branch"         "Weights"    { 1×1×18×24  dlarray}
    "conv_branch"         "Bias"       { 1×1×24     dlarray}
    "groupnorm_branch"    "Offset"     { 1×1×24     dlarray}
    "groupnorm_branch"    "Scale"      { 1×1×24     dlarray}
    "fc"                  "Weights"    {10×24576    dlarray}
    "fc"                  "Bias"       {10×1        dlarray}

Initialize Network using Network Data Layout Objects

Create an uninitialized dlnetwork object that has two unconnected inputs.

layers = [
    convolution2dLayer(5,16,Name="conv")
    batchNormalizationLayer
    reluLayer
    fullyConnectedLayer(50)
    flattenLayer
    concatenationLayer(1,2,Name="cat")
    fullyConnectedLayer(10)
    softmaxLayer];

net = dlnetwork(layers,Initialize=false);

View the input names of the network.

net.InputNames

ans = 1×2 cell
    {'conv'}    {'cat/in2'}

Create network data layout objects that represent input data for the inputs. For the first input, specify a batch of 28-by-28 grayscale images. For the second input specify a batch of single-channel feature data.

layout1 = networkDataLayout([28 28 1 NaN],"SSCB");
layout2 = networkDataLayout([1 NaN],"CB");

Initialize the network using the network data layout objects.

net = initialize(net,layout1,layout2)

net = 
  dlnetwork with properties:

         Layers: [8×1 nnet.cnn.layer.Layer]
    Connections: [7×2 table]
     Learnables: [8×3 table]
          State: [2×3 table]
     InputNames: {'conv'  'cat/in2'}
    OutputNames: {'softmax'}
    Initialized: 1

  View summary with summary.

Input Arguments

net — Uninitialized network

dlnetwork object

Uninitialized network, specified as a dlnetwork object.

X1,...,XN — Example network inputs or data layouts

formatted dlarray objects | formatted networkDataLayout objects | unformatted dlarray objects (since R2025a) | unformatted networkDataLayout objects (since R2025a)

Example network inputs or data layouts to use to determine the size and formats of learnable and state parameters, each specified as one of these values:

• Formatted dlarray object

• Formatted networkDataLayout object

• Unformatted dlarray object (since R2025a)

• Unformatted networkDataLayout object (since R2025a)

The software propagates X1,...XN through the network to determine the appropriate sizes and formats of the learnable and state parameters of the dlnetwork object and initializes any unset learnable or state parameters.

To create a neural network that receives unformatted data, use an inputLayer object and do not specify a format. (since R2025a)

Before R2025a: X1,...XN must be formatted dlarray or networkDataLayout objects.

Provide example inputs in the same order as the order specified by the InputNames property of the input network.

Note

Automatic initialization uses only the size and format information of the input data. For initialization that depends on the values on the input data, you must initialize the learnable parameters manually.

Output Arguments

netUpdated — Initialized network

dlnetwork object

Initialized network, returned as an initialized dlnetwork object.

The initialize function does not preserve quantization information. If the input network is a quantized network, then the output network does not contain quantization information.

Version History

Introduced in R2021a

R2025a: Initialize networks with unformatted data

Initialize neural networks with unformatted data by specifying the inputs X1,...,XN as unformatted dlarray or unformatted networkDataLayout objects.

R2023b: Initialize networks containing input layers with unset normalization statistics

Input layers such as imageInputLayer and sequenceInputLayer contain properties that networks use for data normalization. These properties are Mean, StandardDeviation, Min, and Max. The software uses these properties to apply the data normalization method defined by the Normalization property of the layer.

Starting in R2023b, when you initialize a network by creating an initialized dlnetwork or by using the initialize function, the software initializes the Mean, StandardDeviation, Min, and Max properties of input layers if you do not set them when you create the layer and if the normalization method requires them. For normalization methods that use two properties, for example, zscore, the software initializes those properties only if you do not set either property when you create the layer.

• For zerocenter normalization, Mean is initialized to 0.

• For zscore normalization, Mean is initialized to 0 and StandardDeviation is initialized to 1.

• For rescale-symmetric normalization, Min is initialized to -1 and Max is initialized to 1.

• For rescale-zero-one normalization, Min is initialized to 0 and Max is initialized to 1.

By default, the software automatically calculates the normalization statistics during training. To customize the normalization, set the Mean, StandardDeviation, Min, and Max properties of input layers manually.

In previous releases, the software errors when you initialize a network containing an input layer that uses a normalization method requiring properties that you do not specify when you create the layer.