embeddingLayer

Embedding layer

Since R2026a

Description

An embedding layer converts numeric indices to numeric vectors, where the indices correspond to discrete data.

Use embeddings to map discrete data such as integer representations of categorical values or words to numeric vectors.

Creation

Syntax

layer = embeddingLayer(outputSize,maxIndex)

layer = embeddingLayer(outputSize,maxIndex,Name=Value)

Description

layer = embeddingLayer(outputSize,maxIndex) creates an embedding layer and sets the OutputSize and MaxIndex properties.

layer = embeddingLayer(outputSize,maxIndex,Name=Value) also specifies options using one or more name-value arguments. For example, Name="emb" sets the Name property to "emb".

Input Arguments

outputSize — Output size

positive integer

Output size, specified as a positive integer.

The layer maps discrete inputs to numeric vectors with length of the specified size.

This argument sets the OutputSize property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

maxIndex — Number of numeric vectors that the layer can map discrete inputs to

positive integer

Number of numeric vectors that the layer can map discrete inputs to, specified as a positive integer.

This argument sets the MaxIndex property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: embeddingLayer(outputSize,maxIndex,Name="emb") creates an embedding layer and sets the Name property to "emb".

MaskIndex — Mask index

[] (default) | nonnegative integer

Mask index, specified as one of these values:

• [] — The embedding does not contain a vector that corresponds to a mask value.

• nonnegative integer — The embedding vector with the specified index corresponds to a mask value. The mask index must be less than or equal to MaxIndex. If the mask index is 0, then ZeroIndexMapping must be "first-vector".

During training, the layer does not update the embedding vector that corresponds to the mask value.

This argument sets the MaskIndex property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

ZeroIndexMapping — Mapping of zero index values

"zeros" (default) | "first-vector" | "error"

Mapping of zero index values, specified as one of these values:

• "zeros" — Map zero-valued indices to vectors of zeros.

• "first-vector" — Map zero-valued indices to the first vector of the embedding weights.

• "error" — Throw an error when the layer receives zero-valued indices.

This argument sets the ZeroIndexMapping property.

InputLearnables — Learnable parameters from other layers

{} (default) | "weights"

Learnable parameters from other layers, specified as one of these values:

• "weights" — Receive weights from another layer.

• {} — Use the weights in the Weights property.

If InputLearnables specifies that the layer receives weights from another layer, then the layer has two inputs with the names 'in' and 'weights', which correspond to the output data and learnable weights, respectively. In this case, the corresponding properties that contain the initializer, learnable parameters, learning rate factor, and L₂ regularization factor must be [].

If the InputLearnables value is empty, then the layer has one input with the name 'in', which corresponds to the input data. In this case, the layer uses the Weights property for the layer operation.

For more information, see Neural Network Weight Tying.

This argument sets the InputLearnables property.

OutputLearnables — Learnable parameters to output to other layers

{} (default) | "weights"

Learnable parameters to output to other layers, specified as one of these values:

• "weights" — Output weights to another layer.

• {} — Do not output weights.

If OutputLearnables specifies that the layer outputs learnables to another layer, then the layer has two outputs with the names 'out' and 'weights', which correspond to the output data and learnable weights, respectively. In this case, the layer also outputs the weights that it uses in the layer operation.

If the OutputLearnables value is empty, then the layer has one output with the name 'out', which corresponds to the output data.

For more information, see Neural Network Weight Tying.

This argument sets the OutputLearnables property.

WeightsInitializer — Function to initialize weights

"narrow-normal" (default) | "glorot" | "he" | "orthogonal" | "zeros" | "ones" | function handle | []

Function to initialize the weights, specified as one these values:

• "narrow-normal" — Initialize the weights by independently sampling from a normal distribution with zero mean and standard deviation 0.01.

• "glorot" — Initialize the weights with the Glorot initializer [1] (also known as the Xavier initializer). The Glorot initializer independently samples from a normal distribution with zero mean and variance 2/(1 + OutputSize).

• "he" — Initialize the weights with the He initializer [2]. The He initializer samples from a normal distribution with zero mean and variance 2.

• "orthogonal" — Initialize the input weights with Q, the orthogonal matrix given by the QR decomposition of Z = QR for a random matrix Z sampled from a unit normal distribution. [3]

• "zeros" — Initialize the weights with zeros.

• "ones" — Initialize the weights with ones.

• Function handle — Initialize the weights with a custom function. If you specify a function handle, then the function syntax must be of the form weights = func(sz), where sz is the size of the weights. For an example, see Specify Custom Weight Initialization Function.

• [] — Do not initialize the weights. Use the weights from the 'weights' input instead.

The layer initializes the weights only when the Weights and InputLearnables properties are empty.

If InputLearnables is "weights", then the Weights, WeightsInitializer, WeightLearnRateFactor, and WeightL2Factor properties must be empty.

This argument sets the WeightsInitializer property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | char | string | function_handle

Weights — Embedding weights

[] (default) | matrix

Embedding weights, specified as an OutputSize-by-MaxIndex matrix.

The vector Weights(:,i) is the embedding vector for the input value i.

If InputLearnables specifies that the layer receives weights from another layer, then the layer has two inputs with the names 'in' and 'weights', which correspond to the output data and learnable weights, respectively. In this case, the corresponding properties that contain the initializer, learnable parameters, learning rate factor, and L₂ regularization factor must be [].

If the InputLearnables value is empty, then the layer has one input with the name 'in', which corresponds to the input data. In this case, the layer uses the Weights property for the layer operation.

If InputLearnables is "weights", then the Weights, WeightsInitializer, WeightLearnRateFactor, and WeightL2Factor properties must be empty.

This argument sets the Weights property.

Data Types: single | double

WeightLearnRateFactor — Learning rate factor for weights

1 (default) | nonnegative scalar | []

Learning rate factor for the weights, specified as one of these values:

• Nonnegative scalar — Use the specified learning rate factor.

• [] — Use learning rate factor from another layer.

The software multiplies this factor by the global learning rate to determine the learning rate for the weights in this layer. For example, if WeightLearnRateFactor is 2, then the learning rate for the weights in this layer is twice the current global learning rate. The software determines the global learning rate based on the settings you specify using the trainingOptions function.

If InputLearnables is "weights", then the Weights, WeightsInitializer, WeightLearnRateFactor, and WeightL2Factor properties must be empty.

This argument sets the WeightLearnRateFactor property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

WeightL2Factor — L₂ regularization factor for weights

1 (default) | nonnegative scalar | []

L₂ regularization factor for the weights, specified as one of these values:

• Nonnegative scalar — Use the specified regularization factor.

• [] — Use regularization factor from another layer.

The software multiplies this factor by the global L₂ regularization factor to determine the L₂ regularization for the weights in this layer. For example, if WeightL2Factor is 2, then the L₂ regularization for the weights in this layer is twice the global L₂ regularization factor. You can specify the global L₂ regularization factor using the trainingOptions function.

If InputLearnables is "weights", then the Weights, WeightsInitializer, WeightLearnRateFactor, and WeightL2Factor properties must be empty.

This argument sets the WeightL2Factor property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

Name — Layer name

"" (default) | character vector | string scalar

Layer name, specified as a character vector or a string scalar. For Layer array input, the trainnet and dlnetwork functions automatically assign names to unnamed layers.

This argument sets the Name property.

Data Types: char | string

Properties

Embedding

OutputSize — Output size

positive integer

This property is read-only after object creation. To set this property, use the corresponding name-value argument when you create the EmbeddingLayer object.

Output size, specified as a positive integer.

The layer maps discrete inputs to numeric vectors with length of the specified size.

Data Types: double

MaxIndex — Number of numeric vectors that the layer can map discrete inputs to

positive integer

This property is read-only after object creation. To set this property, use the corresponding name-value argument when you create the EmbeddingLayer object.

Number of numeric vectors that the layer can map discrete inputs to, specified as a positive integer.

Data Types: double

MaskIndex — Mask index

[] (default) | positive integer

This property is read-only after object creation. To set this property, use the corresponding name-value argument when you create the EmbeddingLayer object.

Mask index, specified as one of these values:

• [] — The embedding does not contain a vector that corresponds to a mask value.

• nonnegative integer — The embedding vector with the specified index corresponds to a mask value. The mask index must be less than or equal to MaxIndex. If the mask index is 0, then ZeroIndexMapping must be "first-vector".

During training, the layer does not update the embedding vector that corresponds to the mask value.

Data Types: double

ZeroIndexMapping — Mapping of zero index values

'zeros' (default) | 'first-vector''error'

This property is read-only after object creation. To set this property, use the corresponding name-value argument when you create the EmbeddingLayer object.

Mapping of zero index values, specified as one of these values:

• 'zeros' — Map zero-valued indices to vectors of zeros.

• 'first-vector' — Map zero-valued indices to the first vector of the embedding weights.

• 'error' — Throw an error when the layer receives zero-valued indices.

Weight Tying

InputLearnables — Learnable parameters from other layers

{} (default) | {'Weights'}

This property is read-only after object creation. To set this property, use the corresponding name-value argument when you create the EmbeddingLayer object.

Learnable parameters from other layers, specified as one of these values:

• {'Weights'} — Receive weights from another layer.

• {} — Use the weights in the Weights property.

If InputLearnables specifies that the layer receives weights from another layer, then the layer has two inputs with the names 'in' and 'weights', which correspond to the output data and learnable weights, respectively. In this case, the corresponding properties that contain the initializer, learnable parameters, learning rate factor, and L₂ regularization factor must be [].

If the InputLearnables value is empty, then the layer has one input with the name 'in', which corresponds to the input data. In this case, the layer uses the Weights property for the layer operation.

For more information, see Neural Network Weight Tying.

OutputLearnables — Learnable parameters to output to other layers

{} (default) | {'Weights'}

This property is read-only after object creation. To set this property, use the corresponding name-value argument when you create the EmbeddingLayer object.

Learnable parameters to output to other layers, specified as one of these values:

• {'Weights'} — Output weights to another layer.

• {} — Do not output weights.

If OutputLearnables specifies that the layer outputs learnables to another layer, then the layer has two outputs with the names 'out' and 'weights', which correspond to the output data and learnable weights, respectively. In this case, the layer also outputs the weights that it uses in the layer operation.

If the OutputLearnables value is empty, then the layer has one output with the name 'out', which corresponds to the output data.

For more information, see Neural Network Weight Tying.

Parameters and Initialization

WeightsInitializer — Function to initialize weights

'narrow-normal' (default) | 'glorot' | 'he' | 'orthogonal' | 'zeros' | 'ones' | function handle

Function to initialize the weights, specified as one these values:

• 'narrow-normal' — Initialize the weights by independently sampling from a normal distribution with zero mean and standard deviation 0.01.

• 'glorot' — Initialize the weights with the Glorot initializer [1] (also known as the Xavier initializer). The Glorot initializer independently samples from a normal distribution with zero mean and variance 2/(1 + OutputSize).

• 'he' — Initialize the weights with the He initializer [2]. The He initializer samples from a normal distribution with zero mean and variance 2.

• 'orthogonal' — Initialize the input weights with Q, the orthogonal matrix given by the QR decomposition of Z = QR for a random matrix Z sampled from a unit normal distribution. [3]

• 'zeros' — Initialize the weights with zeros.

• 'ones' — Initialize the weights with ones.

• Function handle — Initialize the weights with a custom function. If you specify a function handle, then the function syntax must be of the form weights = func(sz), where sz is the size of the weights. For an example, see Specify Custom Weight Initialization Function.

• [] — Do not initialize the weights. Use the weights from the 'weights' input instead.

The layer initializes the weights only when the Weights and InputLearnables properties are empty.

Data Types: double | char | function_handle

Weights — Embedding weights

[] (default) | numeric array

Embedding weights, specified as an OutputSize-by-MaxIndex matrix.

The vector Weights(:,i) is the embedding vector for the input value i.

If InputLearnables specifies that the layer receives weights from another layer, then the layer has two inputs with the names 'in' and 'weights', which correspond to the output data and learnable weights, respectively. In this case, the corresponding properties that contain the initializer, learnable parameters, learning rate factor, and L₂ regularization factor must be [].

If the InputLearnables value is empty, then the layer has one input with the name 'in', which corresponds to the input data. In this case, the layer uses the Weights property for the layer operation.

If InputLearnables is "weights", then the Weights, WeightsInitializer, WeightLearnRateFactor, and WeightL2Factor properties must be empty.

Data Types: single | double

Learning Rate and Regularization

WeightLearnRateFactor — Learning rate factor for weights

1 (default) | nonnegative scalar | [] (since R2026a)

Learning rate factor for the weights, specified as one of these values:

• Nonnegative scalar — Use the specified learning rate factor.

• [] (since R2026a) — Use learning rate factor from another layer.

The software multiplies this factor by the global learning rate to determine the learning rate for the weights in this layer. For example, if WeightLearnRateFactor is 2, then the learning rate for the weights in this layer is twice the current global learning rate. The software determines the global learning rate based on the settings you specify using the trainingOptions function.

If InputLearnables is "weights", then the Weights, WeightsInitializer, WeightLearnRateFactor, and WeightL2Factor properties must be empty.

Data Types: double

WeightL2Factor — L₂ regularization factor for weights

1 (default) | nonnegative scalar | []

L₂ regularization factor for the weights, specified as one of these values:

• Nonnegative scalar — Use the specified regularization factor.

• [] — Use regularization factor from another layer.

The software multiplies this factor by the global L₂ regularization factor to determine the L₂ regularization for the weights in this layer. For example, if WeightL2Factor is 2, then the L₂ regularization for the weights in this layer is twice the global L₂ regularization factor. You can specify the global L₂ regularization factor using the trainingOptions function.

If InputLearnables is "weights", then the Weights, WeightsInitializer, WeightLearnRateFactor, and WeightL2Factor properties must be empty.

Data Types: double

Layer

Name — Layer name

'' (default) | character vector

Layer name, specified as a character vector. For Layer array input, the trainnet and dlnetwork functions automatically assign names to unnamed layers.

Data Types: char

NumInputs — Number of inputs

Read-only: 1 | 2

This property is read-only.

Number of inputs to the layer.

If InputLearnables specifies that the layer receives weights from another layer, then the layer has two inputs with the names 'in' and 'weights', which correspond to the output data and learnable weights, respectively. In this case, the corresponding properties that contain the initializer, learnable parameters, learning rate factor, and L₂ regularization factor must be [].

If the InputLearnables value is empty, then the layer has one input with the name 'in', which corresponds to the input data. In this case, the layer uses the Weights property for the layer operation.

Data Types: double

InputNames — Input names

Read-only: {'in'} | {'in','weights'}

This property is read-only.

Input names of the layer.

If InputLearnables specifies that the layer receives weights from another layer, then the layer has two inputs with the names 'in' and 'weights', which correspond to the output data and learnable weights, respectively. In this case, the corresponding properties that contain the initializer, learnable parameters, learning rate factor, and L₂ regularization factor must be [].

If the InputLearnables value is empty, then the layer has one input with the name 'in', which corresponds to the input data. In this case, the layer uses the Weights property for the layer operation.

NumOutputs — Number of outputs

Read-only: 1 | 2

This property is read-only.

Number of outputs to the layer.

If OutputLearnables specifies that the layer outputs learnables to another layer, then the layer has two outputs with the names 'out' and 'weights', which correspond to the output data and learnable weights, respectively. In this case, the layer also outputs the weights that it uses in the layer operation.

If the OutputLearnables value is empty, then the layer has one output with the name 'out', which corresponds to the output data.

Data Types: double

OutputNames — Output names

Read-only: {'out'} | {'out','weights'}

This property is read-only.

Output names of the layer.

If OutputLearnables specifies that the layer outputs learnables to another layer, then the layer has two outputs with the names 'out' and 'weights', which correspond to the output data and learnable weights, respectively. In this case, the layer also outputs the weights that it uses in the layer operation.

If the OutputLearnables value is empty, then the layer has one output with the name 'out', which corresponds to the output data.

Examples

Create Embedding Layer

Create an embedding layer with an output size of 100 and a maximum index of 512.

layer = embeddingLayer(100,512)

layer = 
  EmbeddingLayer with properties:

                Name: ''
    ZeroIndexMapping: 'zeros'
           MaskIndex: []

   Hyperparameters
          OutputSize: 100
            MaxIndex: 512

   Learnable Parameters
             Weights: []

  Show all properties

Include an embedding layer in a layer array.

layers = [
    sequenceInputLayer(1)
    embeddingLayer(300,512)
    lstmLayer(100)
    fullyConnectedLayer(10)
    softmaxLayer];

Algorithms

Layer Input and Output Formats

Most layers in a layer array or layer graph pass data to subsequent layers as formatted dlarray objects. The format of a dlarray object is a string of characters in which each character describes the corresponding dimension of the data. The format consists of one or more of these characters:

• "S" — Spatial

• "C" — Channel

• "B" — Batch

• "T" — Time

• "U" — Unspecified

For example, you can describe 2-D image data that is represented as a 4-D array, where the first two dimensions correspond to the spatial dimensions of the images, the third dimension corresponds to the channels of the images, and the fourth dimension corresponds to the batch dimension, as having the format "SSCB" (spatial, spatial, channel, batch).

You can interact with these dlarray objects in automatic differentiation workflows, such as those for:

• developing a custom layer

• using a functionLayer object

• using the forward and predict functions with dlnetwork objects

EmbeddingLayer objects support data with any valid format. If the format of the input data contains a "C" (channel) dimension, then the format of the output data matches the format of the input data. If the format of the input data does not have a "C" (channel) dimension, then the format of the output data matches the format of the input data with an additional "C" (channel) dimension.

If InputLearnables specifies that the layer receives weights from another layer, then the corresponding input weights must be either an unformatted dlarray object, or a formatted dlarray object with a format that contains only "U" (unspecified) dimensions.

If OutputFormats specifies that the layer outputs weights to another layer, then the format of the output weights depends on the value of InputLearnables:

• If InputLearnables also specifies that the layer receives weights from another layer, then the weights output data is identical to the weights input data.

• If InputLearnables does not specify that the layer receives weights from another layer, then the weights output data is unformatted.

References

[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the Difficulty of Training Deep Feedforward Neural Networks." In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 249–356. Sardinia, Italy: AISTATS, 2010. https://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf

[2] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification." In 2015 IEEE International Conference on Computer Vision (ICCV), 1026–34. Santiago, Chile: IEEE, 2015. https://doi.org/10.1109/ICCV.2015.123

[3] Saxe, Andrew M., James L. McClelland, and Surya Ganguli. "Exact Solutions to the Nonlinear Dynamics of Learning in Deep Linear Neural Networks.” Preprint, submitted February 19, 2014. https://arxiv.org/abs/1312.6120.

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

• The InputLearnables and OutputLearnables properties must be empty.

GPU Code Generation

Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

Refer to the usage notes and limitations in the C/C++ Code Generation section. The same usage notes and limitations apply to GPU code generation.

Version History

Introduced in R2026a