eluLayer

Exponential linear unit (ELU) layer

Since R2019a

Description

An ELU activation layer performs the identity operation on positive inputs and an exponential nonlinearity on negative inputs.

The layer performs the following operation:

$$f\left(x\right)=\{\begin{array}{cc}x,& x\ge 0\\ \alpha \text{(exp(}x\text{) - 1)},& x<0\end{array}$$

The default value of α is 1. Specify a value of α for the layer by setting the Alpha property.

Creation

Syntax

layer = eluLayer

layer = eluLayer(alpha)

layer = eluLayer(___,'Name',Name)

Description

layer = eluLayer creates an ELU layer.

layer = eluLayer(alpha) creates an ELU layer and specifies the Alpha property.

layer = eluLayer(___,'Name',Name) additionally sets the optional Name property using any of the previous syntaxes. For example, eluLayer('Name','elu1') creates an ELU layer with the name 'elu1'.

Properties

ELU

Alpha — Nonlinearity parameter

1 (default) | numeric scalar

Nonlinearity parameter α, specified as a numeric scalar. The minimum value of the output of the ELU layer equals -α and the slope at negative inputs approaching 0 is α.

Layer

Name — Layer name

'' (default) | character vector | string scalar

Layer name, specified as a character vector or a string scalar. For Layer array input, the trainNetwork, assembleNetwork, layerGraph, and dlnetwork functions automatically assign names to layers with the name ''.

Data Types: char | string

NumInputs — Number of inputs

1 (default)

This property is read-only.

Number of inputs of the layer. This layer accepts a single input only.

Data Types: double

InputNames — Input names

{"in"} (default)

This property is read-only.

Input names of the layer. This layer accepts a single input only.

Data Types: cell

NumOutputs — Number of outputs

1 (default)

This property is read-only.

Number of outputs of the layer. This layer has a single output only.

Data Types: double

OutputNames — Output names

{'out'} (default)

This property is read-only.

Output names of the layer. This layer has a single output only.

Data Types: cell

Examples

Create ELU Layer

Create an exponential linear unit (ELU) layer with the name 'elu1' and a default value of 1 for the nonlinearity parameter Alpha.

layer = eluLayer('Name','elu1')

layer = 
  ELULayer with properties:

     Name: 'elu1'
    Alpha: 1

   Learnable Parameters
    No properties.

   State Parameters
    No properties.

  Show all properties

Include an ELU layer in a Layer array.

layers = [
    imageInputLayer([28 28 1])
    convolution2dLayer(3,16)
    batchNormalizationLayer
    eluLayer
    
    maxPooling2dLayer(2,'Stride',2)
    convolution2dLayer(3,32)
    batchNormalizationLayer
    eluLayer
    
    fullyConnectedLayer(10)
    softmaxLayer
    classificationLayer]

layers = 
  11x1 Layer array with layers:

     1   ''   Image Input             28x28x1 images with 'zerocenter' normalization
     2   ''   2-D Convolution         16 3x3 convolutions with stride [1  1] and padding [0  0  0  0]
     3   ''   Batch Normalization     Batch normalization
     4   ''   ELU                     ELU with Alpha 1
     5   ''   2-D Max Pooling         2x2 max pooling with stride [2  2] and padding [0  0  0  0]
     6   ''   2-D Convolution         32 3x3 convolutions with stride [1  1] and padding [0  0  0  0]
     7   ''   Batch Normalization     Batch normalization
     8   ''   ELU                     ELU with Alpha 1
     9   ''   Fully Connected         10 fully connected layer
    10   ''   Softmax                 softmax
    11   ''   Classification Output   crossentropyex

Algorithms

Layer Input and Output Formats

Layers in a layer array or layer graph pass data to subsequent layers as formatted dlarray objects. The format of a dlarray object is a string of characters, in which each character describes the corresponding dimension of the data. The formats consists of one or more of these characters:

• "S" — Spatial

• "C" — Channel

• "B" — Batch

• "T" — Time

• "U" — Unspecified

For example, 2-D image data represented as a 4-D array, where the first two dimensions correspond to the spatial dimensions of the images, the third dimension corresponds to the channels of the images, and the fourth dimension corresponds to the batch dimension, can be described as having the format "SSCB" (spatial, spatial, channel, batch).

ELULayer objects apply an element-wise operation and supports input data of any format. The layer and does not add or remove any dimensions, so it outputs data with the same format as its inputs.

References

[1] Clevert, Djork-Arné, Thomas Unterthiner, and Sepp Hochreiter. "Fast and accurate deep network learning by exponential linear units (ELUs)." arXiv preprint arXiv:1511.07289 (2015).

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.

GPU Code Generation
Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

Version History

Introduced in R2019a