Check Custom Layer Validity

If you create a custom deep learning layer, then you can use the checkLayer function to check that the layer is valid. The function checks layers for validity, GPU compatibility, correctly defined gradients, and code generation compatibility. To check that a layer is valid, run the following command:

checkLayer(layer,validInputSize)

layer is an instance of the layer and validInputSize is a vector or cell array specifying the valid input sizes to the layer. To check with multiple observations, use the ObservationDimension option. To run the check for code generation compatibility, set the CheckCodegenCompatibility option to 1 (true). For large input sizes, the gradient checks take longer to run. To speed up the check, specify a smaller valid input size.

Check Custom Layer Validity

Check the validity of the example custom layer preluLayer.

The custom layer preluLayer, attached to this example as a supporting file, applies the PReLU operation to the input data. To access this layer, open this example as a live script.

Create an instance of the layer.

layer = preluLayer;

Because the layer has a custom initialize function, initialize the layer using a networkDataFormat object that specifies the expected input size and format of a single observation of typical input to the layer.

Specify a valid input size of [24 24 20], where the dimensions correspond to the height, width, and number of channels of the previous layer output.

validInputSize = [24 24 20];
layout = networkDataLayout(validInputSize,"SSC");
layer = initialize(layer,layout);

Check the layer validity using checkLayer. Specify the valid input size as the size as the size as used to initialize the layer. When you pass data through the network, the layer expects 4-D array inputs, where the first three dimensions correspond to the height, width, and number of channels of the previous layer output, and the fourth dimension corresponds to the observations.

checkLayer(layer,validInputSize)

Skipping multi-observation tests. To enable tests with multiple observations, specify the 'ObservationDimension' option.
For 2-D image data, set 'ObservationDimension' to 4.
For 3-D image data, set 'ObservationDimension' to 5.
For sequence data, set 'ObservationDimension' to 2.
 
Skipping GPU tests. No compatible GPU device found.
 
Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the 'CheckCodegenCompatibility' and 'ObservationDimension' options.
 
Running nnet.checklayer.TestLayerWithoutBackward
.......... ..
Done nnet.checklayer.TestLayerWithoutBackward
__________

Test Summary:
	 12 Passed, 0 Failed, 0 Incomplete, 16 Skipped.
	 Time elapsed: 0.054851 seconds.

The results show the number of passed, failed, and skipped tests. If you do not specify the ObservationsDimension option, or do not have a GPU, then the function skips the corresponding tests.

Check Multiple Observations

For multi-observation image input, the layer expects an array of observations of size h-by-w-by-c-by-N, where h, w, and c are the height, width, and number of channels, respectively, and N is the number of observations.

To check the layer validity for multiple observations, specify the typical size of an observation and set the ObservationDimension option to 4.

checkLayer(layer,validInputSize,ObservationDimension=4)

Skipping GPU tests. No compatible GPU device found.
 
Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the 'CheckCodegenCompatibility' and 'ObservationDimension' options.
 
Running nnet.checklayer.TestLayerWithoutBackward
.......... ........
Done nnet.checklayer.TestLayerWithoutBackward
__________

Test Summary:
	 18 Passed, 0 Failed, 0 Incomplete, 10 Skipped.
	 Time elapsed: 0.030498 seconds.

In this case, the function does not detect any issues with the layer.

List of Tests

The checkLayer function checks the validity of a custom layer by performing a series of tests.

Intermediate Layers

The checkLayer function uses these tests to check the validity of custom intermediate layers (layers of type nnet.layer.Layer).

Test	Description
functionSyntaxesAreCorrect	The syntaxes of the layer functions are correctly defined.
predictDoesNotError	predict function does not error.
forwardDoesNotError	When specified, the forward function does not error.
forwardPredictAreConsistentInSize	When forward is specified, forward and predict output values of the same size.
backwardDoesNotError	When specified, backward does not error.
backwardIsConsistentInSize	When backward is specified, the outputs of backward are consistent in size: The derivatives with respect to each input are the same size as the corresponding input. The derivatives with respect to each learnable parameter are the same size as the corresponding learnable parameter.
predictIsConsistentInType	The outputs of predict are consistent in type with the inputs.
forwardIsConsistentInType	When forward is specified, the outputs of forward are consistent in type with the inputs.
backwardIsConsistentInType	When backward is specified, the outputs of backward are consistent in type with the inputs.
gradientsAreNumericallyCorrect	When backward is specified, the gradients computed in backward are consistent with the numerical gradients.
backwardPropagationDoesNotError	When backward is not specified, the derivatives can be computed using automatic differentiation.
predictReturnsValidStates	For layers with state properties, the predict function returns valid states.
forwardReturnsValidStates	For layers with state properties, the forward function, if specified, returns valid states.
resetStateDoesNotError	For layers with state properties, the resetState function, if specified, does not error and resets the states to valid states.
codegenPragmaDefinedInClassDef	The pragma "%#codegen" for code generation is specified in class file.
layerPropertiesSupportCodegen	The layer properties support code generation.
predictSupportsCodegen	predict is valid for code generation.
doesNotHaveStateProperties	For code generation, the layer does not have state properties.
functionLayerSupportsCodegen	For code generation, the layer function must be a named function on the path and the Formattable property must be 0 (false).

Some tests run multiple times. These tests also check different data types and for GPU compatibility:

• predictIsConsistentInType

• forwardIsConsistentInType

• backwardIsConsistentInType

To execute the layer functions on a GPU, the functions must support inputs and outputs of type gpuArray with the underlying data type single.

Output Layers

The checkLayer function uses these tests to check the validity of custom output layers (layers of type nnet.layer.ClassificationLayer or nnet.layer.RegressionLayer).

Test	Description
forwardLossDoesNotError	forwardLoss does not error.
backwardLossDoesNotError	backwardLoss does not error.
forwardLossIsScalar	The output of forwardLoss is scalar.
backwardLossIsConsistentInSize	When backwardLoss is specified, the output of backwardLoss is consistent in size: dLdY is the same size as the predictions Y.
forwardLossIsConsistentInType	The output of forwardLoss is consistent in type: loss is the same type as the predictions Y.
backwardLossIsConsistentInType	When backwardLoss is specified, the output of backwardLoss is consistent in type: dLdY must be the same type as the predictions Y.
gradientsAreNumericallyCorrect	When backwardLoss is specified, the gradients computed in backwardLoss are numerically correct.
backwardPropagationDoesNotError	When backwardLoss is not specified, the derivatives can be computed using automatic differentiation.

The forwardLossIsConsistentInType and backwardLossIsConsistentInType tests also check for GPU compatibility. To execute the layer functions on a GPU, the functions must support inputs and outputs of type gpuArray with the underlying data type single.

Generated Data

To check the layer validity, the checkLayer function generates data depending on the type of layer:

Layer Type	Description of Generated Data
Intermediate	Values in the range [-1,1]
Regression output	Predictions and targets with values in the range [-1,1]
Classification output	Predictions with values in the range [0,1]. If you specify the ObservationDimension option, then the targets are one-hot encoded vectors (vectors containing a single 1, and 0 elsewhere). If you do not specify the ObservationDimension option, then the targets are values in the range [0,1].

To check for multiple observations, specify the observation dimension using the ObservationDimension option. If you specify the observation dimension, then the checkLayer function checks that the layer functions are valid using generated data with mini-batches of size 1 and 2. If you do not specify this name-value pair, then the function skips the tests that check that the layer functions are valid for multiple observations.

Diagnostics

If a test fails when you use checkLayer, then the function provides a test diagnostic and a framework diagnostic. The test diagnostic highlights any issues found with the layer. The framework diagnostic provides more detailed information.

Function Syntaxes

The test functionSyntaxesAreCorrect checks that the layer functions have correctly defined syntaxes.

Test Diagnostic	Description	Possible Solution
Incorrect number of input arguments for 'predict' in Layer.	The syntax for the predict function is not consistent with the number of layer inputs.	Specify the correct number of input and output arguments in predict. The predict function syntax depends on the type of layer. Z = predict(layer,X) forwards the input data X through the layer and outputs the result Z, where layer has a single input and a single output. [Z,state] = predict(layer,X) also outputs the updated state parameter state, where layer has a single state parameter. You can adjust the syntaxes for layers with multiple inputs, multiple outputs, or multiple state parameters: For layers with multiple inputs, replace X with X1,...,XN, where N is the number of inputs. The NumInputs property must match N. For layers with multiple outputs, replace Z with Z1,...,ZM, where M is the number of outputs. The NumOutputs property must match M. For layers with multiple state parameters, replace state with state1,...,stateK, where K is the number of state parameters. Tip If the number of inputs to the layer can vary, then use varargin instead of X1,…,XN. In this case, varargin is a cell array of the inputs, where varargin{i} corresponds to Xi. If the number of outputs can vary, then use varargout instead of Z1,…,ZN. In this case, varargout is a cell array of the outputs, where varargout{j} corresponds to Zj. Tip If the custom layer has a dlnetwork object for a learnable parameter, then in the predict function of the custom layer, use the predict function for the dlnetwork. When you do so, the dlnetwork object predict function uses the appropriate layer operations for prediction. If the dlnetwork has state parameters, then also return the network state.
Incorrect number of output arguments for 'predict' in Layer	The syntax for the predict function is not consistent with the number of layer outputs.
Incorrect number of input arguments for 'forward' in Layer	The syntax for the optional forward function is not consistent with the number of layer inputs.	Specify the correct number of input and output arguments in forward. The forward function syntax depends on the type of layer: Z = forward(layer,X) forwards the input data X through the layer and outputs the result Z, where layer has a single input and a single output. [Z,state] = forward(layer,X) also outputs the updated state parameter state, where layer has a single state parameter. [__,memory] = forward(layer,X) also returns a memory value for a custom backward function using any of the previous syntaxes. If the layer has both a custom forward function and a custom backward function, then the forward function must return a memory value. You can adjust the syntaxes for layers with multiple inputs, multiple outputs, or multiple state parameters: For layers with multiple inputs, replace X with X1,...,XN, where N is the number of inputs. The NumInputs property must match N. For layers with multiple outputs, replace Z with Z1,...,ZM, where M is the number of outputs. The NumOutputs property must match M. For layers with multiple state parameters, replace state with state1,...,stateK, where K is the number of state parameters. Tip If the number of inputs to the layer can vary, then use varargin instead of X1,…,XN. In this case, varargin is a cell array of the inputs, where varargin{i} corresponds to Xi. If the number of outputs can vary, then use varargout instead of Z1,…,ZN. In this case, varargout is a cell array of the outputs, where varargout{j} corresponds to Zj. Tip If the custom layer has a dlnetwork object for a learnable parameter, then in the forward function of the custom layer, use the forward function of the dlnetwork object. When you do so, the dlnetwork object forward function uses the appropriate layer operations for training.
Incorrect number of output arguments for 'forward' in Layer	The syntax for the optional forward function is not consistent with the number of layer outputs.
Incorrect number of input arguments for 'backward' in Layer	The syntax for the optional backward function is not consistent with the number of layer inputs and outputs.	Specify the correct number of input and output arguments in backward. The backward function syntax depends on the type of layer. dLdX = backward(layer,X,Z,dLdZ,memory) returns the derivatives dLdX of the loss with respect to the layer input, where layer has a single input and a single output. Z corresponds to the forward function output and dLdZ corresponds to the derivative of the loss with respect to Z. The function input memory corresponds to the memory output of the forward function. [dLdX,dLdW] = backward(layer,X,Z,dLdZ,memory) also returns the derivative dLdW of the loss with respect to the learnable parameter, where layer has a single learnable parameter. [dLdX,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory) also returns the derivative dLdSin of the loss with respect to the state input, where layer has a single state parameter and dLdSout corresponds to the derivative of the loss with respect to the layer state output. [dLdX,dLdW,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory) also returns the derivative dLdW of the loss with respect to the learnable parameter and returns the derivative dLdSin of the loss with respect to the layer state input, where layer has a single state parameter and single learnable parameter. You can adjust the syntaxes for layers with multiple inputs, multiple outputs, multiple learnable parameters, or multiple state parameters: For layers with multiple inputs, replace X and dLdX with X1,...,XN and dLdX1,...,dLdXN, respectively, where N is the number of inputs. For layers with multiple outputs, replace Z and dLdZ with Z1,...,ZM and dLdZ1,...,dLdZM, respectively, where M is the number of outputs. For layers with multiple learnable parameters, replace dLdW with dLdW1,...,dLdWP, where P is the number of learnable parameters. For layers with multiple state parameters, replace dLdSin and dLdSout with dLdSin1,...,dLdSinK and dLdSout1,...,dLdSoutK, respectively, where K is the number of state parameters. To reduce memory usage by preventing unused variables being saved between the forward and backward pass, replace the corresponding input arguments with ~. Tip If the number of inputs to backward can vary, then use varargin instead of the input arguments after layer. In this case, varargin is a cell array of the inputs, where the first N elements correspond to the N layer inputs, the next M elements correspond to the M layer outputs, the next M elements correspond to the derivatives of the loss with respect to the M layer outputs, the next K elements correspond to the K derivatives of the loss with respect to the K state outputs, and the last element corresponds to memory. If the number of outputs can vary, then use varargout instead of the output arguments. In this case, varargout is a cell array of the outputs, where the first N elements correspond to the N the derivatives of the loss with respect to the N layer inputs, the next P elements correspond to the derivatives of the loss with respect to the P learnable parameters, and the next K elements correspond to the derivatives of the loss with respect to the K state inputs. Tip If the layer forward functions support dlarray objects, then the software automatically determines the backward function and you do not need to specify the backward function. For a list of functions that support dlarray objects, see List of Functions with dlarray Support.
Incorrect number of output arguments for 'backward' in Layer	The syntax for the optional backward function is not consistent with the number of layer outputs.

For layers with multiple inputs or outputs, you must set the values of the layer properties NumInputs (or alternatively, InputNames) and NumOutputs (or alternatively, OutputNames) in the layer constructor function, respectively.

Multiple Observations

The checkLayer function checks that the layer functions are valid for single and multiple observations. To check for multiple observations, specify the observation dimension using the ObservationDimension option. If you specify the observation dimension, then the checkLayer function checks that the layer functions are valid using generated data with mini-batches of size 1 and 2. If you do not specify this name-value pair, then the function skips the tests that check that the layer functions are valid for multiple observations.

Test Diagnostic	Description	Possible Solution
Skipping multi-observation tests. To enable checks with multiple observations, specify the 'ObservationDimension' parameter in checkLayer.	If you do not specify the 'ObservationDimension' parameter in checkLayer, then the function skips the tests that check data with multiple observations.	Use the command checkLayer(layer,validInputSize,'ObservationDimension',dim), where layer is an instance of the custom layer, validInputSize is a vector specifying the valid input size to the layer, and dim specifies the dimension of the observations in the layer input. For more information, see Layer Input Sizes.

Functions Do Not Error

These tests check that the layers do not error when passed input data of valid size.

Intermediate Layers.  The tests predictDoesNotError, forwardDoesNotError, and backwardDoesNotError check that the layer functions do not error when passed inputs of valid size. If you specify an observation dimension, then the function checks the layer for both a single observation and multiple observations.

Test Diagnostic	Description	Possible Solution
The function 'predict' threw an error:	The predict function errors when passed data of size validInputSize.	Address the error described in the Framework Diagnostic section. Tip If the layer forward functions support dlarray objects, then the software automatically determines the backward function and you do not need to specify the backward function. For a list of functions that support dlarray objects, see List of Functions with dlarray Support.
The function 'forward' threw an error:	The optional forward function errors when passed data of size validInputSize.
The function 'backward' threw an error:	The optional backward function errors when passed the output of predict.

Output Layers.  The tests forwardLossDoesNotError and backwardLossDoesNotError check that the layer functions do not error when passed inputs of valid size. If you specify an observation dimension, then the function checks the layer for both a single observation and multiple observations.

Test Diagnostic	Description	Possible Solution
The function 'forwardLoss' threw an error:	The forwardLoss function errors when passed data of size validInputSize.	Address the error described in the Framework Diagnostic section. Tip If the forwardLoss function supports dlarray objects, then the software automatically determines the backward loss function and you do not need to specify the backwardLoss function. For a list of functions that support dlarray objects, see List of Functions with dlarray Support.
The function 'backwardLoss' threw an error:	The optional backwardLoss function errors when passed data of size validInputSize.

Outputs Are Consistent in Size

These tests check that the layer function outputs are consistent in size.

Intermediate Layers.  The test backwardIsConsistentInSize checks that the backward function outputs derivatives of the correct size.

The backward function syntax depends on the type of layer.

• dLdX = backward(layer,X,Z,dLdZ,memory) returns the derivatives dLdX of the loss with respect to the layer input, where layer has a single input and a single output. Z corresponds to the forward function output and dLdZ corresponds to the derivative of the loss with respect to Z. The function input memory corresponds to the memory output of the forward function.

• [dLdX,dLdW] = backward(layer,X,Z,dLdZ,memory) also returns the derivative dLdW of the loss with respect to the learnable parameter, where layer has a single learnable parameter.

• [dLdX,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory) also returns the derivative dLdSin of the loss with respect to the state input, where layer has a single state parameter and dLdSout corresponds to the derivative of the loss with respect to the layer state output.

• [dLdX,dLdW,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory) also returns the derivative dLdW of the loss with respect to the learnable parameter and returns the derivative dLdSin of the loss with respect to the layer state input, where layer has a single state parameter and single learnable parameter.

You can adjust the syntaxes for layers with multiple inputs, multiple outputs, multiple learnable parameters, or multiple state parameters:

• For layers with multiple inputs, replace X and dLdX with X1,...,XN and dLdX1,...,dLdXN, respectively, where N is the number of inputs.

• For layers with multiple outputs, replace Z and dLdZ with Z1,...,ZM and dLdZ1,...,dLdZM, respectively, where M is the number of outputs.

• For layers with multiple learnable parameters, replace dLdW with dLdW1,...,dLdWP, where P is the number of learnable parameters.

• For layers with multiple state parameters, replace dLdSin and dLdSout with dLdSin1,...,dLdSinK and dLdSout1,...,dLdSoutK, respectively, where K is the number of state parameters.

To reduce memory usage by preventing unused variables being saved between the forward and backward pass, replace the corresponding input arguments with ~.

Tip

If the number of inputs to backward can vary, then use varargin instead of the input arguments after layer. In this case, varargin is a cell array of the inputs, where the first N elements correspond to the N layer inputs, the next M elements correspond to the M layer outputs, the next M elements correspond to the derivatives of the loss with respect to the M layer outputs, the next K elements correspond to the K derivatives of the loss with respect to the K state outputs, and the last element corresponds to memory.

If the number of outputs can vary, then use varargout instead of the output arguments. In this case, varargout is a cell array of the outputs, where the first N elements correspond to the N the derivatives of the loss with respect to the N layer inputs, the next P elements correspond to the derivatives of the loss with respect to the P learnable parameters, and the next K elements correspond to the derivatives of the loss with respect to the K state inputs.

The derivatives dLdX1, …, dLdXn must be the same size as the corresponding layer inputs, and dLdW1,…,dLdWk must be the same size as the corresponding learnable parameters. The sizes must be consistent for input data with single and multiple observations.

Test Diagnostic	Description	Possible Solution
Incorrect size of 'dLdX' for 'backward'.	The derivatives of the loss with respect to the layer inputs must be the same size as the corresponding layer input.	Return the derivatives dLdX1,…,dLdXn with the same size as the corresponding layer inputs X1,…,Xn.
Incorrect size of the derivative of the loss with respect to the input 'in1' for 'backward'
The size of 'Z' returned from 'forward' must be the same as for 'predict'.	The outputs of predict must be the same size as the corresponding outputs of forward.	Return the outputs Z1,…,Zm of predict with the same size as the corresponding outputs Z1,…,Zm of forward.
Incorrect size of the derivative of the loss with respect to 'W' for 'backward'.	The derivatives of the loss with respect to the learnable parameters must be the same size as the corresponding learnable parameters.	Return the derivatives dLdW1,…,dLdWk with the same size as the corresponding learnable parameters W1,…,Wk.

Tip

If the layer forward functions support dlarray objects, then the software automatically determines the backward function and you do not need to specify the backward function. For a list of functions that support dlarray objects, see List of Functions with dlarray Support.

Output Layers.  The test forwardLossIsScalar checks that the output of the forwardLoss function is scalar. When the backwardLoss function is specified, the test backwardLossIsConsistentInSize checks that the outputs of forwardLoss and backwardLoss are of the correct size.

The syntax for forwardLoss is loss = forwardLoss(layer,Y,T). The input Y corresponds to the predictions made by the network. These predictions are the output of the previous layer. The input T corresponds to the training targets. The output loss is the loss between Y and T according to the specified loss function. The output loss must be scalar.

If the forwardLoss function supports dlarray objects, then the software automatically determines the backward loss function and you do not need to specify the backwardLoss function. For a list of functions that support dlarray objects, see List of Functions with dlarray Support.

The syntax for backwardLoss is dLdY = backwardLoss(layer,Y,T). The input Y contains the predictions made by the network and T contains the training targets. The output dLdY is the derivative of the loss with respect to the predictions Y. The output dLdY must be the same size as the layer input Y.

Test Diagnostic	Description	Possible Solution
Incorrect size of 'loss' for 'forwardLoss'.	The output loss of forwardLoss must be a scalar.	Return the output loss as a scalar. For example, if you have multiple values of the loss, then you can use mean or sum.
Incorrect size of the derivative of loss 'dLdY' for 'backwardLoss'.	When backwardLoss is specified, the derivatives of the loss with respect to the layer input must be the same size as the layer input.	Return derivative dLdY with the same size as the layer input Y. If the forwardLoss function supports dlarray objects, then the software automatically determines the backward loss function and you do not need to specify the backwardLoss function. For a list of functions that support dlarray objects, see List of Functions with dlarray Support.

Consistent Data Types and GPU Compatibility

These tests check that the layer function outputs are consistent in type and that the layer functions are GPU compatible.

If the layer forward functions fully support dlarray objects, then the layer is GPU compatible. Otherwise, to be GPU compatible, the layer functions must support inputs and return outputs of type gpuArray (Parallel Computing Toolbox).

Many MATLAB^® built-in functions support gpuArray (Parallel Computing Toolbox) and dlarray input arguments. For a list of functions that support dlarray objects, see List of Functions with dlarray Support. For a list of functions that execute on a GPU, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox). To use a GPU for deep learning, you must also have a supported GPU device. For information on supported devices, see GPU Computing Requirements (Parallel Computing Toolbox). For more information on working with GPUs in MATLAB, see GPU Computing in MATLAB (Parallel Computing Toolbox).

Intermediate Layers.  The tests predictIsConsistentInType, forwardIsConsistentInType, and backwardIsConsistentInType check that the layer functions output variables of the correct data type. The tests check that the layer functions return consistent data types when given inputs of the data types single, double, and gpuArray with the underlying types single or double.

Tip

If you preallocate arrays using functions such as zeros, then you must ensure that the data types of these arrays are consistent with the layer function inputs. To create an array of zeros of the same data type as another array, use the "like" option of zeros. For example, to initialize an array of zeros of size sz with the same data type as the array X, use Z = zeros(sz,"like",X).

Test Diagnostic	Description	Possible Solution
Incorrect type of 'Z' for 'predict'.	The types of the outputs Z1,…,Zm of the predict function must be consistent with the inputs X1,…,Xn.	Return the outputs Z1,…,Zm with the same type as the inputs X1,…,Xn.
Incorrect type of output 'out1' for 'predict'.
Incorrect type of 'Z' for 'forward'.	The types of the outputs Z1,…,Zm of the optional forward function must be consistent with the inputs X1,…,Xn.
Incorrect type of output 'out1' for 'forward'.
Incorrect type of 'dLdX' for 'backward'.	The types of the derivatives dLdX1,…,dLdXn of the optional backward function must be consistent with the inputs X1,…,Xn.	Return the derivatives dLdX1,…,dLdXn with the same type as the inputs X1,…,Xn.
Incorrect type of the derivative of the loss with respect to the input 'in1' for 'backward'.
Incorrect type of the derivative of loss with respect to 'W' for 'backward'.	The type of the derivative of the loss of the learnable parameters must be consistent with the corresponding learnable parameters.	For each learnable parameter, return the derivative with the same type as the corresponding learnable parameter.

Tip

If the layer forward functions support dlarray objects, then the software automatically determines the backward function and you do not need to specify the backward function. For a list of functions that support dlarray objects, see List of Functions with dlarray Support.

Output Layers.  The tests forwardLossIsConsistentInType and backwardLossIsConsistentInType check that the layer functions output variables of the correct data type. The tests check that the layers return consistent data types when given inputs of the data types single, double, and gpuArray with the underlying types single or double.

Test Diagnostic	Description	Possible Solution
Incorrect type of 'loss' for 'forwardLoss'.	The type of the output loss of the forwardLoss function must be consistent with the input Y.	Return loss with the same type as the input Y.
Incorrect type of the derivative of loss 'dLdY' for 'backwardLoss'.	The type of the output dLdY of the optional backwardLoss function must be consistent with the input Y.	Return dLdY with the same type as the input Y.

Tip

If the forwardLoss function supports dlarray objects, then the software automatically determines the backward loss function and you do not need to specify the backwardLoss function. For a list of functions that support dlarray objects, see List of Functions with dlarray Support.

Correct Gradients

The test gradientsAreNumericallyCorrect checks that the gradients computed by the layer functions are numerically correct. The test backwardPropagationDoesNotError checks that the derivatives can be computed using automatic differentiation.

Intermediate Layers.  When the optional backward function is not specified, the test backwardPropagationDoesNotError checks that the derivatives can be computed using automatic differentiation. When the optional backward function is specified, the test gradientsAreNumericallyCorrect tests that the gradients computed in backward are numerically correct.

Test Diagnostic	Description	Possible Solution
Expected a dlarray with no dimension labels, but instead found labels.	When the optional backward function is not specified, the layer forward functions must output dlarray objects without dimension labels.	Ensure that any dlarray objects created in the layer forward functions do not contain dimension labels.
Unable to backward propagate through the layer. Check that the 'forward' function fully supports automatic differentiation. Alternatively, implement the 'backward' function manually.	One or more of the following: When the optional backward function is not specified, the layer forward functions do not support dlarray objects. When the optional backward function is not specified, the tracing of the input dlarray objects in the forward functions have been broken. For example, by using the extractdata function.	Check that the forward functions support dlarray objects. For a list of functions that support dlarray objects, see List of Functions with dlarray Support. Check that the derivatives of the input dlarray objects can be traced. To learn more about the derivative trace of dlarray objects, see Derivative Trace. Alternatively, define a custom backward function by creating a function named backward. To learn more, see Specify Custom Layer Backward Function.
Unable to backward propagate through the layer. Check that the 'predict' function fully supports automatic differentiation. Alternatively, implement the 'backward' function manually.
The derivative 'dLdX' for 'backward' is inconsistent with the numerical gradient.	One or more of the following: When the optional backward function is specified, the derivative is incorrectly computed The forward functions are non-differentiable at some input points Error tolerance is too small	If the layer forward functions support dlarray objects, then the software automatically determines the backward function and you can omit the backward function. For a list of functions that support dlarray objects, see List of Functions with dlarray Support. Check that the derivatives in backward are correctly computed. If the derivatives are correctly computed, then in the Framework Diagnostic section, manually check the absolute and relative error between the actual and expected values of the derivative. If the absolute and relative errors are within an acceptable margin of the tolerance, then you can ignore this test diagnostic.
The derivative of the loss with respect to the input 'in1' for 'backward' is inconsistent with the numerical gradient.
The derivative of loss with respect to 'W' for 'backward' is inconsistent with the numerical gradient.

Tip

If the layer forward functions support dlarray objects, then the software automatically determines the backward function and you do not need to specify the backward function. For a list of functions that support dlarray objects, see List of Functions with dlarray Support.

Output Layers.  When the optional backwardLoss function is not specified, the test backwardPropagationDoesNotError checks that the derivatives can be computed using automatic differentiation. When the optional backwardLoss function is specified, the test gradientsAreNumericallyCorrect tests that the gradients computed in backwardLoss are numerically correct.

Test Diagnostic	Description	Possible Solution
Expected a dlarray with no dimension labels, but instead found labels	When the optional backwardLoss function is not specified, the forwardLoss function must output dlarray objects without dimension labels.	Ensure that any dlarray objects created in the forwardLoss function does not contain dimension labels.
Unable to backward propagate through the layer. Check that the 'forwardLoss' function fully supports automatic differentiation. Alternatively, implement the 'backwardLoss' function manually	One or more of the following: When the optional backwardLoss function is not specified, the layer forwardLoss function does not support dlarray objects. When the optional backwardLoss function is not specified, the tracing of the input dlarray objects in the forwardLoss function has been broken. For example, by using the extractdata function.	Check that the forwardLoss function supports dlarray objects. For a list of functions that support dlarray objects, see List of Functions with dlarray Support. Check that the derivatives of the input dlarray objects can be traced. To learn more about the derivative trace of dlarray objects, see Derivative Trace. Alternatively, define a custom backward loss function by creating a function named backwardLoss. To learn more, see Specify Custom Output Layer Backward Loss Function.
The derivative 'dLdY' for 'backwardLoss' is inconsistent with the numerical gradient.	One or more of the following: The derivative with respect to the predictions Y is incorrectly computed Function is non-differentiable at some input points Error tolerance is too small	Check that the derivatives in backwardLoss are correctly computed. If the derivatives are correctly computed, then in the Framework Diagnostic section, manually check the absolute and relative error between the actual and expected values of the derivative. If the absolute and relative errors are within an acceptable margin of the tolerance, then you can ignore this test diagnostic.

Tip

If the forwardLoss function supports dlarray objects, then the software automatically determines the backward loss function and you do not need to specify the backwardLoss function. For a list of functions that support dlarray objects, see List of Functions with dlarray Support.

Valid States

For layers with state properties, the test predictReturnsValidStates checks that the predict function returns valid states. When forward is specified, the test forwardReturnsValidStates checks that the forward function returns valid states. The test resetStateDoesNotError checks that the resetState function returns a layer with valid state properties.

Test Diagnostic	Description	Possible Solution
Error using 'predict' in Layer. 'State' must be real-values numeric array or unformatted dlarray object.	State outputs must be real-valued numeric arrays or unformatted dlarray objects.	Ensure that the states identified in the Framework Diagnostic are real-valued numeric arrays or unformatted dlarray objects.
Error using 'resetState' in Layer. 'State' must be real-values numeric array or unformatted dlarray object	State properties of returned layer must be real-valued numeric arrays or unformatted dlarray objects.

Code Generation Compatibility

If you set the CheckCodegenCompatibility option to 1 (true), then the checkLayer function checks the layer for code generation compatibility.

The test codegenPragmaDefinedInClassDef checks that the layer definition contains the code generation pragma %#codegen. The test layerPropertiesSupportCodegen checks that the layer properties support code generation. The test predictSupportsCodegen checks that the outputs of predict are consistent in dimension and batch size.

Code generation supports intermediate layers with 2-D image or feature input only. Code generation does not support layers with state properties (properties with attribute State).

The checkLayer function does not check that functions used by the layer are compatible with code generation. To check that functions used by the custom layer also support code generation, first use the Code Generation Readiness app. For more information, see Check Code by Using the Code Generation Readiness Tool (MATLAB Coder).

Test Diagnostic	Description	Possible Solution
Specify '%#codegen' in the class definition of custom layer	The layer definition does not include the pragma "%#codegen" for code generation.	Add the %#codegen directive (or pragma) to your layer definition to indicate that you intend to generate code for this layer. Adding this directive instructs the MATLAB Code Analyzer to help you diagnose and fix violations that result in errors during code generation.
Nonscalar layer properties must be type single or double or character array for custom layer	The layer contains non-scalar properties of type other than single, double, or character array.	Convert non-scalar properties to use a representation of type single, double, or character array. For example, convert a categorical array to an array of integers of type double representing the categories.
Scalar layer properties must be numeric, logical, or string for custom layer	The layer contains scalar properties of type other than numeric, logical, or string.	Convert scalar properties to use a numeric representation, or a representation of type logical or string. For example, convert a categorical scalar to an integer of type double representing the category.
For code generation, 'Z' must have the same number of dimensions as the layer input.	The number of dimensions of the output Z of predict does not match the number of dimensions of the layer inputs.	In the predict function, return the outputs with the same number of dimensions as the layer inputs.
For code generation, 'Z' must have the same batch size as the layer input.	The size of the batch size of the output Z of predict does not match the size of the batch size of the layer inputs.	In the predict function, return the outputs with the batch size as the layer inputs.