# idGaussianProcess

Gaussian process regression mapping function for nonlinear ARX and Hammerstein-Wiener models (requires Statistics and Machine Learning Toolbox)

## Description

An idGaussianProcess object implements a Gaussian process (GP) regression model, and is a nonlinear mapping function for estimating nonlinear ARX and Hammerstein-Wiener models. This mapping object, which is also referred to as a nonlinearity, incorporates RegressionGP (Statistics and Machine Learning Toolbox) objects that the mapping function creates using Statistics and Machine Learning Toolbox™. The mapping object contains three components: an offset, a nonlinear component, which, in this case, is the GP kernel, and a linear component that uses a combination of linear weights.

The input to the mapping object can be an internal signal of a nonlinear black-box model, such as one of the following signals:

• Vector of the regressors of a nonlinear ARX model

• True model input of a Hammerstein-Wiener model

• Output of the linear block of a Hammerstein-Wiener model

Mathematically, idGaussianProcess is a function that maps m inputs X(t) = [x(t1),x2(t),…,xm(t)]T to a scalar output y(t) using the following relationship:

$y\left(t\right)={y}_{0}+Χ{\left(t\right)}^{T}PL+G\left(Χ\left(t\right),\theta \right)\right)$

Here,

• X(t) is an m-by-1 vector of inputs, or regressors.

• y0 is the output offset, a scalar.

• P is an m-by-p projection matrix, where m is the number of regressors and p is the number of linear weights. m must be greater than or equal to p.

• L is a p-by-1 vector of weights.

• G(X,θ) is the regressive Gaussian process that constitutes the kernel of the idGaussianProcess object. G has a mean of zero and a covariance that the user specifies by choosing a kernel, and can be expressed generally as

$G\left(X\right)=GP\left(0,K\left({X}_{test},{X}_{train},\theta \right)\right)$

A zero-mean Gaussian process G predicts the output Ytest for a given input Xtest using the following relationship:

$G\left({X}_{test}\right)=K\left({X}_{test},{X}_{train}\right){\left[K\left({X}_{train},{X}_{train}\right)+{\sigma }_{n}{}^{2}I\right]}^{-1}{Y}_{train}$

Here:

• K(Xtest,Xtrain) is the covariance kernel function.

• Xtrain is a matrix representing the set of training inputs.

• Xtest is a matrix representing the set of test inputs.

• Ytrain is the vector of outputs from the training set.

• σn is the standard deviation of the additive measurement noise.

Gaussian process modeling is especially useful when you have only limited measurement data. For more information about creating GP regression models, see fitrgp (Statistics and Machine Learning Toolbox).

Use idGaussianProcess as the value of the OutputFcn property of an idnlarx model or the OutputNonlinearity property (but not the InputNonlinearity property) of an idnlhw object. For example, specify idGaussianProcess when you estimate an idnlarx model with the following command.

sys = nlarx(data,regressors,idGaussianProcess)
When nlarx estimates the model sys, it essentially estimates the parameters of the idGaussianProcess function.

You can use a similar approach when you specify input or output linearities using the nlhw command. For example, specify idGaussianProcess as an input nonlinearity with the following command.

sys = nlhw(data,orders,idGaussianProcess,idSaturation)

You can configure the idGaussianProcess function to disable components and fix parameters. To omit the linear component, set LinearFcn.Use to false. To omit the offset, set Offset.Use to false. To specify known values for the linear function and the offset, set their Value attributes directly and set the corresponding Free attributes to False. To modify the estimation options, set the option property in EstimationOptions. For example, to change the fit method to 'exact', use G.EstimationOptions.FitMethod = 'exact'. Use evaluate to compute the output of the function for a given vector of inputs.

## Creation

### Description

G = idGaussianProcess creates an idGaussianProcess object G with the kernel function 'SquaredExponential' and default kernel parameters. The number of inputs is determined during model estimation and the number of outputs is 1.

example

G = idGaussianProcess(kernelFunction) specifies a specific kernel.

G = idGaussianProcess(kernelFunction,kernelParameters) initializes the parameters of the specified kernel to the values in kernelParameters.

example

G = idGaussianProcess(kernelFunction,kernelParameters,UseLinearFcn) specifies whether the function uses a linear function as a subcomponent.

example

G = idGaussianProcess(kernelFunction,kernelParameters,UseLinearFcn,UseOffset) specifies whether the function uses an offset term y0 parameter.

### Input Arguments

expand all

Kernel covariance function, specified as character array or string. For information about the individual options, see Kernel (Covariance) Function in fitrgp (Statistics and Machine Learning Toolbox).

This argument sets the G.Kernel.KernelFunction property.

Initial values for the kernel parameters, specified as a vector. The size of the vector and the values depend on the choice of kernelFunction. For more information, see Kernel Parameters in fitrgp (Statistics and Machine Learning Toolbox).

This argument sets the G.Kernel.Parameters.Value property.

Option to use the linear function subcomponent, specified as true or false. This argument sets the value of the G.LinearFcn.Use property.

Option to use an offset term, specified as true or false. This argument sets the value of the G.Offset.Use property.

## Properties

expand all

Input signal names for the inputs to the mapping object, specified as a 1-by-m cell array, where m is the number of input signals. This property is determined during estimation.

Output signal name for the output of the mapping object, specified as a 1-by-1 cell array. This property is determined during estimation.

Properties of the GP kernel, specified as follows:

• KernelFunction —Covariance kernel function K, specified as one of the values listed in the kernelFunction argument description. For more information about these options, see Kernel (Covariance) Function in fitrgp (Statistics and Machine Learning Toolbox).

• Parameters — Parameters used in the kernel function, specified as the following properties:

• Value — Values of the kernel parameters, specified as a vector.

• Names — Names of the kernel parameters

• InputProjection — Projection matrix used to project inputs onto a lower dimensional subspace.

The size of the value vector and the values depend on the choice of KernelFunction. For more information, see Kernel Parameters in fitrgp (Statistics and Machine Learning Toolbox).

• Free — Option to estimate parameters, specified as a logical scalar. If all the parameters have finite values, such as when the idGaussianProcess object corresponds to a previously estimated model, then setting Free to false causes the parameters of the kernel G(X) to remain unchanged during estimation. The default value is true.

Parameters of the linear function, specified as follows:

• Use — Option to use the linear function in the idGaussianProduct model, specified as a scalar logical. The default value is true.

• Value — Linear weights that compose L', specified as a 1-by-p vector.

• InputProjection — Input projection matrix P, specified as an m-by-p matrix, that transforms the detrended input vector of length m into a vector of length p.

• Free — Option to update entries of Value during estimation, specified as a 1-by-p logical vector. The software honors the Free specification only if the starting value of Value is finite. The default value is true.

Parameters of the offset term, specified as follows:

• Use — Option to use the offset in the idGaussianProcess model, specified as a scalar logical. The default value is true.

• Value — Offset value, specified as a scalar.

• Free — Option to update Value during estimation, specified as a scalar logical. The software honors the Free specification of false only if the value of Value is finite. The default value is true.

Estimation options for the nonlinear block of the idGaussianProcess model, specified as follows. For more information on any of these options, see fitrgp (Statistics and Machine Learning Toolbox).

• FitMethod — Method to use for estimating the parameters of the idGaussianProcess nonlinear model, specified as one of the items in the following table.

Option Description
'auto'

Software selects the method automatically (default)

'exact'

Exact GP regression

'sd'

Subset of data points approximation

'sr'

Subset of regressors approximation

'fic'

Fully independent conditional approximation

• ActiveSetMethod — Active set selection method, specified as one of the items in the following table.

Option Description
'random'

Random selection (default)

'sgma'

Sparse greedy matrix approximation

'entropy'Differential entropy-based selection
'likelihood'

Subset of regressors log likelihood-based selection

• SparseFitRegularization — Regularization standard deviation for the sparse methods subset of regressors ('sr') and the fully independent conditional approximation ('fic'), specified as a positive scalar value.

• Optimizer — Optimizer to use for parameter estimation, specified as one of the items in the following table.

Option Description
'quasinewton'Dense, symmetric rank-1-based, quasi-Newton approximation to the Hessian (default)
'lbfgs'LBFGS-based quasi-Newton approximation to the Hessian
'fminsearch'Unconstrained nonlinear optimization using the simplex search method of Lagarias et al. [see fitrgp (Statistics and Machine Learning Toolbox)]
'fminunc'Unconstrained nonlinear optimization (requires an Optimization Toolbox™ license)
'fmincon'Constrained nonlinear optimization (requires an Optimization Toolbox license)
• OptimizerOptions — Options for the optimizer, specified as a structure or object. When Optimizer is set or changed, the software automatically updates the value of OptimizerOptions to match the defaults for the corresponding optimizer. Use the properties of the OptimizerOptions option set to change the values from their defaults.

## Examples

collapse all

Load the input/output data from twotankdata and construct an iddata object z.

z = iddata(y,u,0.8,'timeunit','hours');

Create an idGaussianProduct mapping object g that uses a Matern kernel with the parameter 3/2.

g = idGaussianProcess('Matern32');

Estimate a nonlinear ARX model that uses g as the output function.

sys = nlarx(z,[4 4 1],g)
sys =
Nonlinear ARX model with 1 output and 1 input
Inputs: u1
Outputs: y1

Regressors:
Linear regressors in variables y1, u1
List of all regressors

Output function: Gaussian process function using a Matern32 kernel
Sample time: 0.8 hours

Status:
Estimated using NLARX on time domain data "z".
Fit to estimation data: 97.14% (prediction focus)
FPE: 2.82e-05, MSE: 2.795e-05

Display the postestimation properties of g.

disp(sys.OutputFcn.Input)
Function inputs

Name: {1x8 cell}
Mean: [-4.7062e-17 -5.3807e-17 -5.2324e-17 -8.3748e-18 1.0174e-15 ... ]
Range: [2x8 double]
disp(sys.outputFcn.Offset)
Output Offset: initialized to -8.11e-17
Use: 1
Value: -8.1058e-17
Free: 1
disp(sys.outputFcn.NonlinearFcn)
GP kernel and its parameters

KernelFunction: 'Matern32'
Parameters: '<Kernel parameters>'
Free: 1
Inputs: {1x8 cell}
Outputs: {'y1(t):Nonlinear'}

Compare the output of sys with the measured output z.

compare(z,sys)

The nonlinear model shows a good fit to the estimation data.

Create an idGaussianProcess object G that sets the UseLinearFcn argument to 0. Since this argument is third in the syntax, you must also specify kernelFunction and kernelParameters. Set kernelFunction to its default value of 'SquaredExponential'. Set the kernelParameters argument to [], which specifies no initialization for the parameters.

kernelFunction = 'SquaredExponential';
kernelParameters = [];
UseLinearFcn = 0;
G = idGaussianProcess(kernelFunction,kernelParameters,UseLinearFcn)
G =
Gaussian Process Function

Nonlinear Function: Gaussian process function using a SquaredExponential kernel
Linear Function: not in use
Output Offset: uninitialized

Inputs: {1x0 cell}
Outputs: {1x0 cell}
Kernel: 'GP kernel and its parameters'
LinearFcn: 'Linear function parameters'
Offset: 'Offset parameters'
EstimationOptions: 'Estimation option set'

The properties of G are consistent with your inputs.

Create an idGaussianProcess object that has no offset block by first creating a default object, and then, using dot notation to set the G.Offset.Use property directly.

G = idGaussianProcess;
G.Offset.Use = 0
G =
Gaussian Process Function

Nonlinear Function: Gaussian process function using a SquaredExponential kernel
Linear Function: uninitialized
Output Offset: not in use

Inputs: {1x0 cell}
Outputs: {1x0 cell}
Kernel: 'GP kernel and its parameters'
LinearFcn: 'Linear function parameters'
Offset: 'Offset parameters'
EstimationOptions: 'Estimation option set'

The function description identifies the output offset as not in use.

## Version History

Introduced in R2021b

expand all

Not recommended starting in R2022a

Behavior changed in R2022a