idGaussianProcess

Gaussian process regression mapping function for nonlinear ARX models (requires Statistics and Machine Learning Toolbox)

Description

An `idGaussianProcess` object implements a Gaussian process regression model, and is a nonlinear mapping function for estimating nonlinear ARX models. This mapping object, which is also referred to as a nonlinearity, incorporates `RegressionGP` (Statistics and Machine Learning Toolbox) objects that the mapping function creates using Statistics and Machine Learning Toolbox™. The mapping object contains three components: a linear component that uses a combination of linear weights, an offset, and a nonlinear component.

Mathematically, `idGaussianProcess` is a function that maps m inputs X(t) = [x(t1),x2(t),…,xm(t)]T to a scalar output y(t) using the following relationship:

`$y\left(t\right)={y}_{0}+{\left(Χ\left(t\right)-\overline{X}\right)}^{T}PL+G\left(Χ\left(t\right),\theta \right)\right)$`

Here,

• X(t) is an m-by-1 vector of inputs, or regressors, with mean $\overline{Χ}$.

• y0 is the output offset, a scalar.

• P is an m-by-p projection matrix, where m is the number of regressors and p is the number of linear weights. m must be greater than or equal to p.

• L is a p-by-1 vector of weights.

• G(X,θ) is the regressive Gaussian process that constitutes the nonlinear component of the `idGaussianProcess` object. G has a mean of zero and a covariance that the user specifies by choosing a kernel, and can be expressed generally as

`$G\left(X\right)=GP\left(0,K\left({X}_{test},{X}_{train},\theta \right)\right)$`

The Gaussian process G can be defined more precisely as the predicted value of a test input for a given value of θ. In this case, the relationship becomes:

`$G\left(X\right)=K\left({X}_{test},{X}_{train}\right){\left[K\left({X}_{train},{X}_{train}\right)+{\sigma }_{n}{}^{2}I\right]}^{-1}{Y}_{train}$`

Here:

• K(Xtest,Xtrain) is the covariance kernel function.

• Xtrain is a matrix representing the set of training inputs.

• Xtest is a matrix representing the set of test inputs.

• Ytrain is the vector of outputs from the training set.

• σn is the standard deviation of the additive measurement noise.

For more information about creating Gaussian process regression models, see `fitrgp` (Statistics and Machine Learning Toolbox).

Use `idGaussianProcess` as the value of the `OutputFcn` property of an `idnlarx` model. For example, specify `idGaussianProcess` when you estimate an `idnlarx` model with the following command.

`sys = nlarx(data,regressors,idGaussianProcess)`
When `nlarx` estimates the model, it essentially estimates the parameters of the `idGaussianProcess` function.

You can configure the `idGaussianProcess` function to disable components and fix parameters. To omit the linear component, set `LinearFcn.Use` to `false`. To omit the offset, set `Offset.Use` to `false`. To specify known values for the linear function and the offset, set their `Value` attributes directly and set the corresponding `Free` attributes to `False`. To modify the estimation options, set the option property in `EstimationOptions`. For example, to change the fit method to `'exact'`, use `G.EstimationOptions.FitMethod = 'exact'`. Use `evaluate` to compute the output of the function for a given vector of inputs.

Creation

Syntax

``G = idGaussianProcess``
``G = idGaussianProcess(kernel)``
``G = idGaussianProcess(kernel,kernelParameters)``
``G = idGaussianProcess(kernel,kernelParameters,UseLinearFcn)``
``G = idGaussianProcess(kernel,kernelParameters,UseLinearFcn,UseOffset)``

Description

````G = idGaussianProcess` creates an `idGaussianProcess` object `G` with the kernel function `'SquaredExponential'` and default kernel parameters. The number of inputs is determined during model estimation and the number of outputs is 1. ```

example

````G = idGaussianProcess(kernel)` specifies a specific kernel.```
````G = idGaussianProcess(kernel,kernelParameters)` initializes the parameters of the specified kernel to the values in `kernelParameters`. ```
````G = idGaussianProcess(kernel,kernelParameters,UseLinearFcn)` specifies whether the function uses a linear function as a subcomponent.```
````G = idGaussianProcess(kernel,kernelParameters,UseLinearFcn,UseOffset)` specifies whether the function uses an offset term y0 parameter.```

Input Arguments

expand all

Kernel covariance function, specified as character array or string. For information about the options, see Kernel (Covariance) Function in `fitrgp` (Statistics and Machine Learning Toolbox).

This argument sets the `G.NonlinearFcn.KernelFunction` property.

Initial values for the kernel parameters, specified as a vector. The size of the vector and the values depend on the choice of `kernel`. For more information, see Kernel Parameters in `fitrgp` (Statistics and Machine Learning Toolbox).

This argument sets the `G.NonlinearFcn.Parameters` property.

Option to use the linear function subcomponent, specified as `true` or `false`. This argument sets the value of the `G.LinearFcn.Use` property.

Option to use an offset term, specified as `true` or `false`. This argument sets the value of the `G.Offset.Use` property.

Properties

expand all

Input signal information for signals used for estimation, specified as vectors of m property-specific values, where m is the number of input signals. The `Input` properties for each input signal are as follows:

• `Name` — Names of the input signals, specified as a 1-by-m string or character array, where m is the number of inputs

• `Mean` — Mean of the input signals, specified as a numeric scalar

• `Range` — Ranges of the input signals, specified as a 2-by-m numeric array that contains the minimum and maximum values

Output signal information, specified as property-specific values. The `Output` properties are as follows:

• `Name` — Name of the output signal, specified as a string or a character array

• `Mean` — Mean of the output signal, specified as a numeric scalar

• `Range` — Range of the output signal, specified as a 2-by-1 numeric array that contains the minimum and maximum values

Parameters of the linear function, specified as follows:

• `Use` — Option to use the linear function in the `idGaussianProduct` model, specified as a scalar logical. The default value is `true`.

• `Value` — Linear weights that compose L', specified as a 1-by-p vector.

• `InputProjection` — Input projection matrix P, specified as an m-by-p matrix, that transforms the detrended input vector of length m into a vector of length p.

• `Free` — Option to update entries of `Value` during estimation, specified as a 1-by-p logical vector. The software honors the `Free` specification only if the starting value of `Value` is finite. The default value is `true`.

• `Minimum` — Minimum bound on `Value`, specified as a 1-by-p vector. If `Minimum` is specified with a finite value and the starting value of `Value` is finite, then the software enforces that minimum bound during model estimation.

• `Maximum` — Maximum bound on `Value`, specified as a 1-by-p vector. If `Maximum` is specified with a finite value and the starting value of `Value` is finite, then the software enforces that maximum bound during model estimation.

• `SelectedInputIndex` — Indices of `idGaussianProcess` inputs (see `Input.Name`) that are used as inputs to the linear function, specified as an 1-by-nr integer vector, where nr is the number of inputs. For nonlinear ARX models, the `RegressorUsage` property determines these indices..

Parameters of the offset term, specified as follows:

• `Use` — Option to use the offset in the `idGaussianProcess` model, specified as a scalar logical. The default value is `true`.

• `Value` — Offset value, specified as a scalar.

• `Free` — Option to update `Value` during estimation, specified as a scalar logical. The software honors the `Free` specification of `false` only if the value of `Value` is finite. The default value is `true`.

• `Minimum` — Minimum bound on `Value`, specified as a numeric scalar or `–Inf`. If `Minimum` is specified with a finite value and the value of `Value` is finite, then the software enforces that minimum bound during model estimation. The default value is `-Inf`.

• `Maximum` — Maximum bound on `Value`, specified as a numeric scalar or `Inf`. If `Maximum` is specified with a finite value and the starting value of `Value` is finite, then the software enforces that maximum bound during model estimation. The default value is `Inf`.

Parameters of the nonlinear function, specified as follows:

• `KernelFunction` — Kernel covariance kernel function, specified as one of the values listed in the `kernel` argument description. For more information about these options, see Kernel (Covariance) Function in `fitrgp` (Statistics and Machine Learning Toolbox).

• `Parameters` — Parameters of the `idGaussianProcess` kernel function, specified as a vector. The size of the vector and the values depend on the choice of `KernelFunction`. For more information, see Kernel Parameters in `fitrgp` (Statistics and Machine Learning Toolbox).

• `Free` — Option to estimate parameters, specified as a logical scalar. If all the parameters have finite values, such as when the `idGaussianProcess` object corresponds to a previously estimated model, then setting `Free` to `false` causes the parameters of the nonlinear function S(X) to remain unchanged during estimation. The default value is `true`.

• `SelectedInputIndex` — Indices of `idGaussianProcess` inputs (see `Input.Name`) that are used as inputs to the nonlinear function, specified as an 1-by-nr integer vector, where nr is the number of inputs. For nonlinear ARX models, the `RegressorUsage` property determines these indices.

Estimation options for the nonlinear block of the `idGaussianProcess` model, specified as follows. For more information on any of these options, see `fitrgp` (Statistics and Machine Learning Toolbox).

• `FitMethod` — Method to use for estimating the parameters of the `idGaussianProcess` nonlinear model, specified as one of the items in the following table.

Option Description
`'auto'`

Software selects the method automatically (default)

`'exact'`

Exact Gaussian process regression

`'sd'`

Subset of data points approximation

`'sr'`

Subset of regressors approximation

`'fic'`

Fully independent conditional approximation

• `ActiveSetMethod` — Active set selection method, specified as one of the items in the following table.

Option Description
`'random'`

Random selection (default)

`'sgma'`

Sparse greedy matrix approximation

`'entropy'`Differential entropy-based selection
`'likelihood'`

Subset of regressors log likelihood-based selection

• `SparseFitRegularization` — Regularization standard deviation for the sparse methods subset of regressors (`'sr'`) and the fully independent conditional approximation (`'fic'`), specified as a positive scalar value.

• `Optimizer` — Optimizer to use for parameter estimation, specified as one of the items in the following table.

Option Description
`'quasinewton'`Dense, symmetric rank-1-based, quasi-Newton approximation to the Hessian (default)
`'lbfgs'`LBFGS-based quasi-Newton approximation to the Hessian
`'fminsearch'`Unconstrained nonlinear optimization using the simplex search method of Lagarias et al. [see `fitrgp` (Statistics and Machine Learning Toolbox)]
`'fminunc'`Unconstrained nonlinear optimization (requires an Optimization Toolbox™ license)
`'fmincon'`Constrained nonlinear optimization (requires an Optimization Toolbox license)
• `OptimizerOptions` — Options for the optimizer, specified as a structure or object. When `Optimizer` is set or changed, the software automatically updates the value of `OptimizerOptions` to match the defaults for the corresponding optimizer. Use the properties of the `OptimizerOptions` option set to change the values from their defaults.

Examples

collapse all

Load the input/output data from `twotankdata` and construct an `iddata` object `z`.

```load twotankdata u y z = iddata(y,u,0.8,'timeunit','hours');```

Create an `idGaussianProduct` mapping object `g` that uses a Matern kernel with the parameter 3/2.

`g = idGaussianProcess('Matern32');`

Estimate a nonlinear ARX model that uses `g` as the output function.

`sys = nlarx(z,[4 4 1],g)`
```sys = Nonlinear ARX model with 1 output and 1 input Inputs: u1 Outputs: y1 Regressors: Linear regressors in variables y1, u1 List of all regressors Output function: Gaussian process function using a Matern32 kernel. Sample time: 0.8 hours Status: Estimated using NLARX on time domain data "z". Fit to estimation data: 97.08% (prediction focus) FPE: 2.945e-05, MSE: 2.92e-05 ```

Display the postestimation properties of `g`.

`disp(sys.OutputFcn.Input)`
```Function inputs Name: {1x8 cell} Mean: [0.2700 0.2699 0.2697 0.2696 6.4286 6.4286 6.4286 6.4286] Range: [2x8 double] ```
`disp(sys.outputFcn.Offset)`
```Output Offset: initialized to 0.27 Use: 1 Value: 0.2702 Free: 1 ```
`disp(sys.outputFcn.NonlinearFcn)`
```GP kernel and its parameters KernelFunction: 'Matern32' Parameters: '<Kernel parameters>' Free: 1 SelectedInputIndex: [1 2 3 4 5 6 7 8] ```

Compare the output of `sys` with the measured output `z`.

`compare(z,sys)`

The nonlinear model shows a good fit to the estimation data.

Introduced in R2021b

Get trial now