Fully Independent Conditional Approximation for GPR Models

The fully independent conditional (FIC) approximation[1] is a way of systematically approximating the true GPR kernel function in a way that avoids the predictive variance problem of the SR approximation while still maintaining a valid Gaussian process. You can specify the FIC method for parameter estimation by using the 'FitMethod','fic' name-value pair argument in the call to fitrgp. For prediction using FIC, you can use the 'PredictMethod','fic' name-value pair argument in the call to fitrgp.

Approximating the Kernel Function

The FIC approximation to k(xi,xj|θ) for active set AN={1,2,...,n} is given by:


That is, the FIC approximation is equal to the SR approximation if ij. For i=j, the software uses the exact kernel value rather than an approximation. Define an n-by-n diagonal matrix Ω(X|θ,A) as follows:


The FIC approximation to K(X,X|θ) is then given by:

K^FIC(X,X|θ,A)=K^SR(X,X|θ,A)+ Ω(X|θ,A)= K(X,XA|θ)K(XA,XA|θ)1K(XA,X|θ)+Ω(X|θ,A).

Parameter Estimation

Replacing K(X,X|θ) by K^FIC(X,X|θ,A) in the marginal log likelihood function produces its FIC approximation:


As in the exact method, the software estimates the parameters by first computing β^(θ,σ2), the optimal estimate of β, given θ and σ2. Then it estimates θ, and σ2 using the β-profiled marginal log likelihood. The FIC estimate to β for given θ, and σ2 is

β^FIC(θ,σ2,A)=[HT(K^FIC(X,X|θ,A)+σ2 IN)1H*]1HT(K^FIC(X,X|θ,A)+σ2 IN)1y**,


Using β^FIC(θ,σ2,A), the β-profiled marginal log likelihood for FIC approximation is:





The FIC approximation to the distribution of ynew given y, X, xnew is


where μFIC and ΣFIC are the FIC approximations to μ and Σ given in prediction using exact GPR method. As in the SR case, μFIC and ΣFIC are obtained by replacing all occurrences of the true kernel with its FIC approximation. The final forms of μFIC and ΣFIC are as follows:

μFIC= K(xnewT,XA|θ) BA1 K(XA,X|θ) Λ(θ,σ2,A)1(yHβ),





[1] Candela, J. Q. A Unifying View of Sparse Approximate Gaussian Process Regression. Journal of Machine Learning Research. Vol 6, pp. 1939–1959, 2005.

See Also


Related Topics