problem with fminsearch in estimating hyper-parameters for GP

I am working on Gaussian Process Regression with time series problem, the task is to build the model without using any library or MATLAB fitgpr function, I have the function myGP with which I run an optimization task to estimate the hyper-parameters :

function [cov,mu,loss,theta]=myGP(x,y,x_star,y_hat)
segmaSE = 1;
lengthSE = 1;
N=size(y,1);
theta0 =[segmaSE lengthSE];
kernelFunc = @(x1,x2,theta)(theta(1)^2)*exp(-0.5*(pdist2(x1/theta(2), x2/theta(2)).^2));
marginal_likelihood =@(y,x,theta,N) -0.5*y'*pinv(kernelFunc(x,x,theta))*y - 0.5*log(abs(kernelFunc(x,x,theta))) - N*.5 * log (2*pi);
fun = @(theta)marginal_likelihood(y,x,theta,N);
theta=fminsearch(fun,theta0);
     k_x_x=kernelFunc(x,x,theta);%+theta(end)*eye(size(x,1),size(x,1));%k(x,x)
     k_x_xs=kernelFunc(x,x_star,theta);%k(x,x*)
     k_xs_xs=kernelFunc(x_star,x_star,theta);%k(x*,x*)
  %%Calculate prediction / cov / RMSE
     Ki=k_x_xs'*pinv(k_x_x);
     mu=Ki*y;
     cov=k_xs_xs - Ki*k_x_xs;
     loss=rmse(y_hat,mu);    
     cov=diag(cov);

end

However, I have a problem when I run the function fminsearch I got the following problem

Assignment has more non-singleton rhs dimensions than non-singleton subscripts

Error in fminsearch (line 200)
fv(:,1) = funfcn(x,varargin{:});

Error in myGP (line 21)
theta=fminsearch(fun,theta0);

Any help about this error will be very appreciated

Answer 1

[Walter Roberson]

You do not give us any information about the sizes of the variables, which makes it difficult to test.

I notice that you always call kernelFunc() with (x, x, theta). If x is scalar or row vector then the result of the pdist2() call will be 0. If x is N x M for N > 1 then the result of the pdist2() will be N x N. The exp() will not change that, and multiplying by the scalar will not change that.

pinv() of N x N will be N x N . In order for pinv()*y to work, y must be N x P for some P, with the * giving an N x P result. The y' * before that would be * of a P x N, so that would be P x N * N * P, giving a P x P result. You multiply that by -0.5 and you subtract 0.5*log(abs(kernelFunc(x,x,theta))) where we have already determined that the kernelFunc returns an N x N result. P x P minus N x N will work under the circumstance that P is 1 or P is N; either way the subtraction going to return a N x N result .

Now, the objective function return value for fminsearch needs to be a scalar. For N x N to be scalar, then x would have had to have been N x M = 1 x M -- but in that case the pdist2() would return 0 making it essentially useless to make the pdist2() call.

This suggests that your formula is either fundamentally incorrect (but it does not look to be wrong), or else that you are asking for a matrix minimization, which fminsearch() cannot handle.

[Mohamad Alsioufi]

Thank you for the answer, for the variables sizes x: N*1 y: N*1 x_star:M*1 y_hat:M*1 what I am looking for is to minimize the hyper-parameters vector theta

[Walter Roberson]

The result of your formula is NxN, not a vector or scalar.

fminsearch cannot be used to minimize a vector or matrix result, only a scalar. Perhaps you need to minimize the sum of squares of the matrix.

Answer 2

[Matt J]

Any help about this error will be very appreciated

You waste your own time by using the Answers forum as a tool to diagnose error messages. Using Matlab debugging tools,

https://www.mathworks.com/help/matlab/matlab_prog/set-breakpoints.html#buxup99-1

you could trace the problem yourself far quicker.

[Mohamad Alsioufi]

Thank you for your answer I will check this!