How does Matlab build-in Quasi-Newton method calculate the gradient?

Hi there,

I use matlab build-in function 'fminunc' to run some optimization for my image registration problem. I set the options as:

options = optimset('LargeScale', 'off', 'Display', 'iter-detailed', 'TolX', 0.0001, 'TolFun', 0.001, 'GradObj', 'off', 'DerivativeCheck', 'off';

I think it then uses Quasi-Newton (L-BFGS) method to do the optimization.

When the 'GradObj' is 'off', then the 'fminunc' calculate the gradient of the parameters need to be optimzed for me. (I suppose the Matlab use finite difference method to get the gradient)

I also try to provide the gradient by myself using finite difference method as well, then I set 'GradObj' is 'on'.

However, I always got much better results when I use the Matlab's calculation (When 'GradObj' is 'off').

May I ask where can I find the codes the matlab build-in function calculating the gradient please? I wish to compare with my own implementation to find the mistakes I might make.

Thanks very much and have a nice day.

Aaron

Answer 1

[Andy]

I don't have a direct answer for you, but you can dig into it a little bit further by typing

edit fminunc

at the command line. It looks like some internal functions are called, not all of which are editable (perhaps some are MEX functions?). But you can see a couple of them:

edit optimfcnchk
edit sfminbx

Perhaps you can gain some insight from these functions, but it may just be the case that the algorithm is buried in a MEX function and TMW does not intend for users to be able to see it.

[Andy]

Ree, your "answer" below is not really another answer, but is a comment on this answer. It would be better for you to use the "comment" feature. The answers could change order over time, and it will be difficult to tell what your comment is responding to.

As for the finitedifferences function, it is probably an internal helper function. No help will come up, because TMW does not intend for you to use it. I think this might just be a case where TMW does not want you looking under the hood. Another option is to look on the FEX for other users' implementations of the finite differences method and compare those to your own. They may not be as fast as the built-in function, but they will at least have all of their code available to you.

Good luck.

Answer 2

[Aaronne]

Hi Andy,

Thanks for your reply. Yes, I just try to dig out the 'fminunc.m' and find that it calls 'fminusub.m' to solve the medium scale optimization.

Then in the 'fminusub.m' it calls 'finitedifferences' to solve the gradient.

[gradFd,~,~,numEvals] = finitedifferences(x,funfcn{3},[],[],[],f,[],[], ...
        1:numberOfVariables,finDiffOpts,sizes,gradFd,[],[],finDiffFlags,[],varargin{:});

However, the 'finitedifferences' is a '.p' file, and we can't see the real implementation. But any one knows how to use this 'finitedifferences' function? Because I use

help finitedifferences

give me nothing...

Thanks again for anyone who help me. Cheers, Aaron