fmincon trying to evaluate a vector full of NaN

Dear all,

This problem may be related to question I had. This is the link just in case:

http://www.mathworks.nl/matlabcentral/answers/38065-matlab-does-not-find-function-after-it-has-found-it-50-times

I am running a rather large fmincon optimization with 25 design variables. Earlier this week I had a strange problem (see link above) where Matlab did not find a file even though it found (and executed) that file multiple times before. I do not know if these errors are related. Now I have another problem I cannot explain. At the 218th evaluation the entire x vector is NaN. So after having evaluated 217 different x vectors, fmincon tries to evaluate a vector full of NaN. The objective and constraints are then all NaN as well. All subsequent evaluations are NaN vectors, until the optimizer halts because the maximum number of iterations has been reached. Can anyone explain this strange behavior of fmincon?

Here some additional information to solve the puzzle:

I use an SQP algorithm. The objective and constraint functions are ill behaved. Within the code sub functions have the habit of crashing for certain x vectors and throw exceptions. These are caught and the affected variables defined NaN, so that the returns (objective and constrained function values) are subsequently also NaN. SQP supposedly handles NaN returns by reducing the step size until the returns are real again. This happens a few times during execution of the code.

I saved all x vectors and the resulting objective and constraint values in a history file. There I can see that evaluation number 173 is a new step, in which the returns are NaN. Thus, as described above, the solver reduces the step size for evaluation 174. This still yields NaN returns. fmincon reduces stepsizes the subsequent evaluations because the returns are still NaN. This happens until evaluation 217. At the 217th evaluation the x vector is almost the same as the x vector of the 173rd evaluation. This tells me that the last step the solver took, the x vector of the 173rd evaluation, is very close to a region where the returns are NaN. So far so clear. But then, instead of exiting or trying a new step, fmincon tries only NaN vectors. That I don't understand.

Any help would be greatly appreciated.

Answer 1

[Sargondjani]

maybe you can post important bits of your codes? (or the whole code?)

if you use finite differences, one possible cause is that the gradient of one (or several) variables becomes extremely large: in the next step matlab will then try to evaluate irrealistic function values. this could be a cause of the problem

a possible workaround is to use the active-set algorithm with options: 'AlwaysHonorConstraints',''bounds'. this way the algorithm always honours the bounds and so prevent matlab from evaluating irrealistic values for the inputs...

[Vincent]

Dear Sargondjani, thx for answering.

The code is large and spans multiple mfiles and pfiles. The only possibility I see is to attach or upload it.

I don't think it has to do with fmincon taking large steps. Why would that result in an NaN x vector? Besides, every variable is bound and sqp honors the bounds always.

The active-set algorithm does not work because in this case as it cannot recover from NaN (this is on my Matlab 2013 wish-list, please dear developers). NaN returns are unavoidable for this optimization.

[Sargondjani]

o yes, you are right about sqp: thanks for reminding, haha

i suppose subsequent x values are NaN because the gradient/Hessian give NaN (at 173 and 217 step). did you save history of gradient and hessian as well??

[Vincent]

Dear Sargondjani,

I did not save the history of gradient or hessian. I wonder now, is it possible that the problem arises because the step size is too small? Could I solve the problem by specifying DifMinChange and DifMaxChange?

[Sargondjani]

the finite differences is likely to help in creating problems! if you can supply an analytical gradient,you should always do that, but i guess that's kind of hard for your problem...

anyway, even if you change DiffMinChange i think you should still check if the problem arises because of an NaN in the gradient/Hessian...

[Vincent]

I cannot supply an analytical gradient unfortunately.

How can I take a look into the Hessian? I could not find how I see Hessian values while the optimization is running.

Lets assume the problem indeed lies in the Hessian, how could I solve this problem? Could anybody from Mathworks please help?