Too often people think that there is some magic salve they can apply to a nonlinear optimization to make it work. I suppose this is because others seem to make them work with no apparent problems. So what is wrong here? Where is the disconnect? There may be several reasons for failure. I'll just guess at a couple of them.
- Your data may be pure crap, insufficient to fit the model you desperately need to use. (And, no, I'm not insulting you or your data. But it is often the case that people have wildly insufficient data, and have no idea this is the case.) I don't know that, since I've not seen any data from you. But it is very often the case. A rank deficiency warning may well be a good hint of that possibility though. The answer is to get better data that will indeed support that model.
- On the other end, your data may be reasonable, but you may have simply chosen a model that has no possibility of being fit, thus an insane choice of model for this data. Again, I've seen no model, so I can only hazard a wild guess. But too often people try to fit high order polynomial models, or sums of many exponential terms. In either case it is trivially easy to generate problems that will see rank deficiencies, and there are many other poor choices of model one can make. The answer is probably to choose a better model.
- You may have serious scaling issues. So if some of your parameters are scaled to be on the order of 1e10, and others on the order of 1, then expect to see issues in the numerical linear algebra. This is easily fixable of course, by a simple scaling. Just use a better choice of units, thus perhaps kilometers versus nanometers.
- You may have chosen poor starting values, that cause the optimizer to diverge into limbo, to get lost. The answer to this is to choose better starting values.
It is not always easy to get better data. You may have what you have and cannot get more. Or you may be forced to use a specific model. And better starting values are often difficult to guess. Sorry, but not all things are trivial to do. The easy problems are already solved anyway.
Of course, one reason why some people manage to make their optimization work more easily is they know these things already. Throwing a different optimizer at the problem usually won't help. It will still probably see problems.
SOMETIMES a constrained optimizer might help IF you can find an intelligent set of constraints. That is, keep the problem from going to the bad place.
Another thing that often can help is to use a partitioned nonlinear least squares estimation scheme, since the reduces the effective nonlinear dimensionality of the problem. It makes the search space smaller, and thus easier to search. You can find tools for this in my own fminspleas, or in the pleas code I posted with my optimization tips and tricks submission.)
