Computational Efficiency for Decoupled Optimisation

Hi all

Just looking for some best practice advice for setting up an optimisation study with a large number of variables. The cost function consists of multiple sub-models that don't interact with each other, but the outputs of all of them are summed to provide the cost value. Does anyone know if it's more computationally efficient to have one large optimisation process or to optimise each of the sub-models separately?

Thanks for your help,

Martin

Answer 1

[Matt J]

You haven't mentioned any constraints, so I'm assuming fminunc would apply to your problem. If you do have constraints, note that they too would have to be independent for each sub-model in order for the overall optimization to be decouplable.

Assuming you have the memory to store the gradient and Hessian in sparse form, it should be more efficient to solve simultaneously, however, you would have to set up the option parameter input judiciously.

https://www.mathworks.com/help/optim/ug/fminunc.html#butpb7p-options

It would be necessary to use the trust-region reflective algorithm, which allows you to take advantage of the block sparsity of your Hessian, via either the HessPattern, HessianMultiplyFcn, or HessianFcn options. Note that with the trust-region method, you are also required to supply your own gradient calculation (SpecifyObjectiveGradient=true).

Answer 2

[Alan Weiss]

Because the problems can be solved separately, then it will certainly save memory to solve them separately. Whether it saves time to do so depends on a lot of details. You can try solving just a few at once and then separately, timing the solutions, then take a few more and try again, see if a pattern emerges. For a very large problem it will be necessary to solve sub-problems because of memory issues, but where the optimal cutoff is for problem size I cannot say.

Alan Weiss

MATLAB mathematical toolbox documentation