Running on AWS server does not benefit from more cores in parallel pool?
1 visualizzazione (ultimi 30 giorni)
Mostra commenti meno recenti
I am using the built in Genetic Algorithm functionality in the Optimization Toolbox and time is a serious constraint when running the code.
The problem in question is an integer (binary) problem with linear constraints.
The solutions are on the order of a 400x1 vector of ones and zeroes, and the population size is 500. All other (relevant) Genetic Algorithm options are left as default.
The main bottleneck on run time is within the fitness function (it does not receive the population in vectorized form because this conflicts with using a parallel pool). It has to perform numerous operations on a 3D matrix that is roughly 300000x4x10. These operations are vectorized as much as possible to take advantage of Matlab's speed.
I start a parallel processing pool before calling the algorithm as follows:
delete(gcp('nocreate'));
c = parcluster('local'); % build the 'local' cluster object
numberOfCores = 6;
parpool(numberOfCores);
With 6 cores it takes roughly five minutes for each generation to complete (when running as a compiled DLL called by .NET). I have tried to set up an AWS server with 20 cores in hopes of speeding up the run time of each generation, but instead the run time per generation was roughly the same or slightly worse. (Matlab verified that the 20 core pool was properly created).
Does anyone know why that would be the case? Is there some way to estimate if there is a "sweet spot" for number of cores in a parallel processing pool compared to any potential overhead associated with parallelizing operations?
1 Commento
Run Zhu
il 9 Nov 2017
Maybe it is off the topic. But the performance one would obtain is typically: optimization model > optimization algorithm > parallel computing. As in your case, any particular reason to choose GA? Since it is usually slower than intlinprog and patternsearch, consider other solver would be more beneficial.
Risposte (2)
Walter Roberson
il 9 Nov 2017
patternsearch is not "particularly susceptible to local minima": it is designed to have a reasonable chance of escaping local minima (but will certainly get stuck if they are deep enough.)
Since all of your inputs are integer (not a mix of integer and non-integer) you can use patternsearch using the techniques described at https://www.mathworks.com/matlabcentral/answers/285753-pattern-search-with-integer-decision-variable#answer_223391
0 Commenti
Vedere anche
Categorie
Scopri di più su Surrogate Optimization in Help Center e File Exchange
Prodotti
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!