minibatchqueue dispatch in background slow
1 visualizzazione (ultimi 30 giorni)
Mostra commenti meno recenti
Is there a reason using minibatchqueue with dispatchInBackground set to true would be slower than false?
I'm using a machine with A100 GPU (40Gig) and 50 cores
When I turn the dispatchInBackground option on, it's like the minibatchqueue preprocess function gets in an infinite loop prepping data. The parallel pool turns on, but it only a portion of the four bars on the parallel indicator ever turns green. If I turn the option off, it's slow but runs fine.
2 Commenti
Joss Knight
il 14 Nov 2022
Hard to answer without more information really, like some example code. It might be slower if you're accidentally using a remote cluster instead of a local one or if there is otherwise some problem for your pool workers reading the files. Or perhaps your data is in memory rather than on the filesystem, in which case you usually shouldn't use dispatchInBackground. If file access and preprocessing is trivial in terms of performance then you may find that the real bottleneck is communication between the pool and your client MATLAB. Or it could be you have preprocessing code that is very compute intensive and the workers are competing with each other and the client for resources.
Risposte (0)
Vedere anche
Categorie
Scopri di più su Parallel and Cloud in Help Center e File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!