Is is possible to run a batch job on an MJS Cloud Center cluster with SpmdEnabled set to false?
1 visualizzazione (ultimi 30 giorni)
Mostra commenti meno recenti
I'm running some parallel pool computations on an MJS cluster (created with Cloud Center) using the batch command. Used this help article to set that up: https://www.mathworks.com/help/parallel-computing/run-a-batch-job.html#bu62o45.
I'm running with a pool of several hundred workers using the 'Pool' argument to the batch command. Unfortunately, the entire job will fail if any of the workers crash, which happens quite frequently.
Searching online, I've found that setting SpmdEnabled to false when using the parpool command will allow the task to complete on the remaining workers. I'd like to set this flag, but can't seem to find a way to do it using the batch command. Is there another way to disable SPMD support but also use the batch command with a parallel pool to submit a job to a cloud cluster?
0 Commenti
Risposta accettata
Edric Ellis
il 4 Gen 2023
Unfortunately, this option is not supported at the moment for batch jobs. I realise it's probably rather a big change to your code, but you could use independent tasks using createJob and createTask.
4 Commenti
Edric Ellis
il 6 Gen 2023
I'm afraid you'd need to change your parfor loop into a series of independent tasks using createTask. (Likewise, you wouldn't be able to use parsim).
Più risposte (0)
Vedere anche
Categorie
Scopri di più su Get Started with Parallel Computing Toolbox in Help Center e File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!