multi cpu trainign using trainignOptions

I am trying to train a deep neural network network over a large dataset (several GB) and I want to use multiple-cpu's to train the network as I have a lot more cpu cores and RAM than gpus to train the network. I have explored the pair

'ExecutionEnvironment', 'parallel'

But my computer defaults to training on the GPU and skips the remaining workers that I have initialized. There seems to be no way to stop training the network on multi-cpu instead of GPU which is possible if I don't have a GPU. I figured I have to software disable the GPU using nvidia tools which is cumbersome, is there an alternative to do it on MATLAB programatically? As I might run a secon training on GPU using another instance of MATLAB. Any help will be appreciated!

Thanks,

Naren Nallapareddy

Answer 1

[Johannes Bergstrom]

Training a network on a single low-performance GPU is usually faster than training on multiple CPUs. So you will likely need a very large number of CPU cores to beat even a weak modern GPU.

If you still want to disable training using GPUs, then you can follow Options 2 in this answer: https://au.mathworks.com/matlabcentral/answers/408056-i-have-3-gpus-how-can-i-make-matlab-workers-see-only-one-of-my-gpus

and use setenv('CUDA_VISIBLE_DEVICES', ' ')

[Naren Nallapareddy]

Thanks, this solves my problem!

[Vincent Rougeau-Moss]

I have tried this and all the options proposed in (I have 3 GPUs. How can I make MATLAB workers see only one of my GPUs?) and none of these options work in my case. Parpool starts with several workers, then stop and start using the only GPU I have.

It seems that in my case (with Matlab 2018b on Ubuntu 18.04 LTS), the only possible option is to completely disable the possibility of using the GPU, for example by changing the driver of the GPU, but it is too complicated for a simple comparison test CPU / GPU.

Also, doing "gpuDevice([]);" doesn't work to disable the GPU.