Force CPU Parallel pool

My GPU is a GTX 970 with only 4 Gb of availble memory. I would like to force training of my UNET segmentation network on the CPU 16 core ryzen 5950x with availble system memory instead.

How can I force a multicore/multithreaded parallell pool on the CPU. By default matlab selects GPU and experiences memory errors.

Is there a way to "hide" the GPU from matlab?

options = trainingOptions('adam', ...
    'InitialLearnRate',2e-4, ...
    'LearnRateSchedule','piecewise', ...
    'LearnRateDropFactor',0.85, ...
    'LearnRateDropPeriod', 3, ... 
    'MaxEpochs',60, ... 
    'MiniBatchSize',20, ...
    'Shuffle', 'every-epoch', ...
    'VerboseFrequency', 5, ...
    'ValidationData', Validationds, ... 
     'ValidationFrequency', 30, ...
    'ExecutionEnvironment', 'parallel', ...
     'Plots','training-progress');

Answer 1

[Joss Knight]

You should use 'ExecutionEnvironment','cpu' for training on your local machine. This is multithreaded and will use all your cores. Parallel training on CPU is only useful for multinode clusters.

In practice you will likely find that your 16 core CPU is still slower than training on your GTX 970 with the MiniBatchSize reduced so that you can fit into its memory - worth checking though.

Answer 2

[Edric Ellis]

There's no direct way to specify this using trainingOptions, but what you can do is disable the GPUs on the workers by running this command in your desktop MATLAB before creating the parallel pool:

setenv('CUDA_VISIBLE_DEVICES', '')

You can then check that this has worked by running

spmd
    gpuDeviceCount
end

This should return 0 on each worker.