Parallel and Cloud
Train deep networks on multiple GPUs, clusters, and clouds, using Parallel Computing Toolbox™. Scale up deep learning with multiple GPUs locally or on clusters, and train multiple networks interactively or in batch jobs. To learn about options, see Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud.
Functions
deep.gpu.deterministicAlgorithms | Set determinism of deep learning operations on the GPU to get reproducible results (Since R2024b) |
deep.gpu.fastAttentionAlgorithms | Disable fast attention algorithms used by deep learning operations on the GPU (Since R2026a) |
Topics
Parallel, GPU, and Cloud Fundamentals
- Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud
Explore options for deep learning with MATLAB® in parallel and using multiple GPUs, locally or in the cloud. - Deep Learning with MATLAB on Multiple GPUs
Speed up deep neural network training using multiple GPUs locally or in the cloud. - Run Experiments in Parallel
Run multiple simultaneous trials or one trial at a time on multiple workers. - Deep Learning with Big Data
Train deep neural networks using large amounts of data. - Resolve GPU Memory Issues
Troubleshoot errors when MATLAB cannot allocate the requested GPU memory.
Training in the Cloud
- Deep Learning in the Cloud
Access MATLAB in the cloud for deep learning. - Work with Deep Learning Data in the Cloud
Example workflows for uploading data to the cloud.
- Cloud AI Workflow Using MathWorks Cloud Center
Example workflows for training, importing data, and optimizing a deep neural network in the cloud using MathWorks Cloud Center.
- Cloud AI Workflow Using the Deep Learning Container
Example workflows for training, importing data, and optimizing a deep neural network in the cloud using the Deep Learning Container.
Custom Training Loops
- Run Custom Training Loops on a GPU and in Parallel
Speed up custom training loops by running on a GPU, in parallel using multiple GPUs, or on a cluster. - Train Network in Parallel with Custom Training Loop
This example shows how to set up a custom training loop to train a network in parallel.
Featured Examples
Train Network in the Cloud Using Automatic Parallel Support
Train a convolutional neural network using MATLAB® automatic support for parallel training.
Train Network Using Automatic Multi-GPU Support
Use multiple GPUs on your local machine for deep learning training using automatic parallel support.
Compare Classification Network Architectures Using Experiment
Train a deep learning network for classification using Experiment Manager.
Compare Dropout Probabilities and Filter Configurations for Image Regression Using Experiment
Train a deep learning network for regression using Experiment Manager.
Train Deep Learning Networks in Parallel
Run multiple deep learning experiments on your local machine. Using this example as a template, you can modify the network layers and training options to suit your specific application needs. You can use this approach with a single or multiple GPUs. If you have a single GPU, the networks train one after the other in the background. The approach in this example enables you to continue using MATLAB® while deep learning experiments are in progress.
Train Network Using Federated Learning
Train a network using federated learning. Federated learning is a technique that enables you to train a network in a distributed, decentralized way [1].
Send Deep Learning Batch Job to Cluster
Train a deep neural network on a cluster while you continue working in MATLAB® or when you close it.
Use batch to Train Multiple Deep Learning Networks
Train multiple networks in batch jobs using a cluster so that you can continue working or close MATLAB® during training.
Use parfor to Train Multiple Deep Learning Networks
Use a parfor loop to perform a parameter sweep on a training option.
Use parfeval to Train Multiple Deep Learning Networks
Use parfeval to perform a parameter sweep on the depth of the network architecture for a deep learning network and retrieve data during training.
