Distributed Arrays
Parallel Computing Toolbox™ supports distributed arrays to partition large arrays across
multiple MATLAB® workers. You operate on the entire array as a single entity,
however, workers operate only on their part of the array, and automatically
transfer data between themselves when necessary. Simultaneous execution is
supported by the single program multiple data (spmd)
language construct to facilitate communication between workers. Use
distributed-enabled matrix operations and functions to work directly with these
arrays without further modification. You can use distributed arrays in
Parallel Computing Toolbox to run big data applications using the combined memory of your
cluster.
Functions
Classes
Topics
Distributed Arrays
- Create and Use Distributed Arrays
When your data array is too big to fit into the memory of a single machine, you can create adistributedarray. - Run MATLAB Functions with Distributed Arrays
MATLAB functions that operate on distributed arrays. - Distributing Arrays to Parallel Workers
Usedatastoreordistributedto create distributed arrays and partition the data among your workers. - Nondistributed Versus Distributed Arrays
Describes the various types of arrays used in communicating jobs. - Use Distributed Arrays to Solve Systems of Linear Equations with Direct Methods
This example shows how to solve a system of linear equations of the form in parallel with a direct method using distributed arrays. - Use Distributed Arrays to Solve Systems of Linear Equations with Iterative Methods
For large-scale mathematical computations, iterative methods can be more efficient than direct methods. - Resolve Error: Client Lost Connection to Worker
Troubleshoot lost connection to worker error when running code on parallel pools. (Since R2026a)
Single Program Multiple Data (spmd)
- Choose Between spmd, parfor, and parfeval
Compare and contrastspmdagainst other parallel computing functionality such asparforandparfeval. - Run Single Programs on Multiple Data Sets
Usespmdstatements to run the same code on multiple datasets and control codistributed arrays. - Working with Codistributed Arrays
Describes how to use codistributed arrays for calculation. - Access Worker Variables with Composites
Composite objects in the MATLAB client session let you directly access data values on the workers. - Use Message Passing to Estimate Pi
This example shows the basics of working with message passing inspmdstatements, similar to collective operations in the Message Passing Interface (MPI) specification. - Train Network in Parallel with Custom Training Loop (Deep Learning Toolbox)
This example shows how to set up a custom training loop to train a network in parallel. - Looping over a Distributed Range (for-drange)
Describes how to program afor-loop with codistributed arrays. - Use spmdReduce to Achieve MPI_Allreduce Functionality
This example shows how to perform reduction operations, such as additions, finding products, minimums, or maximums, across all workers in a parallel pool. - Communicate Between Workers in SPMD Computations
Exchange data between workers usingspmdSendandspmdReceive, similar to point-to-point communication in the Message Passing Interface (MPI) standard. - Fit Distributed Logistic Regression Using drange
This example shows how to use distributed arrays andfor-drange-loops to implement logistic regression on large data. - Compute Parallel Prefix Scans with SPMD Computations
Implement a reusable, scalable, SPMD based prefix scan using a parallel workers. - Work with Remote Data
Work with remote data in Amazon S3™, Azure® Blob Storage, or HDFS™.
Related Information
Featured Examples
Use Distributed Arrays to Solve Systems of Linear Equations with Direct Methods
Solve a system of linear equations of the form Ax=b in parallel with a direct method using distributed arrays.
Use Distributed Arrays to Solve Systems of Linear Equations with Iterative Methods
For large-scale mathematical computations, iterative methods can be more efficient than direct methods. This example shows how you can solve systems of linear equations of the form Ax=b in parallel using distributed arrays with iterative methods.
Solve Differential Equation Using Multigrid Preconditioner on Distributed Discretization
Solve Poisson's equation using a preconditioned iterative solver and distributed arrays. By using distributed arrays, you can scale up the calculation using the memory of a cluster of machines, not just the memory of a single machine. You can scale up without changing your code.
Profiling Explicit Parallel Communication
Profile explicit communication to the nearest neighbor worker. It illustrates the use of spmdSend, spmdReceive, and spmdSendReceive, showing both the slow (incorrect) and the fast (optimal) way of implementing this algorithm. The problem is explored using the parallel profiler. For getting started with parallel profiling, see Profiling Parallel Code.
Profiling Load Unbalanced Codistributed Arrays
Profile the implicit communication that occurs when using an unevenly distributed array. For getting started with parallel profiling, see Profiling Parallel Code.
Resource Contention in Task Parallel Problems
Why it is difficult to give a concrete answer to the question "How will my parallel code perform on my multicore machine or on my cluster?"
