Array stored on GPU
gpuArray object represents an array stored in GPU memory. A
large number of functions in MATLAB® and in other toolboxes support
gpuArray objects, allowing you to run your code on GPUs
with minimal changes to the code. To work with
gpuArray objects, use any
gpuArray-enabled MATLAB function such as
mldivide. To find a full list of
gpuArray-enabled functions in
MATLAB and in other toolboxes, see GPU-supported functions. For more information, see Run MATLAB Functions on a GPU.
If you want to retrieve the array from the GPU, for example when using a function that
does not support
gpuArray objects, use the
You can load MAT files containing
gpuArray data as in-memory arrays when a GPU is not
gpuArray object loaded without a GPU is limited and you cannot use it for
computations. To use a
gpuArray object loaded without a GPU, retrieve the contents using
gpuArray to convert an array in the MATLAB workspace into a
gpuArray object. Some MATLAB functions also allow you to create
gpuArray objects directly.
For more information, see Establish Arrays on a GPU.
X — Array
numeric array | logical array
Array to transfer to the GPU, specified as a numeric or logical array. The GPU
device must have sufficient free memory to store the data. If
is already a
You can also transfer sparse arrays to the GPU.
supports only sparse arrays of double-precision.
G = gpuArray(magic(3));
Complex Number Support: Yes
There are several methods for examining the characteristics of a
gpuArray object. Most behave like the MATLAB functions of the same name.
Several MATLAB toolboxes include functions with
gpuArray support. To view
lists of all functions in these toolboxes that support
gpuArray objects, use
the links in the following table. Functions in the lists with information indicators have
limitations or usage notes specific to running the function on a GPU. You can check the usage
notes and limitations in the Extended Capabilities section of the function reference page. For
information about updates to individual
gpuArray-enabled functions, see the
|Toolbox Name||List of Functions with ||GPU-Specific Documentation|
|Statistics and Machine Learning Toolbox™||Functions with
||Analyze and Model Data on GPU (Statistics and Machine Learning Toolbox)|
|Image Processing Toolbox™||Functions with
||GPU Computing (Image Processing Toolbox)|
|Deep Learning Toolbox™|
*(see also Deep Learning with GPUs)
Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud (Deep Learning Toolbox)
Deep Learning with MATLAB on Multiple GPUs (Deep Learning Toolbox)
|Computer Vision Toolbox™||Functions with
||GPU Code Generation and Acceleration (Computer Vision Toolbox)|
|Communications Toolbox™||Functions with
||Code Generation and Acceleration Support (Communications Toolbox)|
|Signal Processing Toolbox™||Functions with
||Code Generation and GPU Support (Signal Processing Toolbox)|
|Audio Toolbox™||Functions with
||Code Generation and GPU Support (Audio Toolbox)|
|Wavelet Toolbox™||Functions with
||Code Generation and GPU Support (Wavelet Toolbox)|
|Curve Fitting Toolbox™||Functions with
For a list of functions with
gpuArray support in all
MathWorks® products, see
gpuArray-supported functions. Alternatively, you can
filter by product. On the Help bar, click Functions.
In the function list, browse the left pane to select a product, for example, MATLAB. At the bottom of the left pane, select GPU Arrays. If you
select a product that does not have
gpuArray-enabled functions, then the
GPU Arrays filter is not available.
Transfer Data to and from the GPU
To transfer data from the CPU to the GPU, use the
Create an array
X = [1,2,3];
X to the GPU.
G = gpuArray(X);
Check that the data is on the GPU.
ans = logical 1
Calculate the element-wise square of the array
GSq = G.^2;
Transfer the result
GSq back to the CPU.
XSq = gather(GSq)
XSq = 1×3 1 4 9
Check that the data is not on the GPU.
ans = logical 0
Create Data on the GPU Directly
You can create data directly on the GPU directly by using some MATLAB functions and specifying the option "
Create an array of random numbers directly on the GPU.
G = rand(1,3,"gpuArray")
G = 0.3640 0.5421 0.6543
Check that the output is stored on the GPU.
ans = logical 1
Use MATLAB Functions with the GPU
This example shows how to use
gpuArray-enabled MATLAB functions to operate with
gpuArray objects. You can check the properties of your GPU using the
ans = CUDADevice with properties: Name: 'NVIDIA RTX A5000' Index: 1 ComputeCapability: '8.6' SupportsDouble: 1 DriverVersion: 11.6000 ToolkitVersion: 11.2000 MaxThreadsPerBlock: 1024 MaxShmemPerBlock: 49152 (49.15 KB) MaxThreadBlockSize: [1024 1024 64] MaxGridSize: [2.1475e+09 65535 65535] SIMDWidth: 32 TotalMemory: 25553076224 (25.55 GB) AvailableMemory: 25153765376 (25.15 GB) MultiprocessorCount: 64 ClockRateKHz: 1695000 ComputeMode: 'Default' GPUOverlapsTransfers: 1 KernelExecutionTimeout: 0 CanMapHostMemory: 1 DeviceSupported: 1 DeviceAvailable: 1 DeviceSelected: 1
Create a row vector that repeats values from -15 to 15. To transfer it to the GPU and create a
gpuArray object, use the
X = [-15:15 0 -15:15 0 -15:15]; gpuX = gpuArray(X); whos gpuX
Name Size Bytes Class Attributes gpuX 1x95 760 gpuArray
To operate with
gpuArray objects, use any
gpuArray-enabled MATLAB function. MATLAB automatically runs calculations on the GPU. For more information, see Run MATLAB Functions on a GPU. For example, use
gpuE = expm(diag(gpuX,-1)) * expm(diag(gpuX,1)); gpuM = mod(round(abs(gpuE)),2); gpuF = gpuM + fliplr(gpuM);
Plot the results.
If you need to transfer the data back from the GPU, use
gather. Transferring data back to the CPU can be costly, and is generally not necessary unless you need to use your result with functions that do not support
result = gather(gpuF); whos result
Name Size Bytes Class Attributes result 96x96 73728 double
In general, running code on the CPU and the GPU can produce different results due to numerical precision and algorithmic differences between the GPU and CPU. Answers from the CPU and GPU are both equally valid floating point approximations to the true analytical result, having been subjected to different roundoff behavior during computation. In this example, the results are integers and
round eliminates the roundoff errors.
Perform Monte Carlo Integration Using
This example shows how to use MATLAB functions and operators with
gpuArray objects to compute the integral of a function by using the Monte Carlo integration method.
Define the number of points to sample. Sample points in the domain of the function, the interval
[-1,1] in both
x and y coordinates, by creating random points with the
rand function. To create a random array directly on the GPU, use the
rand function and specify "
gpuArray". For more information, see Establish Arrays on a GPU.
n = 1e6; x = 2*rand(n,1,"gpuArray")-1; y = 2*rand(n,1,"gpuArray")-1;
Define the function to integrate, and use the Monte Carlo integration formula on it. This function approximates the value of by sampling points within the unit circle. Because the code uses
gpuArray-enabled functions and operators on
gpuArray objects, the computations automatically run on the GPU. You can perform binary operations such as element-wise multiplication using the same syntax that you use for MATLAB arrays. For more information about
gpuArray-enabled functions, see Run MATLAB Functions on a GPU.
f = x.^2 + y.^2 <= 1; result = 4*1/n*f*ones(n,1,"gpuArray")
result = 3.1403
If you need better performance, or if a function is not available on the GPU,
gpuArraysupports the following options:
To precompile and run purely element-wise code on
gpuArrayobjects, use the
To run C++ code containing CUDA® device code or library calls, use a MEX function. For more information, see Run MEX-Functions Containing CUDA Code.
To run existing GPU kernels written in CUDA C++, use the MATLAB CUDAKernel interface. For more information, see Run CUDA or PTX Code on GPU.
To generate CUDA code from MATLAB code, use GPU Coder™. For more information, see Get Started with GPU Coder (GPU Coder).
You can control the random number stream on the GPU using
None of the following can exceed
The number of elements of a dense array.
The number of nonzero elements of a sparse array.
The size in any given dimension. For example,
zeros(0,3e9,"gpuArray")is not allowed.
You can also create a
gpuArray object using some MATLAB functions by specifying a
gpuArray output. The following
table lists the MATLAB functions that enable you to create
directly. For more information, see the Extended Capabilities section of the function
Run code in the background using MATLAB®
backgroundPool or accelerate code with Parallel Computing Toolbox™
This function fully supports thread-based environments. For more information, see Run MATLAB Functions in Thread-Based Environment.
Introduced in R2010b