gpuArray size limit is much less than the GPU maximum memory

When I work on using gpuArray to speed up my computation, I vectorized all my parallel element-wise operations and then obtained 2 big matrices as gpuArrays, to perform some basic multiplications/summations, etc. I thought this might be the better way to go compared to use the arrayFun. It did work well, and, to further speed up, I want to maximize the memory usage of the GPU. This means, instead of performing matrices operations several times, it will be better to perform bigger matrix operations only one time. Then I found matlab has some limit on the array size as intmax('int32'). However, this number is much less than the maximum GPU memory of my device. So I wonder if there is a way to overcome this.

[Walter Roberson]

As far as I know, indexing on NVIDIA is restricted to int32.

Answer 1

[Joss Knight]

See

https://uk.mathworks.com/matlabcentral/answers/2145604-gpu-array-max-dimensions-size-i-e-int32-of-2e9-not-same-as-other-cuda-enabled-languages-i-e?s_tid=srchtitle

[z223641s]

Thank you for the guideline. So I think at present I will just split the arrays within the int32 index limit. However, this then requires a repeated process for loading the splitted data one by one and then compute, although by theory I think it is feasible to do these repeated processes in parallel since the memory is enough. For the computation part I tried using the splitted data with arrayfun feature and did achieve to compute all the splitted data on GPU at the same time. However, I have trouble in loading the splitted data to GPU in parallel. I strated by using a for-loop to load the splitted parts one by one to GPU, and later I changed the for to par-for loop, but this seems not working.