Multiple combinations of huge data set

1 view (last 30 days)
Hello I'm following this post:
The think is I need to do the same with the following this range of data
elements = {400:499,500:599,600:699,700:900}; %cell array with N vectors to combine
combinations = cell(1, numel(elements)); %set up the varargout result
[combinations{:}] = ndgrid(elements{:});
combinations = cellfun(@(x) x(:), combinations,'uniformoutput',false); %there may be a better way to do this
result = [combinations{:}]; % NumberOfCombinations by N matrix. Each row is unique.
When I run the code you posted just changing the values to those I need, I reach a limit of 201000000 x 4, the last combination calculated is 403,513,620, 729.
How can I modify the code or to get all calculations, last one should be 499,599,699,799,900?
Even if I need to create more than one variable file to save the calculations will be good.
Thank you.

Answers (1)

James Browne
James Browne on 15 Jun 2019
Edited: James Browne on 15 Jun 2019
I would think you are maxing out your memory allocation, considering you have 201 million rows in your result...If you can increase your memory allocation for MATLAB, that might fix it, otherwise, you have to implement the algorithm for creating all combinations, manually, and then intermittently save results to a data file so that you do not max out your memory. There may be another solution but those are the options that would pop into my head if I were trying to solve this problem~

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by