out of memory in perms.m

why do I encounter "out of memory" error when using perms.m?how can I solve it?

Answer 1

[John D'Errico]

Get more memory, or solve smaller problems.

For a vector of length n, perms will create a result that is of size

factorial(n)*n*8

bytes. Does that not seem large?

psize = @(n) factorial(n).*n*8/(1024)^3;

psize computes the amount of memory used in GIGABYTES.

psize(10)
ans =
      0.27037

psize(11)
ans =
       3.2714

psize(15)
ans =
   1.4614e+05

so perms(15) will call for 146 terabytes of memory. Unless your computer is terribly well equipped, that may be excessive. Do you work for the NSA?

It is trivially easy to write a very simple line of code that will require more memory than held in all the computers in the world.

So the answer is, don't. Find a different way around your problem. Mathematics is not about being able to test the capabilities of any computer you will ever find. It is about how to avoid doing exactly that.

Answer 2

[James Tursa]

Probably because you used too large of a number. E.g.,

>> perms(1:3)
ans =
     3     2     1
     3     1     2
     2     3     1
     2     1     3
     1     2     3
     1     3     2
>> perms(1:30)
??? Maximum variable size allowed by the program is exceeded.

If you are trying to generate all possible permutations of something that will then be fed into a downstream algorithm, you will need to figure out another way to solve your problem.

Answer 3

[Jan]

The problem occurres, because the output is too large to be held in the available memory.

perms works with doubles internally. You can use FEX: VChooseKO to operate on e.g. UINT8 values, if this is possible for your problem:

VChooseKO(uint8(1):uint8(3), 3)

While perms creates an index matrix at first and the output in a second step, it requires a lot of temporary memory - if the output is of the type double or uint64, the same size as the output array, such that you need the double size of the output array as free RAM. VChooseKO creates the output directly and with the type of the input, such that e.g. uint8 needs an 8.th of the memory only. Together with the lack of the need for a temporary index matrix, this can be a factor of 16 in the memory consumption. But remember that the number of permutations grows rapidly with the input size - a factor of 16 might mean, that you can process only 1 or 2 additional elements before you get the Out Of Memory problems.

The output of VChooseK is ordered in the opposite direction compared to perms.

[sahar]

Dear Simon thank you for this answer. But still I can't solve the problem. I couldn't find any code for VchooseKO function to add to Matlab 2014. can you help me to find the code?

[Stephen23]

@sahar: look at Jan Simon's answer: the text "FEX: VChooseKO" is a link to another page. Click the link, and then click the big blue button labelled "Download". Unzip the zip file into your current directory. The MEX file will need to be compiled for your machine, you can read the instructions for more info.