How to taken values from big matrix ?

Hi, I've a many matrix inside a folder (about 10,000). Each matrix is 1500x1500. I wrote a code to open cyclically each file (with comand for), but now I want to create the 'history' for each cell. I try to explain better, I'd that for each for cycle, the each value inside a cell is stored at new matrix composed from 10,000x1, and write each value for each cell one below the other.

[Adam]

I'm not sure I understand what you mean. I assume by 'cell' you are referring to an element of a matrix, but you are saying you have 10,000 matrices each of size 1500x1500.

Unless you are only changing one of those 1500x1500 data values per matrix you will need a lot more than a 10,000x1 array to store a history for all elements of all matrices.

[Stefano Alberti]

I'd read each value of 1500x1500 matrix about 10,000, store it each value in a new file, then read a new matrix and put below in the same file the value for each element of matrix.

Example:

Matrix 1

Position 1,1 value 0.5

Matrix 2

Position 1,1 value 1.0

...

create a new file with there are in column, each values

0.5

1 ...

And do this for each matrix 1500x1500 elements.

It is possible ?

[Adam]

So you want to just pile up 10,000 1500x1500 matrices into a single 1500x1500x10000 array?

That would require ~20 GB of raw storage even if the data values are just uint8 type. It would be 80 GB if you use 'single' and twice as much again for doubles.

In terms of file space it is theoretically possible, but in terms of using the created file that is another matter entirely.

[Stefano Alberti]

It's a simple double. Furthmore, I can selected much less cells (about 100/200) and do that.

[Jan]

@Stefano: Please note, that the term "cell" means a cell in Matlab. I still do not understand the question. Are you talking about ASCII text files or are you able to use the more efficient binary format?

[Stefano Alberti]

Yes, it's ascii format.

[Jan]

So you want to import 10'000 files containing 1500x1500 matrices. Then you want to store all data in one file. Correct? How do you want to store the result? How do you want to write the bunch of matrices "one below the other"? As 15'000'000x1500 matrix? In wich order? Are you aware that such huge ASCII files are not useful, because ASCII-files are only though for the purpose, that they are readable by human. But nobody can read such a pile of values without getting mad. You are talking about files of 90 GB, when I assume 4 digits per value.

[Guillaume]

My understanding is that Stefano wants a file with 1500x1500 columns (2,250,000 columns!) and 10,000 lines.

While, yes, it is possible to create such a file, it's probably not a good idea. As has been pointed out, the file will be huge and impossible to import whole into memory.