out of memory error

Hello

I want to read a database with over a million arrays, consequently matlab shows an error "Out of memory". I want to read a matrix x(1500000,1) which is some kind of Random matrix,and then I want to find matrix V which contains information about a network of variables in matrix x. That is, if any variable see the other element,the V matrix line and column will change to 1,else it will change to 0. (x is a time series, and V becomes a symmetric matrix with arrays of 1 and 0). after that I want to compute the eigenvalues and eigenvectors of V which was created from x .

The source of the problem lies on reading the x matrix. How I can read a matrix which contains too long elements such as x or matrix like x(1500000,1500000)and then run the code without the mentioned error?

This is my code sample

N =1500000;

y=data;      % data is matrix "x" , in this line I faced with the problem

V=zeros(N,N);

      for i=1:N
          V(i,i)=1; 
          if i>1
           V(i,i-1)=1;
          end
   % I  find a line between every point of the series,and then move along the siries point. if we can find a line between two point,they will be able  to see each other and the respected array will change to 1,and if between any two points,another one exist so the points are not able to see each other and the respected array turns to 0. y=m*x + h is the line equation
          if i<N
          V(i,i+1)=1;
          end

          if i < N-1
              m = y(i+1)-(y(i));
              h=(y(i))-m*x(i);
              for j=i+2:N
                u= m* x(j)+h;
                if y(j) > u
                    V(i,j)=1;
                    V(j,i)=1;
                    m= (y(j)-(y(i)))/(x(j)-x(i));
                    h=(y(i))-m*x(i);
                end

              end

          end

THANK YOU.

Answer 1

[Jan]

A full matrix of size 1500000x1500000 and type double requires 18 TB. If most of the elements are zero, you can create such a huge array in sparse format. See http://www.mathworks.com/help/techdoc/math/f6-8856.html and http://www.mathworks.com/help/techdoc/math/f6-8856.html.

Answer 2

[Farhadtah]

Hello I knew tha! let me clarify my concerns. If I use sparse code,matlab will ignore the zero variables! But these varibles are vital for my computation. does matlab cosider the zeros in caculation? for example we hav 3*3 matrix with 2 zeros line 2 ! these will affect the eigenvalues!

[Walter Roberson]

MATLAB *will* consider the zeros in its calculations.

Warning: computing the eigenvalues of a 18TB sparse matrix might overflow your physical memory.