Efficient handeling of large matrix

Hello, I would like to efficiently handle a large matrix which consists of a block of matrices, which are the same along the diagonals. In order to save memory I would now, if there is an efficient way to handle this matrix. Saving as a sparse matrix does not help... Thanks a lot in advance

A =

   1.0e-04 *

    0.4059    0.0125         0         0         0         0         0         0
    0.0125    0.4059         0         0         0         0         0         0
    0.0845    0.0208    0.4059    0.0125         0         0         0         0
    0.0208    0.0845    0.0125    0.4059         0         0         0         0
    0.0425    0.0170    0.0845    0.0208    0.4059    0.0125         0         0
    0.0170    0.0425    0.0208    0.0845    0.0125    0.4059         0         0
    0.0267    0.0135    0.0425    0.0170    0.0845    0.0208    0.4059    0.0125
    0.0135    0.0267    0.0170    0.0425    0.0208    0.0845    0.0125    0.4059

[Thorsten]

What operations do you want to run on this matrix?

[M.K.123]

I want to use it in an linear equation system and solve it using the backslash operator or something comparable.

Answer 1

[Image Analyst]

How big is big? Like hundreds of megabytes or elements? Gigabytes? Have you looked at memmapfile()? I haven't used it but I know it's meant for dealing with gigantic arrays. For more info see this link or call your local friendly MATLAB tech support consultant.

[M.K.123]

It will be around 5*10^5 elements and I want to use it a linear equation system, which I will solve with the backslash operator or something similar. If I understood the mmemapfile() correctly, with this method I can only access the matrix faster but later for solving I have to use the original matrix again?

[Image Analyst]

I don't think it's all that huge - fewer elements than a garden variety digital image. So you have around 8 variables, and like about 60 thousand observations and want to fit a model to them? Have you tried it? How long does it take?

[M.K.123]

If I solve my system using the backslash operator and a matrix size of 27600X27600 elements ,it works fine and the solver is finished in about 120seconds (CPU 4x3.40GHz, 16GB RAM). If I increase now the number of elements slightly to e.g. 35000x35000 than it does not solve the problem anymore.

[Image Analyst]

OK, that's 761 million elements or 6 gigabytes - quite a bit larger than you first said. I don't know why you'd have a linear system with 27600 variables. Either you or I, or both, don't have a good concept of what you're trying to solve for and whether you even need a matrix this big. What does this matrix represent?

[M.K.123]

Ok I changed the way the system is solved and now don't solve the whole system at once but line by line, which is considerably faster. But thank you anyway for your great help!!

[Bibigul]

@ M.K123, I am having the same problem , i am dealing with a matrix size of more than 35000x35000 . can you please help me how to solve this line by line?

thanks in advance.

kind Regards Bibigul