Why does my long simulation slow down over time?

I am running long simulations involving a double for-loop with each approx. 20 intervals, and each internal process taking anywhere between an hour and a day. When initiating this simulation it will start off at a rapid pace, and will slow down at a rate which appears to be correlated to the decreasing availability of RAM on the computer. This, however, puzzles me as I have set it up so that nothing, apart from a handful of matrices, is stored in between the individual processes and all matrices are assigned before use so nothing grows without being pre-assigned.

Does anyone know of any tricks I can use? Is this just to be expected for long matlab runs in general? using tic/toc or profile only emphasizes that the whole thing slows down.

Thanks in advance

[Star Strider]

Are you running any solvers in your loops (ode, optimization, etc.)?

[Jakob Sievers]

No. It is basic math and matrix-operations only. It is an iterative inverse routine that improves on a solution to a problem until it is within desired statistical bounds but nothing beyond basic matrix operations is going on.

[Star Strider]

If you're taking matrix inverses (please do not use inv) or doing matrix division, how are you doing it? I suggest mldivide.

[Jakob Sievers]

I have no inverse operations taking place either. Basically the iterative process just consists of a number of matrix multiplications. Whenever an iterative loop is finished, everything, save for a few small matrices, should be re-assigned to empty nan-matrices prior to initiation of the next iterative loop. This is why I wonder about the gradual decrease in speed. Nothing (of significance) is being kept between each iterative process.

[Walter Roberson]

Which MATLAB version are you using, and which operating system? 32 or 64 bit ?

[Star Strider]

The only other thing I can suggest is that you clear the matrices you no longer need at the end of an iteration or alternatively assign them to the empty matrix ( [] ), and re-allocate them at the beginning of the next iteration.

[Jakob Sievers]

@Walter: I'm running 64bit R2011b on a Win7 quadcore 2.20Ghz laptop with 8GB of RAM.

@StarStrider: I am re-allocating everything at the beginning of each iteration.

[Jakob Sievers]

I think the comment by "Image Analyst" just led me straight to the actual problem. I will check this out. Thanks guys.

Answer 1

[Image Analyst]

Jakob, do any of your operations display an image in an axes? If so, all of those images "stack up" in the axes control and sap memory. They're all in there even though you only see the top one displayed. Be sure to use "cla" before loading a new image in, and it will be faster, otherwise it will get slower and slower as more and more images get displayed.

[Jakob Sievers]

I am not displaying anything, other than a waitbar which quite frequently jumps back and forth rather than going from beginning to the end. This is intentional because the routine is set to do N iterations and then evaluate whether the statistical requirement has been reached. If not, it allocates X new spaces in the various matrices and runs until N+X and repeats this cycle until the requirement is met. It just occurred to me though that this right here may actually just be the problem. I am not adding that many new points each time, so for most runs (say of a few hours duration) it wont give my system any problems, but for some of the longer runs (say a day) the routine ends up doing it quite a lot, thus continuously increasing the bulk of information the system has to throw around.

I think we have a winner :) I will have to check this and see if it will help if I save information in intervals and clearing, rather than keeping everything active the whole time.