save -append problem: long duration to save an existing variable in a big mat-file

Hi all,

I have a very peculiar problem with the save command. I have a lang mat-file (ca. 300 MB, compressed, created with MATLAB > 7). Now I save an additional signal in the file using

save(filename, signalname, '-append');

If the variable signalname doesn't exist in the file the copy process is very fast. But, if the file contains already a variable signalname the whole process takes too long to end (some minutes!!!). If I try to append more variables that already exist in the file, it takes evern more!! Is this a bug? Am I doing something wrong?

Thanks a lot in advance for any advice!

Emmanuel

Answer 1

[Jan]

This seems reasonable. Overwriting an existing variable requires to decompress the file at first to remove the old value. The larger the file, the more time is required.

You see the same problem for other compressed files also: ZIP, compressed SSDs, or a large number of books packed in a cardboard box for.

Answer 2

[Diderot]

I got a similar problem. What's more, the new data file after several variables appended is much larger than the original, even though I only updated the previously existing variables with the same size instead of creating any new variables. This is really confusing. Would anyone share insights? Thanks!

[Jan]

Please do not append a new question to an existing thread. Note that you cannot accept an answer, if it is not your question and that your message is not an asnwer to the original question. Thanks.

Answer 3

[Pritesh Parmar]

I had the same issue with MATLAB's "save -append" with updating variables with large dataset. Try the following code: fastSaveUpdate - Efficiently update variables in MAT-file - File Exchange - MATLAB Central

Pritesh Parmar