Find and replace (overwriting) to the middle of an existing binary file

I'm trying to open a binary file for replacing without erasing all the content. For example I can read my binary file and convert it to the struct as follows, what I want to do is that to replace data section with my own data:

    if ~exist('filename', 'var')
    error(['''' filename ''' does not exist']); end
    
    % Open file
    fclose all;
    fid = fopen(filename);
    if fid < 3; error 'Error while opening file'; end
    
    % Read one property at the time
    while ~feof(fid)        
        % Read field name (keyword) and array size
        keyword = deblank(fread(fid, 8, 'uint8=>char')');
        num = fread(fid, 1, 'int32=>double', 0, 'b');
        
        % Read and interpret data type
        conv = 'single=>double';
        wsize = 4;
        
        % Read data array, which may be split into several consecutive arrays
        data = [];
        remnum = num;
        while remnum > 0
            % Read array size
            buflen = fread(fid, 1, 'int32=>double', 0, 'b');
            bufnum = buflen / wsize;
            
            % Read data and append to array
            %%% REPLACE DATA WITH NEW DATA %%%%
            data = [data; fread(fid, bufnum, conv, 0, 'b')]; %<<========HERE========
            
            remnum = remnum - bufnum;
        end
    end
        
    fclose(fid);

Answer 1

[Jeremy Hughes]

fopen(filename,'a') % append mode

https://www.mathworks.com/help/matlab/ref/fopen.html#btrnibn-1-permission

Then use fseek to go to where you want to overwrite.

[Behzad Hosseinzadeh]

Does fwrite function overwrite by using fseek? because fseek moves to specified position in file. I don't know how to overwrite the specific value.

[Walter Roberson]

For overwriting, you should open the file with 'r+' not 'a'

fwrite() does not exactly overwrite by using fseek(). Instead, there are several cases:

• if the open permissions was 'r' then writing is an error
• if the open permissions were 'a' or 'a+' or 'A', then each fwrite() or fprintf() internally does an fseek() to end of file before doing the writing, and the writing is followed by an internal fflush() system call except 'A' does not fflush(). (But see notes below)
• if the open permission was 'r+' or 'w' or 'w+' or 'W', then if the current seek position is beyond end of file, each fwrite() or fprintf() does an fseek() to end of file before doing the writing, and the writing is followed by an internal fflush() system call except 'W' does not fflush(). THIS IS NOT POSIX COMPLIANT; see notes
• if the open permission was 'r+' or 'w' or 'w+' or 'W', then the current seek permission is not beyond end of file, each fwrite() or fprintf() scribbles into the middle of the file, and the writing is followed by an internal fflush() systme call except 'W' does not fflush()

So you never want to use 'a' or 'a+' or 'A' with a file you want to change the middle of, because those modes always write to end of file no matter where you were positioned.

'w' and 'W' only permit seeking and writing, and any attempt to read with them will return empty. But they do permit seeking to places already written in order to write different information.

'r+' and 'w+' permit reading and writing, including seeking to any position to read or write there. The difference between them is that 'w+' removes any existing file content first.

Note:

1. POSIX says that if you seek past end of file and write there, that the operating system is permitted to fill in zeros from the old end of file to the seek position, but it is also permitted to leave "holes" in the file that read back as zero but do not necessarily occupy any physical storage. So as far as POSIX is concerned, it is completely correct (and expected procedure) to extend a file by seeking to the desired position and writing something (even just one byte) there. However, MATLAB does not comply with this; instead if you seek past end of file and attempt to write, the data will be written at end of file with no gap.
2. I indicated earlier that the append modes did an internal fseek() to end of file each time that writing is done. This is not exactly true, in that the MATLAB level of code does not do the fseek() [or at least is not required to do the fseek()]. Instead, any seeking for this is done at the level of the I/O driver. The difference is that if MATLAB does the fseek() then the file system would be required to flush any existing buffered output, but if the file system is the one doing any internal seeking, then it is permitted to notice that it is already at end of file (if it is) and refrain from flushing at that point.

[Jeremy Hughes]

Yes, Walter is correct. I didn't read carefully enough. 'r+' mode allows you to move to a position in the file using fseek. fwrite should then write into the file at that position

[Behzad Hosseinzadeh]

@Walter Roberson thanks for your great explanation. It works for me now.