"diff" function doesn't work properly with small numbers

For some reason when difference between n and n+1 is too small diff function assumes the solution is 0.

There are +-290 data points on the plot, The precision is 10^(-10), As far as i know Matlab works on 16 or 32 digits so it shouldn't be a problem.

Technically on the plot there should be on no constants, Just increase and decrease of value.

Pomiary=cisnienie300920151701average300
Czas = Pomiary{:, 4};
Temperatura = Pomiary{:, 5}; 
CzasDMY= Czas / 86400  + datenum(1970, 1, 1); 
y = Temperatura;
x = CzasDMY;
ydiff=diff(y,1); 
wieksze = (ydiff > 0);
mniejsze = (ydiff < 0);
gora = y;
dol = y;
gora(~wieksze) = NaN;
dol(~mniejsze) = NaN;

plot(x,y,'b',x, gora, 'r', x, dol, 'g');

grid on;
xlim tight;
xlim("auto");
ylim("auto");
legend("Constant", "Increasing", "Decreasing");
legend("Position", [0.15754,0.1468,0.20438,0.12165]);

[Fangjun Jiang]

If you provide a few example values for y, there might be an reasonalbe explaination for this.

[Sylwester]

36 1023.08766260000
37 1023.03861350000
38 1023.01522350000
39 1023.01522350000
40 1022.96080630000

Used NOT operator on increased and decreased arrays to find places where values in the row were the same, I'm gonna check if i should just move x arrays by +1 or -1.

Update, moving arrays by +-1 didn't work

[dpb]

The reasonable explanation for this is that the difference in temperatures for some cases is less than the resolution of a double.

There are several possibilities of whty this might be, starting with measuring temperature to that precision would be unrealistic. If the vaules were calculated, it's possible they were then written to another text file format first that only had limited precision.

Without the actual input data, it's not possible to say from whence the observed results came, but the problem is not in the diff() function, but in the expectations for it given the actual inputs.

Answer 1

[Fangjun Jiang]

The data value and results make sense. There is no problem using diff() to process your data based on your example data.

%%
format long
y=[36 1023.08766260000
    37 1023.03861350000
    38 1023.01522350000
    39 1023.01522350000
    40 1022.96080630000]

y = 5×2

1.0e+03 * 0.036000000000000 1.023087662600000 0.037000000000000 1.023038613500000 0.038000000000000 1.023015223500000 0.039000000000000 1.023015223500000 0.040000000000000 1.022960806300000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

ydiff=diff(y,1)

ydiff = 4×2

1.000000000000000 -0.049049100000047 1.000000000000000 -0.023389999999949 1.000000000000000 0 1.000000000000000 -0.054417200000103

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

wieksze = (ydiff > 0)

wieksze = 4×2 logical array
   1   0
   1   0
   1   0
   1   0

mniejsze = (ydiff < 0)

mniejsze = 4×2 logical array
   0   1
   0   1
   0   0
   0   1

By default, MATLAB uses 64 bits floating-point data to represent a numeric value.

At around value 1023, its relative accuracy is 1e-13, sufficient to represent your data precision 10e-10.

The problem you observed comes from your raw data. Note that y(3,2) and y(4,2) are exactly the same by visual observation.

eps(1023)

ans = 
     1.136868377216160e-13

Check the document for eps(). You will understand the issue better.

doc eps

[Sylwester]

They are meant to be the same, The issue is that for some reason function for marking if value increased/decreased has holes in it and skips points unless difference is high enough despite precision being ok.

I tried using eps() function on "diff" and it failed.

.

Also i changed dataset for this task from working on temperature to pressure so the values should make sense despite variable being named "temperatura"

[Fangjun Jiang]

The length of diff() output is 1 smaller than its input length. Your code didn't seem to consider this.

diff(1:3)

ans = 1×2

1 1

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

Answer 2

[dpb]

X=[
36 1023.08766260000
37 1023.03861350000
38 1023.01522350000
39 1023.01522350000
40 1022.96080630000];
diff(X)

ans = 4×2

1.0000 -0.0490 1.0000 -0.0234 1.0000 0 1.0000 -0.0544

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

As hypothesized above, some of the temperature values are identical owing to the apparent rounding to seven (7) decimal digits.

You would have to have at least one more decimal place in the above between the 3rd and 4th data values in order for the difference to not be identically zero.

If you're transferring data from one place to another, to avoid this don't use text files but save the whole internal precision by using .mat files or binary formatted transfer if from some external source. Besides being able to retain full precision (note that precision does not necessarily imply accuracy), it's much more efficient in speed and memory/disk space.