Find mean of steps in broken signal

Hi,

I have a partially broken signal with NaNs and 5 steps as indicated below.

How can I find the mean level of each of the 5 steps?

The signal is attached.

The signal looks like the following:

Thanks.

Answer 1

[Star Strider]

One approach —

LD = load('y.mat');
y = LD.y;
x = 0:numel(y)-1;

Lv = islocalmax(y>25, 'FlatSelection','all');

start = strfind(Lv, [0 1]);
stop = strfind(Lv, [1 0]);

for k = 1:numel(start)
    ymean(k,:) = mean(y(start(k) : stop(k)), 'omitnan');
    xmean(k,:) = mean(x(start(k) : stop(k)), 'omitnan');
end

Flat_Means = table(xmean, ymean, 'VariableNames',{'X Centre','Y Mean'})

Flat_Means = 9×2 table
    X Centre    Y Mean
    ________    ______

      1618      28.811
      1651      30.648
      1691      26.499
      1725      28.549
      1789      35.506
      1828      33.727
      1841      33.103
      1876      33.305
      1939      39.992

figure
plot(x, y)
hold on
plot(x(Lv), y(Lv), '.r')
hold off
grid
ylim([min(ylim) 70])
text(xmean, ymean, compose('\\mu = %.2f \\rightarrow',ymean), 'Horiz','right', 'Vert','middle', 'Rotation',-80)

.

[Konvictus177]

Thanks, this looks very promosing but your method above finds more than 5 flat regions.

I want the code to find exactly 5 flat regions as indicated in my image above where the signal is mostly flat.

Is there any smart way to get there from your code above?

I am thinking about looking at the length and difference of consecutive points of a flat region. If its shorter than say x points than its not considered flat region, if its longer than x points than its a flat region. I think this works:

min_contiguous_samples = 30; % select regions only if they are at least this length
ind = length>min_contiguous_samples;
start2 = start(ind);
stop2 = stop(ind);

But I still have another problem. Let's say my signal looks a bit worse like the one below where the flat regions are not so proiment. The code would not find the two regions I highlighted. How can I adjust the code to find these regions where the signal is mostly flat? I attached the signal.

[Star Strider]

This version uses the median value of the segment lengths to select the segments to be analysed.

LD1 = load('y.mat');
LD2 = load('y_bad.mat');
y{1} = LD1.y;
y{2} = LD2.y;
x{1} = 0:numel(y{1})-1;
x{2} = 0:numel(y{2})-1;

for k1 = 1:numel(y)
    Lv = islocalmax(y{k1}>25, 'FlatSelection','all');
    
    start = strfind(Lv, [0 1]);
    stop = strfind(Lv, [1 0]);
    ssmtx = [start; stop]
    
    sslen = diff(ssmtx);                                        % Segment Lengths
    % sslen_sts = [min(sslen) median(sslen) max(sslen)]           % Statistics
    Lvss = sslen >= 0.9*median(sslen)                           % Select Columns Of 'ssmtx' Using The 'median' Of The Length Values As The Criterion
    ssmtx = ssmtx(:,Lvss)                                       % Selected 'ssmtx'
    
    xmean = [];
    ymean = [];
    for k2 = 1:size(ssmtx,2)
        ymean(k2,k1) = mean(y{k1}(ssmtx(1,k2) : ssmtx(2,k2)), 'omitnan');
        xmean(k2,k1) = mean(x{k1}(ssmtx(1,k2) : ssmtx(2,k2)), 'omitnan');
    end
    
    Flat_Means = table(xmean(:,k1), ymean(:,k1), 'VariableNames',{'X Centre','Y Mean'})
    
    figure
    plot(x{k1}, y{k1})
    hold on
    plot(x{k1}(Lv), y{k1}(Lv), '.r')
    hold off
    grid
    xlim([x{k1}(find(Lv,1)-100) max(xlim)])                          % 'Zoom' X-Axis
    ylim([min(ylim) 70])
    text(xmean(:,k1), ymean(:,k1)+2, compose('\\mu = %.2f \\rightarrow',ymean(:,k1)), 'Horiz','right', 'Vert','middle', 'Rotation',-80)
    xl1 = xline(ssmtx(1,:), '-g', 'DisplayName','Segment Start');
    xl2 = xline(ssmtx(2,:), '-r', 'DisplayName','Segment Stop');
    legend([xl1(1) xl2(1)], 'Location','SE')
end

ssmtx = 2×9

1613 1628 1690 1695 1761 1828 1834 1852 1906 1625 1676 1694 1757 1819 1830 1850 1902 1974

Lvss = 1×9 logical array
   0   1   0   1   1   0   0   1   1

ssmtx = 2×5

1628 1695 1761 1852 1906 1676 1757 1819 1902 1974

Flat_Means = 5×2 table
    X Centre    Y Mean
    ________    ______

      1651      30.648
      1725      28.549
      1789      35.506
      1876      33.305
      1939      39.992

ssmtx = 2×4

1330 1334 1407 1636 1331 1403 1631 1709

Lvss = 1×4 logical array
   0   1   1   1

ssmtx = 2×3

1334 1407 1636 1403 1631 1709

Flat_Means = 3×2 table
    X Centre    Y Mean
    ________    ______

     1367.5     31.885
       1518      32.04
     1671.5     38.432

The islocalmax function makes this relatively straightforward, however dealing with ‘real-world’ data inevitably involves some compromises. It may not be possible to ‘correctly’ detect all the flat segments you detect visually, because it may not be possible to code those criteria.

The data are noisy, and while there are various ways to deal with that, filtering the data could make it impossible to determine the segment start and stop points. If you want to filter them, I suggest experimenting with movmean or movmedian to start with. They are both essentially lowpass filters, and relatively easy to implement.

.

[Konvictus177]

@Star Strider The updated code works perfect! Thank you very much!

[Star Strider]

As always, my pleasure!