Extracting Zoomed in part of curve and finding rising Edges of data

Hello, I have a plot consisting of "square-ish" TTL pulses and I want to be able to

1: Extract the data thats in the "Zoomed" XLim, and then

2: Find the x-location of each rising edge at a certain y-value (say 3, although I may not have data thats exactly on 3, so the nearest to it)

3: Bin the seperations of each rising edge. As you can see below from the 2nd (zoomed in) plot I am having some drop outs so I want to analyse e.g. if there is any variabity in the drop out seperation as well as the actual rising edge seperation. My actual data consists of thousands of these peaks

(I dont have the signal processing toolbox!)

Heres a sample of my real data

and here's a zoomed in version (this would be what I would want to work with and I obtain it by setting the XLims

So for 1: above, I have this.

ax=app.UIAxes;
newlimits =ax.XLim;
[x,y] = getDataFromGraph(app,ax,1);   %See function below

xminidx = find(x >= newlimits(1),1,"first");
xmaxidx = find(x <= newlimits(2),1,"last");  
X=x(xminidx:xmaxidx,1); Y=y(xminidx:xmaxidx,1);
D=[X,Y];
head(D,8)   %View data

where

        function [datax,datay] = getDataFromGraph(app,ax,n)
            % ax = axes plot is on
            % n is the plot index, 1 is the last plot
            b=ax.Children;    % b(1) is last object added to ax            
            datax=(b(n).XData)';
            datay=(b(n).YData)';
        end

Then for 2, this is where I am struggling, I have tried this:

        rising_edges = find(diff(X > 3) > 0)
        hold(ax,"on");
        xline(ax,X(rising_edges))

But rising_edges is empty.

And for 3 I have no idea how to plot some kind of histogram of the seperations between adjacent pulse

[Voss]

Is rising_edges still empty if you do this?

rising_edges = find(diff(Y > 3) > 0) % use Y not X here
hold(ax,"on");
xline(ax,X(rising_edges))

[Jason]

Thanks Vos, Rising edges now seems correct!

So I now have this:

%Find Rising edges
rising_edges = find(diff(Y > 3) > 0)
hold(ax,"on");
%Plot
plot(ax,X(rising_edges),Y(rising_edges),'r*');   

%Calc adjacent pulse seperation
Xsep=X(rising_edges);
format shortG
diff(Xsep)   % I think this is the adjacent rising edge spacing?

But why when the threshold is 3 are the r* at lower valuesnd not at 3?

[Voss]

"But why when the threshold is 3 are the r* at lower valuesnd not at 3?"

rising_edges is the indices where Y is about to cross above 3 (i.e., the indices ii where Y(ii) <= 3 and Y(ii+1) > 3). Therefore, Y(rising_edges) is an array of numbers that are <= 3.

A simple example with one rising edge:

Y = linspace(0.5,10,7)

Y = 1×7

0.5000 2.0833 3.6667 5.2500 6.8333 8.4167 10.0000

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

rising_edges = find(diff(Y > 3) > 0)

rising_edges = 2

Y(rising_edges)

ans = 2.0833

If you want to plot the markers with a y-coordinate of exactly 3, then:

plot(ax,X(rising_edges),3*ones(numel(rising_edges),1),'r*');

[Jason]

Thanks.

So if i wanted to be strict and compare the seoerstion of rising edges at exactly tbe same y value. Would i need to interp between the Y(ii) <= 3 and Y(ii+1).

Also if i wanted the index to instead be the first value >3, how would I do that?

[Voss]

"Would i need to interp"

You could do that.

"if i wanted the index to instead be the first value >3"

Add 1 to the current definition:

rising_edges = find(diff(Y > 3) > 0) + 1

[Steven Lord]

Just to add one more possible tool to this discussion, you may want to load your data into a live script and use the Find Local Extrema task to interactively experiment with the various options that islocalmax and islocalmin support like min prominence, min separation, and max number of extrema. This will also allow you to see the code that programmatically performs that extrema finding, so you could use the code directly on other data sets after you've interactively tuned the options on one data set.

[Jason]

thanks steven, ultimately I need to use appd esigner, but I would be keen to explore your suggestion. where do I find this "Find Local Extrema task "

thank

[Steven Lord]

The "Open the Task" section on the documentation page to which I linked describes two ways to add the task to a live script.

Answer 1

[Star Strider]

Depending on the result you want, it might be easier to use islocalmax for this —

T1 = readtable('DS0007_REDUCED.CSV')

T1 = 2522x2 table
     Col1      Col2
    _______    ____

    0.00033    4.72
    0.00034     4.8
    0.00035    4.72
    0.00036    4.68
    0.00037    4.72
    0.00038    4.72
    0.00039    4.72
     0.0004    4.72
    0.00041     4.8
    0.00042     4.8
    0.00043    4.72
    0.00044    4.72
    0.00045     4.8
    0.00046    4.72
    0.00047    4.72
    0.00048    4.72

t = T1.Col1;
s = T1.Col2;

% figure
% plot(t, s)
% grid

Lvs = islocalmax(s, 'flatselection', 'first', MinProminence=0.5);
Lve = islocalmin(s, 'flatselection', 'last', MinProminence=0.5);

figure
plot(t, s)
hold on
plot(t(Lvs), s(Lvs), '|r')
plot(t(Lve), s(Lve), '|m')
hold off
grid

figure
plot(t, s)
hold on
plot(t(Lvs), s(Lvs), '|r')
plot(t(Lve), s(Lve), '|m')
hold off
grid
xlim([0.011 0.025])
ylim('padded')

●

Lvt = t >= 0.01;
Ivs = find(Lvs & Lvt);
Ive = find(Lve & Lvt);

for k = 1:numel(Ivs)
    idxrng = Ivs(k) - 10 : Ive(k) + 10;
    segments{k} = [t(idxrng) s(idxrng)];
end

figure
tiledlayout(4,2)
for k = 1:numel(segments)
    nexttile
    plot(segments{k}(:,1), segments{k}(:,2))
    hold on
    plot(t(Ivs(k)), s(Ivs(k)), 'vr')
    plot(t(Ive(k)), s(Ive(k)), '^r')
    hold off
    grid
    title(string(k))
    ylim('padded')
end

.

[Jason]

Wow, thats fantastic. Thanks

[Jason]

Hi, sorry one quick follow on question. Occasionally I get a double count for a single square pulse - how can I ignore the 2nd one

This 2nd one messes up the pulse seperation as the ditance to the next pulse is considered from the 2nd count on this pulse - so I need to get rid of it

Thanks

[Star Strider]

As always, my pleasure!

With respect to the double counts, experiment with the MinProminence value. You may have to nudge it up a bit. I would start by incrementing it from 0.50 to 0.55 by increments of 0.05 or less, until the double counts disappear. That should not significantly affect the pulses without the double counts.

Depending on what you want to do (and if you have the Signal Processing Toolbox), the risetime and falltime functions may also be useful.

.

[Jason]

Thanks. I've noticed that when there are more than 8 pulses the line with idxrng errors and I don't understand why.

for k = 1:numel(Ivs)    
    idxrng = Ivs(k) - 10 : Ive(k) + 10;
    segments{k} = [x(idxrng) y(idxrng)];
end

Is it possible to handle the case when there aremany more pulse and for example just take the 1st 8? (Thats a fantastic graphic btw!!)

BTW, I did try changing the prominence parameter but it didn't have any effect. I also have some occurances like this and was wondering if I should just remove any of the max locations that are within a certain distance 'x1' of another max or a certain distance 'x2' of a min

[Star Strider]

I crafted the loop and the following tiledlayout plot series simply to show the individual pulses in detail, since the original plot doesn’t permiit that.

To make ‘idxrng’ more robust, change it to:

idxrng = max(1, Ivs(k) - 5) : min(Ive(k) + 5, numel(t));

That should work. (I didn’t do that initially because I didn’t realise you would want to use that part of my code.)

I also changed the offsets from 10 to 5 for clarity. You can change them back (or to other values) if you want to.

If you’re still having problems after making this change, please post the data so that I can work with it.

EDIT —

With respect to the switching transients, I was going to suggest adding MinSeparation, however the documentation indicates that does not work with 'flatselection' because the function considers the flat selection to be a single point. (The problem with any sort of filtering — using smoothdata or an elliptic filter — is that it risks obscuring the sharp transitions of the pulses.) One option would be to use islocalmax and islocalmin without using 'flatselection' and using MinSeparation to return ponly the peaks (these being the initial overshsoots of the switching transients).

Using the current data, that would work liike this —

T1 = readtable('DS0007_REDUCED.CSV')

T1 = 2522x2 table
     Col1      Col2
    _______    ____

    0.00033    4.72
    0.00034     4.8
    0.00035    4.72
    0.00036    4.68
    0.00037    4.72
    0.00038    4.72
    0.00039    4.72
     0.0004    4.72
    0.00041     4.8
    0.00042     4.8
    0.00043    4.72
    0.00044    4.72
    0.00045     4.8
    0.00046    4.72
    0.00047    4.72
    0.00048    4.72

t = T1.Col1;
s = T1.Col2;

% figure
% plot(t, s)
% grid

% Lvs = islocalmax(s, 'flatselection', 'first', MinProminence=0.5);
% Lve = islocalmin(s, 'flatselection', 'last', MinProminence=0.5);

Lvs = islocalmax(s, MinSeparation=9, MinProminence=0.5);                % New Call
Lve = islocalmin(s, MinSeparation=9, MinProminence=0.5);                % New Call

figure
plot(t, s)
hold on
plot(t(Lvs), s(Lvs), '|r')
plot(t(Lve), s(Lve), '|m')
hold off
grid

figure
plot(t, s)
hold on
plot(t(Lvs), s(Lvs), '|r')
plot(t(Lve), s(Lve), '|m')
hold off
grid
xlim([0.011 0.025])
ylim('padded')

●

Lvt = t >= 0.01;
Ivs = find(Lvs & Lvt);
Ive = find(Lve & Lvt);

for k = 1:numel(Ivs)
    idxrng = max(1, Ivs(k) - 5) : min(Ive(k) + 5, numel(t));            % New Code
    segments{k} = [t(idxrng) s(idxrng)];
end

figure
tiledlayout(4,2)
for k = 1:numel(segments)
    nexttile
    plot(segments{k}(:,1), segments{k}(:,2))
    hold on
    plot(t(Ivs(k)), s(Ivs(k)), 'vr')
    plot(t(Ive(k)), s(Ive(k)), '^r')
    hold off
    grid
    title(string(k))
    ylim('padded')
end

You can take advantage of the switching transients this way.

I originally used 'flatselection' because it seemed the best option, and it usually is for such pulse trains without switching transients, although it is not universally appropriate. The MinProminence name-value pair is still required.

.

[Jason]

Thankyou, please see the attached which is more realistic of the number of peaks I will be looking at (although its not hard coded). You will see some of the double counts, also I do love those individual pulse plots you have, maybe limit it to just 8 somehow or even 4 (two ends and two in the middle?)

thankyou for all your effort

[Star Strider]

As always, my pleasure!

Adapting my code to your new data turned out not to be as straightforward as I thought it would be. There are still problems, most notably when the first oscillation after the switch is higher than the iniitial peak (or valley). This is the situation in pulses 6 and 10 (and perhaps others).

The MinProminence value can only do so much with respect to detecting the ‘correct’ peak, because it then detects (or fails to detect) others that it should not. This causes problems with the indexing. The best solution for this is to increase the sampling frequency (if possible), so that the signal is sampled such that it detects more points and therefore is likely to return the initial peak of the switching transient as always higher than any subsequent peak. Since that information cannot be recovered (or interpolated) from the present data, t+his is likely the best outcome. I encourage you to experiment with the islocalmax MinProminence parameter.

Try this —

T1 = readtable('TESTPULSEDATA.csv')

T1 = 5827x2 table
     Col1     Col2
    ______    ____

    9.8917     4.6
    9.8917     4.8
    9.8918    4.72
    9.8918     4.6
    9.8918    4.72
    9.8918    4.72
    9.8918    4.68
    9.8918    4.68
    9.8918    4.72
    9.8918    4.72
    9.8918    4.68
    9.8918    4.72
    9.8918    4.72
    9.8919    4.68
    9.8919    4.72
    9.8919    4.72

Ti = mean(diff(T1.Col1))                                                            % Sampling Interval

Ti = 1.0000e-05

Fs = 1/Ti                                                                           % Sampling Frequency

Fs = 1.0000e+05

t = T1.Col1;
s = T1.Col2;

% figure
% plot(t, s)
% grid

% Lvs = islocalmax(s, 'flatselection', 'first', MinProminence=0.5);
% Lve = islocalmin(s, 'flatselection', 'last', MinProminence=0.5);

smax = max(s)

smax = 4.8800

Lvm = [0; 0; s > smax*0.9];                                                         % Pulse Peaks
Lvms = strfind(Lvm(:).', [0 0 1]);                                                  % Pulse Starts
per = mean(diff(Lvms))                                                              % Pulse Period (Index Units)

per = 189.9333

Lvs = islocalmax(s, MinSeparation=ceil(per/3), MinProminence=0.275);                % New Call
Lve = islocalmin(s, MinSeparation=ceil(per/3), MinProminence=0.275);                % New Call

figure
plot(t, s)
hold on
plot(t(Lvs), s(Lvs), '|r')
plot(t(Lve), s(Lve), '|m')
hold off
grid

figure
plot(t, s, '.-')                                                                % Plot Showing Sampling Points
hold on
plot(t(Lvs), s(Lvs), 'vr')
plot(t(Lve), s(Lve), '^m')
hold off
grid
xlim([min(t)-0.0001 min(t)+0.005])
ylim('padded')

% Lvt = t >= 0.01;
% Ivs = find(Lvs & Lvt);
% Ive = find(Lve & Lvt);
Ivs = find(Lvs);                                                                % Start Indices
Ive = find(Lve);                                                                % ENd Indices
maxlen = min([numel(Ive) numel(Ivs)])

maxlen = 31

CheckIdx = [Ivs(1:maxlen) Ive(1:maxlen) Ive(1:maxlen)-Ivs(1:maxlen)];           % Check Indexing Results
disp(CheckIdx)

          52          51          -1
         189         241          52
         379         431          52
         569         621          52
         759         811          52
         949        1001          52
        1142        1191          49
        1329        1381          52
        1519        1571          52
        1709        1761          52
        1902        1951          49
        2089        2141          52
        2279        2331          52
        2469        2521          52
        2659        2711          52
        2849        2901          52
        3039        3091          52
        3229        3281          52
        3419        3471          52
        3609        3661          52
        3799        3851          52
        3989        4041          52
        4179        4231          52
        4369        4421          52
        4559        4611          52
        4749        4801          52
        4939        4991          52
        5129        5181          52
        5319        5371          52
        5509        5561          52
        5699        5751          52

Lvgt = CheckIdx(:,3) > 10;                                                      % ‘Logical Vector Greater Than’
CheckLen = nnz(Lvgt)

CheckLen = 30

Ivs = Ivs(Lvgt);
Ive = Ive(Lvgt);

for k = 1:numel(Ivs)
    idxrng = max(1, Ivs(k) - 5) : min(Ive(k) + 5, numel(t));                    % New Code (Changed From Previous)
    segments{k} = [t(idxrng) s(idxrng)];
end

NrPlots = numel(Ive)

NrPlots = 30

NrCols = 3;
NrRows = ceil(NrPlots/NrCols)

NrRows = 10

figure
tiledlayout(NrRows,NrCols)
for k = 1:numel(segments)
    nexttile
    plot(segments{k}(:,1), segments{k}(:,2))
    hold on
    plot(t(Ivs(k)), s(Ivs(k)), 'vr')
    plot(t(Ive(k)), s(Ive(k)), '^r')
    hold off
    grid
    title(string(k))
    ylim('padded')
end
sgtitle('All Pulses')

figure
tiledlayout(3,2)
for k = 1:ceil(numel(Ive)/6):numel(Ive)
    nexttile
    plot(segments{k}(:,1), segments{k}(:,2))
    hold on
    plot(t(Ivs(k)), s(Ivs(k)), 'vr')
    plot(t(Ive(k)), s(Ive(k)), '^r')
    hold off
    grid
    title(string(k))
    ylim('padded')
end
sgtitle('Selected Pulses')

figure
tiledlayout(2,1)
for k = [6 10]
    nexttile
    plot(segments{k}(:,1), segments{k}(:,2))
    hold on
    plot(t(Ivs(k)), s(Ivs(k)), 'vr')
    plot(t(Ive(k)), s(Ive(k)), '^r')
    hold off
    grid
    title(string(k))
    ylim('padded')
end

This code may still need to be tweaked for other data sets. I made it as robust as I could.

.

[Jason]

Thankyou again, this is beautiful.

Can I ask how the strfind line works?

Lvm = [0; 0; s > smax*0.9];                                                         % Pulse Peaks
Lvms = strfind(Lvm(:).', [0 0 1]);                                                  

% Pulse Starts
per = mean(diff(Lvms))                                                              % Pulse Period (Index Units)
per = 189.9333

[Star Strider]

As always, my pleasure!

This is an atypical application of strfind. It generally works to find specific patterns in strings (string array or character vector), however it turns out to also find specific numerical patterns in numeric vectors. In this instance, I’m looking to see where the logical vector matches the [0 0 1] pattern, indicating that it is detecting a transition. (The results are actually offset by 2 because it returns the index of first instance of the pattern, so here the first 0 in the vector. However since I’m then using it to get the differences in the vectors to estimate the period in index units, the offset doesn’t matter. If I wanted the position of the first 1 in the pattern, I would add 2 to the result.)

I’m using the indices here to get the period of the pulse train to use with the MinSeparation name-value pair.

.

[Jason]

Hi Star Strider, is it possible to suggest a way around this issue that I have found. It occurs when a valley is found before a peak, such as:

I thought the following would handle this:

    Ivs = find(Lvs);                       % Start (Peaks) Indices
    Ive = find(Lve);                       % End (Valleys) Indices
    Ive(1,1)
    Ivs(1,1)
    if Ive(1,1)<=Ivs(1,1)        % If first valley  is before first peak, delete it
        disp('TRUE')
        Ive(1,1)=[];     
    end

But I get the following

    ans =
    261
    ans =
    6001
    TRUE
    A null assignment can have only one non-colon index.
    Error in PulseAnalysis/FindPeakMaxMinAUTOButtonPushed (line 332)
    Ive(1,1)=[];

[Jason]

HA....just found this!

      A = rand(10,2) ; 
      A(1,:) = []   % you can remove a row
      A(1,1) = []   % you cannot remove an element 

[Star Strider]

I’vee been working on this for a while this morning.

I’m having problems getting an outlier detection approach to work that works in both instances (both data sets). It should be possible.

EDIT — (8 Feb 2025 at 18:14)

I checked my code with respect to both of these data sets, and it turns out that the second version of my code is robust to both of them. I also put the code in a function. At present it returns nothing, however if you want it to return an output, I just need to know what that output is. It should be fairly straightforward to add it.

Try this —

csvfiles = dir('*.csv');
csvfiles = [csvfiles; dir('*.CSV')]

csvfiles = 2x1 struct array with fields:
    name
    folder
    date
    bytes
    isdir
    datenum

for k = 1:numel(csvfiles)
    analysePulseTrain(csvfiles(k).name)
end

T1 = 5827x2 table
     Col1     Col2
    ______    ____

    9.8917     4.6
    9.8917     4.8
    9.8918    4.72
    9.8918     4.6
    9.8918    4.72
    9.8918    4.72
    9.8918    4.68
    9.8918    4.68
    9.8918    4.72
    9.8918    4.72
    9.8918    4.68
    9.8918    4.72
    9.8918    4.72
    9.8919    4.68
    9.8919    4.72
    9.8919    4.72

Ti = 1.0000e-05

Fs = 1.0000e+05

smax = 4.8800

per = 189.9333

maxlen = 31

          52          51          -1
         189         241          52
         379         431          52
         569         621          52
         759         811          52
         949        1001          52
        1142        1191          49
        1329        1381          52
        1519        1571          52
        1709        1761          52
        1902        1951          49
        2089        2141          52
        2279        2331          52
        2469        2521          52
        2659        2711          52
        2849        2901          52
        3039        3091          52
        3229        3281          52
        3419        3471          52
        3609        3661          52
        3799        3851          52
        3989        4041          52
        4179        4231          52
        4369        4421          52
        4559        4611          52
        4749        4801          52
        4939        4991          52
        5129        5181          52
        5319        5371          52
        5509        5561          52
        5699        5751          52

CheckLen = 30

NrPlots = 30

NrRows = 10

●

T1 = 2522x2 table
     Col1      Col2
    _______    ____

    0.00033    4.72
    0.00034     4.8
    0.00035    4.72
    0.00036    4.68
    0.00037    4.72
    0.00038    4.72
    0.00039    4.72
     0.0004    4.72
    0.00041     4.8
    0.00042     4.8
    0.00043    4.72
    0.00044    4.72
    0.00045     4.8
    0.00046    4.72
    0.00047    4.72
    0.00048    4.72

Ti = 1.0000e-05

Fs = 1.0000e+05

smax = 5

per = 337.8571

maxlen = 8

          21          19          -2
        1166        1218          52
        1366        1418          52
        1566        1618          52
        1766        1818          52
        1966        2018          52
        2166        2218          52
        2366        2418          52

CheckLen = 7

NrPlots = 7

NrRows = 3

function analysePulseTrain(filename)

titlestr = string(strrep(filename,"_","\_"));

T1 = readtable(filename)

t = T1{:,1};
s = T1{:,2};

Ti = mean(diff(t))                                                                  % Sampling Interval
Fs = 1/Ti                                                                           % Sampling Frequency

% figure
% plot(t, s)
% grid

smax = max(s)
Lvm = [0; 0; s > smax*0.9];                                                         % Pulse Peaks
Lvms = strfind(Lvm(:).', [0 0 1]);                                                  % Pulse Starts
per = mean(diff(Lvms))                                                              % Pulse Period (Index Units)

Lvs = islocalmax(s, MinSeparation=ceil(per/3), MinProminence=0.275);                % New Call
Lve = islocalmin(s, MinSeparation=ceil(per/3), MinProminence=0.275);                % New Call

figure
plot(t, s)
hold on
plot(t(Lvs), s(Lvs), '|r')
plot(t(Lve), s(Lve), '|m')
hold off
grid
title(titlestr)


figure
plot(t, s, '.-')                                                                    % Plot Showing Sampling Points
hold on
plot(t(Lvs), s(Lvs), 'vr')
plot(t(Lve), s(Lve), '^m')
hold off
grid
xlim([min(t)-0.0001 min(t)+0.005])
ylim('padded')
title(titlestr)

Ivs = find(Lvs);                                                                    % Start Indices
Ive = find(Lve);                                                                    % ENd Indices
maxlen = min([numel(Ive) numel(Ivs)])

CheckIdx = [Ivs(1:maxlen) Ive(1:maxlen) Ive(1:maxlen)-Ivs(1:maxlen)];               % Check Indexing Results
disp(CheckIdx)

Lvgt = CheckIdx(:,3) > 10;                                                          % ‘Logical Vector Greater Than’
CheckLen = nnz(Lvgt)
Ivs = Ivs(Lvgt);
Ive = Ive(Lvgt);

for k = 1:numel(Ivs)
    idxrng = max(1, Ivs(k) - 5) : min(Ive(k) + 5, numel(t));                        % New Code (Changed From Previous)
    segments{k} = [t(idxrng) s(idxrng)];
end

NrPlots = numel(Ive)
NrCols = 3;
NrRows = ceil(NrPlots/NrCols)

figure
tiledlayout(NrRows,NrCols)
for k = 1:numel(segments)
    nexttile
    plot(segments{k}(:,1), segments{k}(:,2))
    hold on
    plot(t(Ivs(k)), s(Ivs(k)), 'vr')
    plot(t(Ive(k)), s(Ive(k)), '^r')
    hold off
    grid
    title(string(k))
    ylim('padded')
end
sgtitle(["All Pulses" titlestr])



figure
tiledlayout(3,2)
for k = 1:ceil(numel(Ive)/6):numel(Ive)
    nexttile
    plot(segments{k}(:,1), segments{k}(:,2))
    hold on
    plot(t(Ivs(k)), s(Ivs(k)), 'vr')
    plot(t(Ive(k)), s(Ive(k)), '^r')
    hold off
    grid
    title(string(k))
    ylim('padded')
end
sgtitle(["Selected Pulses" titlestr])

% figure
% tiledlayout(2,1)
% for k = [6 10]
%     nexttile
%     plot(segments{k}(:,1), segments{k}(:,2))
%     hold on
%     plot(t(Ivs(k)), s(Ivs(k)), 'vr')
%     plot(t(Ive(k)), s(Ive(k)), '^r')
%     hold off
%     grid
%     title(string(k))
%     ylim('padded')
% end
end

.

[Jason]

Thanks so much for all your help!

" At present it returns nothing, however if you want it to return an output, I just need to know what that output is. It should be fairly straightforward to add it. "

As you ask, I want to be able to monitor the pulse seperations and identify any anomalies. The below graphic shows what Im tring to do (so in the pic, I want to draw an xline at the x location on the top plot correspondng to the pulse number picked out in the bottom plot.

So I have added the following to your fantastic code.

    % Calc Pulse Seperations, but need to ensure the following  
    if Ive(1,1)<=Ivs(1,1)    %If valley comes before a peak
        disp('TRUE')
        Ive(1,:)=[];         % Cannot do this: Ive(1,1)=[]; 
    end
    
    f=1000;     %y value multiplier (to work in us rather than ms)
    sep=diff(x(Ivs));
    ax=app.UIAxes2;     % Bottom Graph        
    cl=[0.47 0.25 0.80];
    plot(ax,f*sep,'LineWidth',2,'Color',cl); grid(ax,"on");
    title(ax,'Pulse Seperations','Color','w','FontSize',14,'FontWeight','normal'); 
    ylim(ax,'padded'); xlabel(ax,'Pulse #'); ylabel(ax,'Period (us)');
    ytickformat(ax,'%0.2f');

And the code behind my "Find Pulse Number" is

    ax=app.UIAxes;       % Top Graph
    peaks=app.peaks;     % Public property so can access anywhere in GUI
    head(peaks,5)
    pn=app.PeakNumberEditField.Value;
    xloc=peaks(pn)+1           
    hold(ax,"on"); 
    xline(ax,xloc,'g-'); hold(ax,"off"); 

(app.peaks) was previously saved in this line of the peak finding routine

   Lvs = islocalmax(y, MinSeparation=ceil(per/3), MinProminence=mp);  app.peaks=x(Lvs);     
   Lve = islocalmin(y, MinSeparation=ceil(per/3), MinProminence=mp);  app.valleys=x(Lve);

But its not quite showing the xline in the correct location

[Jason]

I think ive solved it, This

xloc=peaks(pn)+1

Should be

xloc=peaks(pn+1)

[Star Strider]

I’m not writing my code in the context of an app, so I’m not sure what changies you’re making to it.

Beyond that, I’m having problems making it robust to everything you want to present to it. (I worked on it for a few hours this morning, then took a break from it.) I’m sure this is possible, just not straightforward.

I’m also running all three (so far) .csv files with it to be certain it works with all of them each time I change it.

[Star Strider]

The significant change here is to get the miidpoint rises and falls of the pulses, and then to use interp1 to find the nearest peak or valley to it (as detected by islocalmax or islocalmin). I have not looked in detaiil at every pulse, however the ones I randomty selected in the plots (that show enough requisite detail) seem to be correct, and the first and last pulse times make sense.

This version actually seems to be robust to every .csv file available to it —

csvfiles = dir('*.csv');
csvfiles = [csvfiles; dir('*.CSV')]

csvfiles = 3x1 struct array with fields:
    name
    folder
    date
    bytes
    isdir
    datenum

for k = 1:numel(csvfiles)
    fn = csvfiles(k).name
    analysePulseTrain(csvfiles(k).name)
end

fn = 'TESTDATA3.csv'

T1 = 149250x2 table
     Var1      Var2
    _______    ____

     0.3015    4.76
     0.3015    4.76
     0.3015    4.76
    0.30151    4.76
    0.30151    4.76
    0.30151    4.68
    0.30151    4.76
    0.30151    4.76
    0.30152    4.76
    0.30152    4.76
    0.30152    4.76
    0.30152    4.68
    0.30152    4.76
    0.30153    4.76
    0.30153    4.76
    0.30153    4.76

Ti = 
        2e-06

Fs = 
      500000

smin = 
        -0.52

smax = 
     5

smdn = 
         2.24

per = 
       1016.4

tper = 
      0.30353

lens = 1×2

147 147

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

maxlen = 
   147

Relevant Times [First Last] —
      0.30173      0.30172
       0.3031      0.30362
        0.305      0.30552
       0.3069      0.30742
       0.3088      0.30932
       0.3107      0.31122
       0.3126      0.31312
       0.3145      0.31502
       0.3164      0.31692
       0.3183      0.31882
       0.3202      0.32072
       0.3221      0.32262
        0.324      0.32452
       0.3259      0.32642
       0.3278      0.32832
       0.3297      0.33022
       0.3316      0.33212
       0.3335      0.33402
       0.3354      0.33592
       0.3373      0.33782
       0.3392      0.33972
       0.3411      0.34162
        0.343      0.34352
       0.3449      0.34542
       0.3468      0.34732
       0.3487      0.34922
       0.3506      0.35112
       0.3525      0.35302
       0.3544      0.35492
       0.3563      0.35682
       0.3582      0.35872
       0.3601      0.36062
        0.362      0.36252
       0.3639      0.36442
       0.3658      0.36632
       0.3677      0.36822
       0.3696      0.37012
       0.3715      0.37202
       0.3734      0.37392
       0.3753      0.37582
       0.3772      0.37772
       0.3791      0.37962
        0.381      0.38152
       0.3829      0.38342
       0.3848      0.38532
       0.3867      0.38722
       0.3886      0.38912
       0.3905      0.39102
       0.3924      0.39292
       0.3943      0.39482
       0.3962      0.39672
       0.3981      0.39862
          0.4      0.40052
        0.412      0.41252
        0.414      0.41452
        0.416      0.41652
        0.418      0.41852
         0.42      0.42052
        0.422      0.42252
        0.424      0.42452
        0.426      0.42652
        0.428      0.42852
         0.43      0.43052
        0.432      0.43252
        0.434      0.43452
        0.436      0.43652
        0.438      0.43852
         0.44      0.44052
        0.442      0.44252
        0.444      0.44452
        0.446      0.44652
        0.448      0.44852
         0.45      0.45052
        0.452      0.45252
        0.454      0.45452
        0.456      0.45652
        0.458      0.45852
         0.46      0.46052
        0.462      0.46252
        0.464      0.46452
        0.466      0.46652
        0.468      0.46852
         0.47      0.47052
        0.472      0.47252
        0.474      0.47452
        0.476      0.47652
        0.478      0.47852
         0.48      0.48052
        0.482      0.48252
        0.484      0.48452
        0.486      0.48652
        0.488      0.48852
         0.49      0.49052
        0.492      0.49252
        0.494      0.49452
        0.496      0.49652
        0.498      0.49852
          0.5      0.50052
        0.502      0.50252
        0.504      0.50452
        0.506      0.50652
        0.508      0.50852
         0.51      0.51052
        0.512      0.51252
        0.514      0.51452
        0.516      0.51652
        0.518      0.51852
         0.52      0.52052
        0.522      0.52252
        0.524      0.52452
        0.526      0.52652
        0.528      0.52852
         0.53      0.53052
        0.532      0.53252
        0.534      0.53452
        0.536      0.53652
        0.538      0.53852
         0.54      0.54052
        0.542      0.54252
        0.544      0.54452
        0.546      0.54652
        0.548      0.54852
         0.55      0.55052
        0.552      0.55252
        0.554      0.55452
        0.556      0.55652
        0.558      0.55852
         0.56      0.56052
        0.562      0.56252
        0.564      0.56452
        0.566      0.56652
        0.568      0.56852
         0.57      0.57052
        0.572      0.57252
        0.574      0.57452
        0.576      0.57652
        0.578      0.57852
         0.58      0.58052
        0.582      0.58252
        0.584      0.58452
        0.586      0.58652
        0.588      0.58852
         0.59      0.59052
        0.592      0.59252
        0.594      0.59452
        0.596      0.59652
        0.598      0.59852

Number of Pulse Starts = 147
Number of Pulse Ends = 147
Check Index = 
         114         111          -3
         801        1061         260
        1751        2011         260
        2701        2961         260
        3651        3910         259
        4601        4861         260
        5551        5811         260
        6501        6761         260
        7451        7711         260
        8401        8661         260
        9351        9611         260
       10301       10561         260
       11251       11511         260
       12201       12461         260
       13151       13411         260
       14101       14361         260
       15051       15310         259
       16001       16261         260
       16951       17210         259
       17901       18161         260
       18851       19111         260
       19801       20061         260
       20751       21010         259
       21701       21961         260
       22651       22910         259
       23601       23861         260
       24551       24811         260
       25501       25760         259
       26451       26711         260
       27401       27661         260
       28351       28610         259
       29301       29561         260
       30251       30511         260
       31201       31460         259
       32151       32410         259
       33101       33361         260
       34051       34311         260
       35001       35261         260
       35951       36211         260
       36901       37161         260
       37851       38110         259
       38801       39061         260
       39751       40011         260
       40701       40961         260
       41650       41911         261
       42601       42861         260
       43551       43810         259
       44501       44761         260
       45451       45711         260
       46401       46661         260
       47351       47611         260
       48301       48561         260
       49251       49511         260
       55251       55510         259
       56251       56511         260
       57250       57511         261
       58251       58510         259
       59251       59510         259
       60250       60511         261
       61251       61510         259
       62251       62510         259
       63251       63511         260
       64250       64510         260
       65251       65511         260
       66250       66511         261
       67251       67510         259
       68251       68511         260
       69250       69511         261
       70251       70511         260
       71251       71511         260
       72250       72511         261
       73250       73510         260
       74251       74511         260
       75250       75511         261
       76251       76511         260
       77251       77510         259
       78250       78511         261
       79250       79511         261
       80251       80511         260
       81250       81510         260
       82250       82510         260
       83251       83511         260
       84250       84511         261
       85250       85510         260
       86251       86510         259
       87251       87510         259
       88250       88511         261
       89251       89510         259
       90250       90510         260
       91251       91511         260
       92251       92511         260
       93250       93510         260
       94250       94510         260
       95251       95511         260
       96250       96511         261
       97250       97510         260
       98251       98511         260
       99250       99510         260
      100250      100511         261
      101251      101510         259
      102250      102510         260
      103250      103511         261
      104251      104510         259
      105250      105511         261
      106250      106511         261
      107251      107510         259
      108251      108511         260
      109250      109511         261
      110251      110510         259
      111250      111511         261
      112250      112510         260
      113251      113510         259
      114250      114511         261
      115250      115510         260
      116250      116510         260
      117250      117511         261
      118250      118510         260
      119251      119510         259
      120250      120511         261
      121250      121511         261
      122251      122510         259
      123250      123510         260
      124250      124511         261
      125251      125510         259
      126250      126511         261
      127250      127510         260
      128251      128510         259
      129250      129511         261
      130250      130510         260
      131251      131509         258
      132250      132510         260
      133250      133511         261
      134250      134510         260
      135250      135511         261
      136250      136511         261
      137251      137510         259
      138250      138510         260
      139250      139510         260
      140250      140510         260
      141250      141511         261
      142250      142510         260
      143251      143510         259
      144250      144510         260
      145250      145510         260
      146250      146510         260
      147250      147510         260
      148250      148510         260

CheckLen = 
   146

NrPlots = 
   146

NrRows = 
    49

fn = 'TESTPULSEDATA.csv'

T1 = 5827x2 table
     Col1     Col2
    ______    ____

    9.8917     4.6
    9.8917     4.8
    9.8918    4.72
    9.8918     4.6
    9.8918    4.72
    9.8918    4.72
    9.8918    4.68
    9.8918    4.68
    9.8918    4.72
    9.8918    4.72
    9.8918    4.68
    9.8918    4.72
    9.8918    4.72
    9.8919    4.68
    9.8919    4.72
    9.8919    4.72

Ti = 
        1e-05

Fs = 
        1e+05

smin = 
        -0.56

smax = 
         4.88

smdn = 
         2.16

per = 
   190

tper = 
       9.8936

lens = 1×2

31 31

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

maxlen = 
    31

Relevant Times [First Last] —
       9.8922       9.8922
       9.8936       9.8941
       9.8955        9.896
       9.8974       9.8979
       9.8993       9.8998
       9.9012       9.9017
       9.9031       9.9036
        9.905       9.9055
       9.9069       9.9074
       9.9088       9.9093
       9.9107       9.9112
       9.9126       9.9131
       9.9145        9.915
       9.9164       9.9169
       9.9183       9.9188
       9.9202       9.9207
       9.9221       9.9226
        9.924       9.9245
       9.9259       9.9264
       9.9278       9.9283
       9.9297       9.9302
       9.9316       9.9321
       9.9335        9.934
       9.9354       9.9359
       9.9373       9.9378
       9.9392       9.9397
       9.9411       9.9416
        9.943       9.9435
       9.9449       9.9454
       9.9468       9.9473
       9.9487       9.9492

Number of Pulse Starts = 31
Number of Pulse Ends = 31
Check Index = 
          52          51          -1
         189         241          52
         379         431          52
         569         621          52
         759         811          52
         949        1001          52
        1142        1191          49
        1329        1381          52
        1519        1571          52
        1709        1761          52
        1902        1951          49
        2089        2141          52
        2279        2331          52
        2469        2521          52
        2659        2711          52
        2849        2901          52
        3039        3091          52
        3229        3281          52
        3419        3471          52
        3609        3661          52
        3799        3851          52
        3989        4041          52
        4179        4231          52
        4369        4421          52
        4559        4611          52
        4749        4801          52
        4939        4991          52
        5129        5181          52
        5319        5371          52
        5509        5561          52
        5699        5751          52

CheckLen = 
    30

NrPlots = 
    30

NrRows = 
    10

fn = 'DS0007_REDUCED.CSV'

T1 = 2522x2 table
     Col1      Col2
    _______    ____

    0.00033    4.72
    0.00034     4.8
    0.00035    4.72
    0.00036    4.68
    0.00037    4.72
    0.00038    4.72
    0.00039    4.72
     0.0004    4.72
    0.00041     4.8
    0.00042     4.8
    0.00043    4.72
    0.00044    4.72
    0.00045     4.8
    0.00046    4.72
    0.00047    4.72
    0.00048    4.72

Ti = 
        1e-05

Fs = 
        1e+05

smin = 
         -0.6

smax = 
     5

smdn = 
          2.2

per = 
       342.71

tper = 
      0.00375

lens = 1×2

8 8

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

maxlen = 
     8

Relevant Times [First Last] —
      0.00053      0.00051
      0.01198       0.0125
      0.01398       0.0145
      0.01598       0.0165
      0.01798       0.0185
      0.01998       0.0205
      0.02198       0.0225
      0.02398       0.0245

Number of Pulse Starts = 8
Number of Pulse Ends = 8
Check Index = 
          21          19          -2
        1166        1218          52
        1366        1418          52
        1566        1618          52
        1766        1818          52
        1966        2018          52
        2166        2218          52
        2366        2418          52

CheckLen = 
     7

NrPlots = 
     7

NrRows = 
     3

function analysePulseTrain(filename)

titlestr = string(strrep(filename,"_","\_"));

format shortG

T1 = readtable(filename)

t = T1{:,1};
s = T1{:,2};

figure
plot(t,s)
grid

Ti = mean(diff(t))                                                                  % Sampling Interval
Fs = 1/Ti                                                                           % Sampling Frequency

[smin,smax] = bounds(s)
smdn = median([smin smax])
Lvm = [0; 0; s <= smdn];                                                            % Pulse Peaks
Lvms = strfind(Lvm(:).', [0 0 1])-2;                                                % Pulse Starts

per = mean(diff(Lvms))                                                              % Pulse Period (Index Units)
tper = t(ceil(per))
dLvms = [0 diff(Lvms)];
tLvms = t(Lvms).';

Lvme = strfind(Lvm(:).', [1 0 0]);                                                  % Pulse Ends
dLvme = [0 diff(Lvme)];
tLvme = t(Lvme).';

Lvs = islocalmax(s, MinSeparation=ceil(per*0.5), MinProminence=0.25);               % New Call
Lve = islocalmin(s, MinSeparation=ceil(per*0.5), MinProminence=0.25);               % New Call

Ivs = find(Lvs);
Ive = find(Lve);

Ivsi = interp1(Ivs, 1:numel(Ivs), Lvms, 'nearest');                                  % Interpolate ‘Lvms’ To 'nearest' ‘Ivs’ Index
Ivei = interp1(Ive, 1:numel(Ive), Lvme, 'nearest');                                  % Interpolate ‘Lvme’ To 'nearest' ‘Ive’ Index

Ivs = Ivs(isfinite(Ivsi));
Ive = Ive(isfinite(Ivei));

figure
plot(t, s)
hold on
% plot(t(Lvs), s(Lvs), '|r')
% plot(t(Lve), s(Lve), '|g')
plot(t(Ivs), s(Ivs), '|r')
plot(t(Ive), s(Ive), '|g')
hold off
grid
title(titlestr)


figure
plot(t, s, '.-')                                                                    % Plot Showing Sampling Points
hold on
% plot(t(Lvs), s(Lvs), 'vr')
% plot(t(Lve), s(Lve), '^m')
plot(t(Ivs), s(Ivs), 'vr')
plot(t(Ive), s(Ive), '^m')
hold off
grid
xlim([min(t)-0.0001 min(t)+0.015])
ylim('padded')
title(titlestr)
% xline(tLvms, '--r')
% xline(tLvme, '--g')

Ivs = find(Lvs);                                                                    % Start Indices
Ive = find(Lve);                                                                    % End Indices
lens = [numel(Ive) numel(Ivs)]
maxlen = min([numel(Ive) numel(Ivs)])

Ivs = Ivs(1:maxlen);
Ive = Ive(1:maxlen);

RelevantTimes = t([Ivs Ive]);
disp("Relevant Times [First Last] —")
disp(RelevantTimes)

disp("Number of Pulse Starts = "+numel(Ivs))
disp("Number of Pulse Ends = "+numel(Ive))

disp("Check Index = ")
CheckIdx = [Ivs(1:maxlen) Ive(1:maxlen) Ive(1:maxlen)-Ivs(1:maxlen)];               % Check Indexing Results
disp(CheckIdx)

Lvgt = CheckIdx(:,3) > 10;                                                          % ‘Logical Vector Greater Than’
CheckLen = nnz(Lvgt)
Ivs = Ivs(Lvgt);
Ive = Ive(Lvgt);

for k = 1:numel(Ivs)
    idxrng = max(1, Ivs(k) - 5) : min(Ive(k) + 5, numel(t));                        % New Code (Changed From Previous)
    segments{k} = [t(idxrng) s(idxrng)];
end

NrPlots = numel(Ive)
NrCols = 3;
NrRows = ceil(NrPlots/NrCols)

figure
tiledlayout(NrRows,NrCols)
for k = 1:numel(segments)
    nexttile
    plot(segments{k}(:,1), segments{k}(:,2))
    hold on
    plot(t(Ivs(k)), s(Ivs(k)), 'vr')
    plot(t(Ive(k)), s(Ive(k)), '^r')
    hold off
    grid
    title(string(k))
    ylim('padded')
end
sgtitle(["All Pulses" titlestr])

figure
tiledlayout(3,2)
for k = 1:ceil(numel(Ive)/6):numel(Ive)
    nexttile
    plot(segments{k}(:,1), segments{k}(:,2))
    hold on
    plot(t(Ivs(k)), s(Ivs(k)), 'vr')
    plot(t(Ive(k)), s(Ive(k)), '^r')
    hold off
    grid
    title(string(k))
    ylim('padded')
end
sgtitle(["Selected Pulses" titlestr])

% figure
% tiledlayout(2,1)
% for k = [6 10]
%     nexttile
%     plot(segments{k}(:,1), segments{k}(:,2))
%     hold on
%     plot(t(Ivs(k)), s(Ivs(k)), 'vr')
%     plot(t(Ive(k)), s(Ive(k)), '^r')
%     hold off
%     grid
%     title(string(k))
%     ylim('padded')
% end
end

EDIT — (12 Feb 2025 at 13:22)

Corrected typographical error in code.

.

[Jason]

Wow, what can I say, thankyou so much. I cant wait to go thru this tomorrow (Im in the UK).

Could I ask when you use Lvm, what does the Lv stand for? I assume m is median?

[Star Strider]

Thank you!

As always, my pleasure!

The ‘m’ is for ‘midpoint’. The ‘s’ and ‘e’ are for ‘start’ and ‘end’ respectively. The initial ‘L’ is for ‘logical’, the ‘I’ for ‘index’ (numerical) and ‘v’ for ‘vector’.

[Jason]

Sorry if its not appropriate here, but im wondering how i should handle my data when it consists of 1M and maybe 10M sampling points from oscilloscope.

As there are just 2 variables can a uitable hold this amount of data?

[Star Strider]

It’s appropriate, although I generally don’t do anything wiith GUIs. I’ve simply never needed to use them (although that might change with a project I’m now working on).

When I checked the uitable documentation, I didn’t see any mention of size limits, so that may only be with respect to available memory. I doubt that table arrays have any such limitation, and they do support tall arrays (see Extended Capabilities for that information). The uitable documentation did not mention their excluding tall arrays (or for that matter, anything at all about tall arrays).

[Jason]

thanks. Im getting into your latst code and was wondering about this line, should it be an equals sign?

Ivs = find(Lvs);                                                                    % Start Indices
Ive = find(Lve);                                                                    % End Indices
lens = [numel(Ive) numel(Ivs)]
maxlen = min([numel(Ive) numel(Ivs)])
Ivs = Ivs(1:maxlen);
Ive - Ive(1:maxlen);                % *********** THIS LINE **********
RelevantTimes = t([Ivs Ive]);
disp("Relevant Times [First Last] —")
disp(RelevantTimes)

[Star Strider]

As always, my pleasure!

It should be an equal sign, and normally something like that would throw an error. (In that instance, I’d have caught it.)

I changed (fixed) it, and re-ran that code, although that change doesn’t seem to make any difference in the results, probably because it didn’t change ‘Ive’, since ‘Ive’ was already defined as (1:maxlen), fortuantely. If it hadn¹t been, the ‘RelevantTimes’ assignment would have thrown a horzcat error, and I‘d have caught it then.

Good pick-up!

.

[Jason]

Im still seeing a few hiccups (as I collect new data)

I fully understand if you have already committed enough time for this.

1: I sometimes get this error

tLvms = t(Lvms).';
Array indices must be positive integers or logical values.
Error in PulseAnalysis/FindPeakMaxMinAUTOButtonPushed (line 303)
tLvms = x(Lvms).';

2: And, i've noticed now that when there is a missing peak (this is exactly what I want to analyse!!)

it captures a false valley.

Perhaps it doesn't matter, as ultimately what Im interested in is the seperations between the peaks which allows me to see how consistent the pulse periods are and highlight any missing peaks. But perhaps these mis-called valleys will show up on the subplots showing a selection of pulses

UPDATE: It does matter, the pulse seperation starts becoming negative

head and tail of CheckIndex:

Check Index = 
     2    51    49
   199   251    52
   399   451    52
   599   651    52
   799   851    52
      999002      992054       -6948
      999202      992254       -6948
      999402      992454       -6948
      999602      992654       -6948
      999802      992854       -6948
CheckLen =

As a quick fix, changing the Min Prominence value to 2 knocked out those false valleys

Lvs = islocalmax(y, MinSeparation=ceil(per*0.5), MinProminence=0.25);               
Lve = islocalmin(y, MinSeparation=ceil(per*0.5), MinProminence=2);       % Changed Value HERE

[Jason]

Could I also ask about the difference in the counts that I also see in Ive:

(see n1 & n2 below)

vs = find(Lvs);
Ive = find(Lve);
Ivsi = interp1(Ivs, 1:numel(Ivs), Lvms, 'nearest');   % Interpolate ‘Lvms’ To 'nearest' ‘Ivs’ Index
Ivei = interp1(Ive, 1:numel(Ive), Lvme, 'nearest');   % Interpolate ‘Lvme’ To 'nearest' ‘Ive’ Index
Ivs = Ivs(isfinite(Ivsi));
Ive = Ive(isfinite(Ivei));
n1=numel(Ivs)                          % First Count *************************

and then a bit later:

disp("Check Index = ")
CheckIdx = [Ivs(1:maxlen) Ive(1:maxlen) Ive(1:maxlen)-Ivs(1:maxlen)];     % Check Indexing Results
head(CheckIdx,5)
tail(CheckIdx,5)
%disp(CheckIdx)
Lvgt = CheckIdx(:,3) > 10;                                                % ‘Logical Vector Greater Than’
CheckLen = nnz(Lvgt)
Ivs = Ivs(Lvgt);
Ive = Ive(Lvgt);
n2=numel(Ivs)                        % Second Count *************************

in some instances Im seeing 628 for n1 and only 146 for n2.

(what exactly is the 2nd chunk of code doing?)

Thanks

[Star Strider]

The offset should have been +2 insteead of -2, since strfind counts from the first index of the pattern. The first detected instance of the pattern returned should then have been 3 rather than 1, and similarly for the rest of them. (I was probably just tired when I wrote that.)

Try this —

csvfiles = dir('*.csv');
csvfiles = [csvfiles; dir('*.CSV')];
fprintf('\n\nNumber of .csv files in this run: %d\n\n', numel(csvfiles))

Number of .csv files in this run: 4

for k = 1:numel(csvfiles)
    fprintf(['\n\n' repmat('=',1,80) '\n'])
    fn = csvfiles(k).name
    analysePulseTrain(csvfiles(k).name)
end

================================================================================

fn = 'TESTDATA3.csv'

T1 = 149250x2 table
     Var1      Var2
    _______    ____

     0.3015    4.76
     0.3015    4.76
     0.3015    4.76
    0.30151    4.76
    0.30151    4.76
    0.30151    4.68
    0.30151    4.76
    0.30151    4.76
    0.30152    4.76
    0.30152    4.76
    0.30152    4.76
    0.30152    4.68
    0.30152    4.76
    0.30153    4.76
    0.30153    4.76
    0.30153    4.76

Sampling Period 	=  2.000E-06 s
Sampling Frequency 	=  5.000E+05 Hz
Amplitude Minimum 	= -5.200E-01
Amplitude Maximum 	=  5.000E+00
Amplitude Median 	=  2.240E+00

lens = 1×2

147 147

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

maxlen = 
   147

Relevant Times [First Last] —
      0.30173      0.30172
       0.3031      0.30362
        0.305      0.30552
       0.3069      0.30742
       0.3088      0.30932
       0.3107      0.31122
       0.3126      0.31312
       0.3145      0.31502
       0.3164      0.31692
       0.3183      0.31882
       0.3202      0.32072
       0.3221      0.32262
        0.324      0.32452
       0.3259      0.32642
       0.3278      0.32832
       0.3297      0.33022
       0.3316      0.33212
       0.3335      0.33402
       0.3354      0.33592
       0.3373      0.33782
       0.3392      0.33972
       0.3411      0.34162
        0.343      0.34352
       0.3449      0.34542
       0.3468      0.34732
       0.3487      0.34922
       0.3506      0.35112
       0.3525      0.35302
       0.3544      0.35492
       0.3563      0.35682
       0.3582      0.35872
       0.3601      0.36062
        0.362      0.36252
       0.3639      0.36442
       0.3658      0.36632
       0.3677      0.36822
       0.3696      0.37012
       0.3715      0.37202
       0.3734      0.37392
       0.3753      0.37582
       0.3772      0.37772
       0.3791      0.37962
        0.381      0.38152
       0.3829      0.38342
       0.3848      0.38532
       0.3867      0.38722
       0.3886      0.38912
       0.3905      0.39102
       0.3924      0.39292
       0.3943      0.39482
       0.3962      0.39672
       0.3981      0.39862
          0.4      0.40052
        0.412      0.41252
        0.414      0.41452
        0.416      0.41652
        0.418      0.41852
         0.42      0.42052
        0.422      0.42252
        0.424      0.42452
        0.426      0.42652
        0.428      0.42852
         0.43      0.43052
        0.432      0.43252
        0.434      0.43452
        0.436      0.43652
        0.438      0.43852
         0.44      0.44052
        0.442      0.44252
        0.444      0.44452
        0.446      0.44652
        0.448      0.44852
         0.45      0.45052
        0.452      0.45252
        0.454      0.45452
        0.456      0.45652
        0.458      0.45852
         0.46      0.46052
        0.462      0.46252
        0.464      0.46452
        0.466      0.46652
        0.468      0.46852
         0.47      0.47052
        0.472      0.47252
        0.474      0.47452
        0.476      0.47652
        0.478      0.47852
         0.48      0.48052
        0.482      0.48252
        0.484      0.48452
        0.486      0.48652
        0.488      0.48852
         0.49      0.49052
        0.492      0.49252
        0.494      0.49452
        0.496      0.49652
        0.498      0.49852
          0.5      0.50052
        0.502      0.50252
        0.504      0.50452
        0.506      0.50652
        0.508      0.50852
         0.51      0.51052
        0.512      0.51252
        0.514      0.51452
        0.516      0.51652
        0.518      0.51852
         0.52      0.52052
        0.522      0.52252
        0.524      0.52452
        0.526      0.52652
        0.528      0.52852
         0.53      0.53052
        0.532      0.53252
        0.534      0.53452
        0.536      0.53652
        0.538      0.53852
         0.54      0.54052
        0.542      0.54252
        0.544      0.54452
        0.546      0.54652
        0.548      0.54852
         0.55      0.55052
        0.552      0.55252
        0.554      0.55452
        0.556      0.55652
        0.558      0.55852
         0.56      0.56052
        0.562      0.56252
        0.564      0.56452
        0.566      0.56652
        0.568      0.56852
         0.57      0.57052
        0.572      0.57252
        0.574      0.57452
        0.576      0.57652
        0.578      0.57852
         0.58      0.58052
        0.582      0.58252
        0.584      0.58452
        0.586      0.58652
        0.588      0.58852
         0.59      0.59052
        0.592      0.59252
        0.594      0.59452
        0.596      0.59652
        0.598      0.59852

Number of Pulse Starts = 147
Number of Pulse Ends = 147
Check Index = 
         114         111          -3
         801        1061         260
        1751        2011         260
        2701        2961         260
        3651        3910         259
        4601        4861         260
        5551        5811         260
        6501        6761         260
        7451        7711         260
        8401        8661         260
        9351        9611         260
       10301       10561         260
       11251       11511         260
       12201       12461         260
       13151       13411         260
       14101       14361         260
       15051       15310         259
       16001       16261         260
       16951       17210         259
       17901       18161         260
       18851       19111         260
       19801       20061         260
       20751       21010         259
       21701       21961         260
       22651       22910         259
       23601       23861         260
       24551       24811         260
       25501       25760         259
       26451       26711         260
       27401       27661         260
       28351       28610         259
       29301       29561         260
       30251       30511         260
       31201       31460         259
       32151       32410         259
       33101       33361         260
       34051       34311         260
       35001       35261         260
       35951       36211         260
       36901       37161         260
       37851       38110         259
       38801       39061         260
       39751       40011         260
       40701       40961         260
       41650       41911         261
       42601       42861         260
       43551       43810         259
       44501       44761         260
       45451       45711         260
       46401       46661         260
       47351       47611         260
       48301       48561         260
       49251       49511         260
       55251       55510         259
       56251       56511         260
       57250       57511         261
       58251       58510         259
       59251       59510         259
       60250       60511         261
       61251       61510         259
       62251       62510         259
       63251       63511         260
       64250       64510         260
       65251       65511         260
       66250       66511         261
       67251       67510         259
       68251       68511         260
       69250       69511         261
       70251       70511         260
       71251       71511         260
       72250       72511         261
       73250       73510         260
       74251       74511         260
       75250       75511         261
       76251       76511         260
       77251       77510         259
       78250       78511         261
       79250       79511         261
       80251       80511         260
       81250       81510         260
       82250       82510         260
       83251       83511         260
       84250       84511         261
       85250       85510         260
       86251       86510         259
       87251       87510         259
       88250       88511         261
       89251       89510         259
       90250       90510         260
       91251       91511         260
       92251       92511         260
       93250       93510         260
       94250       94510         260
       95251       95511         260
       96250       96511         261
       97250       97510         260
       98251       98511         260
       99250       99510         260
      100250      100511         261
      101251      101510         259
      102250      102510         260
      103250      103511         261
      104251      104510         259
      105250      105511         261
      106250      106511         261
      107251      107510         259
      108251      108511         260
      109250      109511         261
      110251      110510         259
      111250      111511         261
      112250      112510         260
      113251      113510         259
      114250      114511         261
      115250      115510         260
      116250      116510         260
      117250      117511         261
      118250      118510         260
      119251      119510         259
      120250      120511         261
      121250      121511         261
      122251      122510         259
      123250      123510         260
      124250      124511         261
      125251      125510         259
      126250      126511         261
      127250      127510         260
      128251      128510         259
      129250      129511         261
      130250      130510         260
      131251      131509         258
      132250      132510         260
      133250      133511         261
      134250      134510         260
      135250      135511         261
      136250      136511         261
      137251      137510         259
      138250      138510         260
      139250      139510         260
      140250      140510         260
      141250      141511         261
      142250      142510         260
      143251      143510         259
      144250      144510         260
      145250      145510         260
      146250      146510         260
      147250      147510         260
      148250      148510         260

CheckLen = 
   146

NrPlots = 
   146

NrRows = 
    49

●

================================================================================

fn = 'TESTPULSEDATA.csv'

T1 = 5827x2 table
     Col1     Col2
    ______    ____

    9.8917     4.6
    9.8917     4.8
    9.8918    4.72
    9.8918     4.6
    9.8918    4.72
    9.8918    4.72
    9.8918    4.68
    9.8918    4.68
    9.8918    4.72
    9.8918    4.72
    9.8918    4.68
    9.8918    4.72
    9.8918    4.72
    9.8919    4.68
    9.8919    4.72
    9.8919    4.72

Sampling Period 	=  1.000E-05 s
Sampling Frequency 	=  1.000E+05 Hz
Amplitude Minimum 	= -5.600E-01
Amplitude Maximum 	=  4.880E+00
Amplitude Median 	=  2.160E+00

lens = 1×2

31 31

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

maxlen = 
    31

Relevant Times [First Last] —
       9.8922       9.8922
       9.8936       9.8941
       9.8955        9.896
       9.8974       9.8979
       9.8993       9.8998
       9.9012       9.9017
       9.9031       9.9036
        9.905       9.9055
       9.9069       9.9074
       9.9088       9.9093
       9.9107       9.9112
       9.9126       9.9131
       9.9145        9.915
       9.9164       9.9169
       9.9183       9.9188
       9.9202       9.9207
       9.9221       9.9226
        9.924       9.9245
       9.9259       9.9264
       9.9278       9.9283
       9.9297       9.9302
       9.9316       9.9321
       9.9335        9.934
       9.9354       9.9359
       9.9373       9.9378
       9.9392       9.9397
       9.9411       9.9416
        9.943       9.9435
       9.9449       9.9454
       9.9468       9.9473
       9.9487       9.9492

Number of Pulse Starts = 31
Number of Pulse Ends = 31
Check Index = 
          52          51          -1
         189         241          52
         379         431          52
         569         621          52
         759         811          52
         949        1001          52
        1142        1191          49
        1329        1381          52
        1519        1571          52
        1709        1761          52
        1902        1951          49
        2089        2141          52
        2279        2331          52
        2469        2521          52
        2659        2711          52
        2849        2901          52
        3039        3091          52
        3229        3281          52
        3419        3471          52
        3609        3661          52
        3799        3851          52
        3989        4041          52
        4179        4231          52
        4369        4421          52
        4559        4611          52
        4749        4801          52
        4939        4991          52
        5129        5181          52
        5319        5371          52
        5509        5561          52
        5699        5751          52

CheckLen = 
    30

NrPlots = 
    30

NrRows = 
    10

================================================================================

fn = 'TestTableData5.csv'

T1 = 17396x2 table
     Var1      Var2 
    _______    _____

    0.70243    -0.04
    0.70244    -0.04
    0.70245    -0.04
    0.70246    -0.04
    0.70247    -0.04
    0.70248    -0.04
    0.70249    -0.04
     0.7025    -0.04
    0.70251    -0.04
    0.70252    -0.04
    0.70253    -0.04
    0.70254    -0.04
    0.70255    -0.04
    0.70256    -0.04
    0.70257    -0.04
    0.70258     3.32

Sampling Period 	=  1.000E-05 s
Sampling Frequency 	=  1.000E+05 Hz
Amplitude Minimum 	= -8.000E-01
Amplitude Maximum 	=  5.040E+00
Amplitude Median 	=  2.120E+00

lens = 1×2

88 85

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

maxlen = 
    85

Relevant Times [First Last] —
      0.70259      0.70311
      0.70449      0.70501
      0.70639      0.70691
      0.70829      0.70881
      0.71019      0.71071
      0.71209      0.71261
      0.71399      0.71451
      0.71589      0.71641
      0.71779      0.71831
      0.71969      0.72021
      0.72159      0.72211
      0.72349      0.72401
      0.72539      0.72591
      0.72729      0.72781
      0.72919      0.72971
      0.73109      0.73161
      0.73299      0.73351
      0.73489      0.73541
      0.73679      0.73731
      0.73869      0.73921
      0.74059      0.74111
      0.74249      0.74301
      0.74439      0.74491
      0.74629      0.74681
      0.74819      0.74871
      0.75009      0.75061
      0.75199      0.75251
      0.75389      0.75441
      0.75579      0.75631
      0.75769      0.75821
       0.7596      0.76011
      0.76149      0.76201
      0.76339      0.76391
      0.76529      0.76581
      0.76719      0.76771
      0.76909      0.76961
        0.771      0.77151
       0.7729      0.77341
       0.7748      0.77531
       0.7767      0.77721
       0.7786      0.77911
      0.78049      0.78101
       0.7824      0.78291
      0.78429      0.78481
       0.7862      0.78671
       0.7881      0.78861
         0.79      0.79051
      0.80198      0.79269
      0.80399       0.7959
      0.80598      0.79868
      0.80799       0.8025
      0.80998       0.8045
      0.81198       0.8065
      0.81399       0.8085
      0.81599       0.8105
      0.81798       0.8125
      0.81999       0.8145
      0.82199       0.8165
      0.82398       0.8185
      0.82599       0.8205
      0.82798       0.8225
      0.82999       0.8245
      0.83199       0.8265
      0.83399       0.8285
      0.83599       0.8305
      0.83799       0.8325
      0.83999       0.8345
      0.84199       0.8365
      0.84399       0.8385
      0.84599       0.8405
      0.84799       0.8425
      0.84999       0.8445
      0.85199       0.8465
      0.85399       0.8485
      0.85599       0.8505
      0.85799       0.8525
      0.85999       0.8545
      0.86199       0.8565
      0.86399       0.8585
      0.86599       0.8605
      0.86799       0.8625
      0.86999       0.8645
      0.87199       0.8665
      0.87399       0.8685
      0.87599       0.8705

Number of Pulse Starts = 85
Number of Pulse Ends = 85
Check Index = 
          17          69          52
         207         259          52
         397         449          52
         587         639          52
         777         829          52
         967        1019          52
        1157        1209          52
        1347        1399          52
        1537        1589          52
        1727        1779          52
        1917        1969          52
        2107        2159          52
        2297        2349          52
        2487        2539          52
        2677        2729          52
        2867        2919          52
        3057        3109          52
        3247        3299          52
        3437        3489          52
        3627        3679          52
        3817        3869          52
        4007        4059          52
        4197        4249          52
        4387        4439          52
        4577        4629          52
        4767        4819          52
        4957        5009          52
        5147        5199          52
        5337        5389          52
        5527        5579          52
        5718        5769          51
        5907        5959          52
        6097        6149          52
        6287        6339          52
        6477        6529          52
        6667        6719          52
        6858        6909          51
        7048        7099          51
        7238        7289          51
        7428        7479          51
        7618        7669          51
        7807        7859          52
        7998        8049          51
        8187        8239          52
        8378        8429          51
        8568        8619          51
        8758        8809          51
        9956        9027        -929
       10157        9348        -809
       10356        9626        -730
       10557       10008        -549
       10756       10208        -548
       10956       10408        -548
       11157       10608        -549
       11357       10808        -549
       11556       11008        -548
       11757       11208        -549
       11957       11408        -549
       12156       11608        -548
       12357       11808        -549
       12556       12008        -548
       12757       12208        -549
       12957       12408        -549
       13157       12608        -549
       13357       12808        -549
       13557       13008        -549
       13757       13208        -549
       13957       13408        -549
       14157       13608        -549
       14357       13808        -549
       14557       14008        -549
       14757       14208        -549
       14957       14408        -549
       15157       14608        -549
       15357       14808        -549
       15557       15008        -549
       15757       15208        -549
       15957       15408        -549
       16157       15608        -549
       16357       15808        -549
       16557       16008        -549
       16757       16208        -549
       16957       16408        -549
       17157       16608        -549
       17357       16808        -549

CheckLen = 
    47

NrPlots = 
    47

NrRows = 
    16

================================================================================

fn = 'DS0007_REDUCED.CSV'

T1 = 2522x2 table
     Col1      Col2
    _______    ____

    0.00033    4.72
    0.00034     4.8
    0.00035    4.72
    0.00036    4.68
    0.00037    4.72
    0.00038    4.72
    0.00039    4.72
     0.0004    4.72
    0.00041     4.8
    0.00042     4.8
    0.00043    4.72
    0.00044    4.72
    0.00045     4.8
    0.00046    4.72
    0.00047    4.72
    0.00048    4.72

Sampling Period 	=  1.000E-05 s
Sampling Frequency 	=  1.000E+05 Hz
Amplitude Minimum 	= -6.000E-01
Amplitude Maximum 	=  5.000E+00
Amplitude Median 	=  2.200E+00

lens = 1×2

8 8

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

maxlen = 
     8

Relevant Times [First Last] —
      0.00053      0.00051
      0.01198       0.0125
      0.01398       0.0145
      0.01598       0.0165
      0.01798       0.0185
      0.01998       0.0205
      0.02198       0.0225
      0.02398       0.0245

Number of Pulse Starts = 8
Number of Pulse Ends = 8
Check Index = 
          21          19          -2
        1166        1218          52
        1366        1418          52
        1566        1618          52
        1766        1818          52
        1966        2018          52
        2166        2218          52
        2366        2418          52

CheckLen = 
     7

NrPlots = 
     7

NrRows = 
     3

function analysePulseTrain(filename)

titlestr = string(strrep(filename,"_","\_"));

format shortG

T1 = readtable(filename)

t = T1{:,1};
s = T1{:,2};

figure
plot(t,s)
grid
title(titlestr)

Ti = mean(diff(t));                                                                 % Sampling Interval
Fs = 1/Ti;                                                                          % Sampling Frequency

[smin,smax] = bounds(s);
smdn = (smax+smin)/2;
Lvm = [0; 0; s <= smdn];                                                            % Pulse Peaks
Lvms = strfind(Lvm(:).',  [0 0 1])+2;                                               % Pulse Starts

per = mean(diff(Lvms));                                                             % Pulse Period (Index Units)
tper = t(ceil(per));
fprintf('\n\nSampling Period \t= %10.3E s\nSampling Frequency \t= %10.3E Hz\nAmplitude Minimum \t= %10.3E\nAmplitude Maximum \t= %10.3E\nAmplitude Median \t= %10.3E\n\n', Ti, Fs, smin, smax, smdn)

dLvms = [0 diff(Lvms)];
tLvms = t(Lvms).';

Lvme = strfind(Lvm(:).', [1 0 0]);                                                  % Pulse Ends
dLvme = [0 diff(Lvme)];
tLvme = t(Lvme).';

Lvs = islocalmax(s, MinSeparation=ceil(per*0.5), MinProminence=0.25);               % New Call
Lve = islocalmin(s, MinSeparation=ceil(per*0.5), MinProminence=0.25);               % New Call

Ivs = find(Lvs);
Ive = find(Lve);

Ivsi = interp1(Ivs, 1:numel(Ivs), Lvms, 'nearest');                                  % Interpolate ‘Lvms’ To 'nearest' ‘Ivs’ Index
Ivei = interp1(Ive, 1:numel(Ive), Lvme, 'nearest');                                  % Interpolate ‘Lvme’ To 'nearest' ‘Ive’ Index

Ivs = Ivs(isfinite(Ivsi));
Ive = Ive(isfinite(Ivei));

figure
plot(t, s)
hold on
% plot(t(Lvs), s(Lvs), '|r')
% plot(t(Lve), s(Lve), '|g')
plot(t(Ivs), s(Ivs), '|r')
plot(t(Ive), s(Ive), '|g')
hold off
grid
title(titlestr)


figure
plot(t, s, '.-')                                                                    % Plot Showing Sampling Points
hold on
% plot(t(Lvs), s(Lvs), 'vr')
% plot(t(Lve), s(Lve), '^m')
plot(t(Ivs), s(Ivs), 'vr')
plot(t(Ive), s(Ive), '^m')
hold off
grid
xlim([min(t)-0.0001 min(t)+0.015])
ylim('padded')
title(titlestr)
% xline(tLvms, '--r')
% xline(tLvme, '--g')

Ivs = find(Lvs);                                                                    % Start Indices
Ive = find(Lve);                                                                    % End Indices
lens = [numel(Ive) numel(Ivs)]
maxlen = min([numel(Ive) numel(Ivs)])

Ivs = Ivs(1:maxlen);
Ive = Ive(1:maxlen);

RelevantTimes = t([Ivs Ive]);
disp("Relevant Times [First Last] —")
disp(RelevantTimes)

disp("Number of Pulse Starts = "+numel(Ivs))
disp("Number of Pulse Ends = "+numel(Ive))

disp("Check Index = ")
CheckIdx = [Ivs(1:maxlen) Ive(1:maxlen) Ive(1:maxlen)-Ivs(1:maxlen)];               % Check Indexing Results
disp(CheckIdx)

Lvgt = CheckIdx(:,3) > 10;                                                          % ‘Logical Vector Greater Than’
CheckLen = nnz(Lvgt)
Ivs = Ivs(Lvgt);
Ive = Ive(Lvgt);

for k = 1:numel(Ivs)
    idxrng = max(1, Ivs(k) - 5) : min(Ive(k) + 5, numel(t));                        % New Code (Changed From Previous)
    segments{k} = [t(idxrng) s(idxrng)];
end

NrPlots = numel(Ive)
NrCols = 3;
NrRows = ceil(NrPlots/NrCols)

figure
tiledlayout(NrRows,NrCols)
for k = 1:numel(segments)
    nexttile
    plot(segments{k}(:,1), segments{k}(:,2))
    hold on
    plot(t(Ivs(k)), s(Ivs(k)), 'vr')
    plot(t(Ive(k)), s(Ive(k)), '^r')
    hold off
    grid
    title(string(k))
    ylim('padded')
end
sgtitle(["All Pulses" titlestr])

figure
tiledlayout(3,2)
for k = 1:ceil(numel(Ive)/6):numel(Ive)
    nexttile
    plot(segments{k}(:,1), segments{k}(:,2))
    hold on
    plot(t(Ivs(k)), s(Ivs(k)), 'vr')
    plot(t(Ive(k)), s(Ive(k)), '^r')
    hold off
    grid
    title(string(k))
    ylim('padded')
end
sgtitle(["Selected Pulses" titlestr])

% figure
% tiledlayout(3,2)
% for k = numel(segments)-5: numel(segments)
%     nexttile
%     plot(segments{k}(:,1), segments{k}(:,2))
%     hold on
%     plot(t(Ivs(k)), s(Ivs(k)), 'vr')
%     plot(t(Ive(k)), s(Ive(k)), '^r')
%     hold off
%     grid
%     title(string(k))
%     ylim('padded')
% end
end

EDIT — Ran all the files again to be certain that my revised code works with all of them. It seems to. Also, some aesthetic changes.

.

[Star Strider]

‘in some instances Im seeing 628 for n1 and only 146 for n2.’

The original data are counted in ‘n1’ however there have to be an equal number of starting and ending indices, annd that is forced in ‘n2’.

My code now seems to be detecting valleys that don’t have a prior peak (or have a very small peak, a small fraction — perhaaps 1% — of the other peak amplitudes in ‘TestTableData5.csv’. The way my code is currently set up, those result in negative pulse lengths for the following pulses and those pulses are eliminated by thresholding the pulse lengths. I’ll have to devise a fix for that, since it didn’t occur in the previous data sets. That will involve either returning those peaks (that might very easily mean returning and filtering a lot of noise, so that might not be practical since those peak amplitudes are similar to the noise amplitude and it may not be possible to distinguish them from the noise) or devising a way of eliminating those valleys from the total. Eliminating them might be easiest.

.

[Star Strider]

I made this adaptable in order to get it to work. (I’ve actually been working on it for several hours a day over the past few days. These data are not easy to work with, part of that being that they’re noisy. I usually deal with that by filtering or smoothing, however that’s not appropiriate here because filtering would destroy the pulse morphology.) It may be necessary for you to make it further adaptable for data that it may still have problems with (and so it hasn’t yet seen), although it works with all the provided files.

The ‘analysePulseTrain’ function now has an output, that being a table with the starting and ending times and indices for each identified pulse in the file.

Try this —

csvfiles = dir('*.csv');
csvfiles = [csvfiles; dir('*.CSV')];
fprintf('\n\nNumber of .csv files in this run: %d\n\n', numel(csvfiles))

Number of .csv files in this run: 4

fprintf(['\n\n' repmat('=',1,80)])

================================================================================

for k = 1:numel(csvfiles)
    fprintf(['\n\n' repmat('=',1,80) '\n'])
    fn{k} = csvfiles(k).name
    PulseTable{k,:} = analysePulseTrain(fn{k});
end

================================================================================

fn = 1x1 cell array
    {'TESTDATA3.csv'}

File: TESTDATA3.csv: 149250 Rows,  2 Columns

================================================================================

fn = 1x2 cell array
    {'TESTDATA3.csv'}    {'TESTPULSEDATA.csv'}

File: TESTPULSEDATA.csv:  5827 Rows,  2 Columns

================================================================================

fn = 1x3 cell array
    {'TESTDATA3.csv'}    {'TESTPULSEDATA.csv'}    {'TestTableData5.csv'}

File: TestTableData5.csv: 17396 Rows,  2 Columns

================================================================================

fn = 1x4 cell array
    {'TESTDATA3.csv'}    {'TESTPULSEDATA.csv'}    {'TestTableData5.csv'}    {'DS0007_REDUCED.CSV'}

File: DS0007_REDUCED.CSV:  2522 Rows,  2 Columns

●

for k = 1:numel(PulseTable)
    fprintf(['\n\n' repmat('=',1,80) '\n'])
    fprintf('\n\nFile: %s\nPulseParameters Size: %5d Rows, %2d Columns\nExcerpt —\n\n',fn{k}, size(PulseTable{k}))
    Pulse_Parameters = PulseTable{k}([1:5 end-4:end],:)
end

================================================================================

File: TESTDATA3.csv
PulseParameters Size:   146 Rows,  5 Columns
Excerpt —

Pulse_Parameters = 10x5 table
    Pulse    Start Time    Stop Time    Start Index    Stop Index
    _____    __________    _________    ___________    __________

       1       0.3031       0.30362            801           1061
       2        0.305       0.30552           1751           2011
       3       0.3069       0.30742           2701           2961
       4       0.3088       0.30932           3651           3910
       5       0.3107       0.31122           4601           4861
     142         0.59       0.59052     1.4425e+05     1.4451e+05
     143        0.592       0.59252     1.4525e+05     1.4551e+05
     144        0.594       0.59452     1.4625e+05     1.4651e+05
     145        0.596       0.59652     1.4725e+05     1.4751e+05
     146        0.598       0.59852     1.4825e+05     1.4851e+05

================================================================================

File: TESTPULSEDATA.csv
PulseParameters Size:    30 Rows,  5 Columns
Excerpt —

Pulse_Parameters = 10x5 table
    Pulse    Start Time    Stop Time    Start Index    Stop Index
    _____    __________    _________    ___________    __________

      1        9.8936       9.8941          189            241   
      2        9.8955        9.896          379            431   
      3        9.8974       9.8979          569            621   
      4        9.8993       9.8998          759            811   
      5        9.9012       9.9017          949           1001   
     26        9.9411       9.9416         4939           4991   
     27         9.943       9.9435         5129           5181   
     28        9.9449       9.9454         5319           5371   
     29        9.9468       9.9473         5509           5561   
     30        9.9487       9.9492         5699           5751   

================================================================================

File: TestTableData5.csv
PulseParameters Size:    84 Rows,  5 Columns
Excerpt —

Pulse_Parameters = 10x5 table
    Pulse    Start Time    Stop Time    Start Index    Stop Index
    _____    __________    _________    ___________    __________

      1       0.70259       0.70311           17            69   
      2       0.70449       0.70501          207           259   
      3       0.70639       0.70691          397           449   
      4       0.70829       0.70881          587           639   
      5       0.71019       0.71071          777           829   
     80       0.86599        0.8665        16357         16408   
     81       0.86799        0.8685        16557         16608   
     82       0.86999        0.8705        16757         16808   
     83       0.87199        0.8725        16957         17008   
     84       0.87399        0.8745        17157         17208   

================================================================================

File: DS0007_REDUCED.CSV
PulseParameters Size:     7 Rows,  5 Columns
Excerpt —

Pulse_Parameters = 10x5 table
    Pulse    Start Time    Stop Time    Start Index    Stop Index
    _____    __________    _________    ___________    __________

      1       0.01198       0.0125         1166           1218   
      2       0.01398       0.0145         1366           1418   
      3       0.01598       0.0165         1566           1618   
      4       0.01798       0.0185         1766           1818   
      5       0.01998       0.0205         1966           2018   
      3       0.01598       0.0165         1566           1618   
      4       0.01798       0.0185         1766           1818   
      5       0.01998       0.0205         1966           2018   
      6       0.02198       0.0225         2166           2218   
      7       0.02398       0.0245         2366           2418   

function PulseParameters = analysePulseTrain(filename)

PulseParameters = [];

titlestr = string(strrep(filename,"_","\_"));

format shortG

T1 = readtable(filename);

fprintf('\n\nFile: %s: %5d Rows, %2d Columns\n\n',filename, size(T1))

t = T1{:,1};
s = T1{:,2};

% figure
% plot(t,s)
% grid
% title(titlestr+" 1")

[Lx,Px] = islocalmax(s);
[Ln,Pn] = islocalmin(s);

% Q0 = [nnz(Lx) nnz(Ln)]

maxkPx = maxk(Px,5);
maxkPn = maxk(Pn,5);

maxkPxm = mean(maxkPx);
maxkPnm = mean(maxkPn);

Pxmax = maxkPxm*0.85;
Pnmax = maxkPnm*0.85;

MinSep = 150;

Lvs = islocalmax(s, MinProminence=Pxmax, MinSeparation=MinSep);
Lve = islocalmin(s, MinProminence=Pnmax, Minseparation=MinSep);

Ivs1 = find(Lvs);
Ive1 = find(Lve);
Ive1 = Ive1(Ive1>Ivs1(1));

% Q3 = [nnz(Lvs) nnz(Lve)]

if numel(Ive1) < numel(Ivs1)
    Lve = islocalmin(s, MinProminence=0.05, Minseparation=MinSep);
end

Ivs = find(Lvs);
Ive = find(Lve);
% Q1 = [numel(Ivs) numel(Ive)]

Ive = Ive(Ive>Ivs(1));
% Q2 = [numel(Ivs) numel(Ive)]

% tStart = t(Ivs(1:min(10,numel(Ivs))));
% Ivsv = Ivs(1:min(10,numel(Ivs)))
% Ivev = Ive(1:min(10,numel(Ive)))

figure
plot(t,s)
hold on
plot(t(Ivs), s(Ivs), '|g', MarkerSize=25)
plot(t(Ive), s(Ive), '|r', MarkerSize=25)
hold off
grid
ylim('padded')
title(titlestr+" 1")

PulseParameters = table((1:numel(Ivs)).', t(Ivs), t(Ive), Ivs, Ive, VariableNames=["Pulse","Start Time","Stop Time","Start Index","Stop Index"]);
end

These are the most challenging data sets I’ve encountered here.

.

[Jason]

Thankyou so much, I wasn't expecting you to spend all your personal time on this - I really am very grateful to you. What is the key features that different here?

[Star Strider]

As always, my pleasure!

I wasn’t going to give up on this until I got a reasonable approach, since it seems to be important to what you’re doing. The key difference is that it is now adaptible. For whatever reason, with the MinProminence value set a a higher value, the last ‘valley’ isn’t detected in some files. With a lower value it is, however with some of the other files, that also identifies a lot of noise. So this approach first uses the higher value, and if then are fewer ‘valleys’ that ‘peaks’ (and all the valleys occurring after the first ‘peak’), it uses the lower value and checks those results. (Thus far it hasn’t needed to add any other adaptations.) With this approach, it results in identifying all the peaks and valleys for each file. It then checks to be sure there are an equal number of beginning and end indices, and calculates and creates the ‘PulseParameters’ table.

It would be difficult to have the results table have the same name as the original file (without the extension), however if you want to store the results files in an identifiable way, create a .mat file with that name:

fn = extractBefore(filename,'.');

save([fn ' PulseParameters.mat'], 'PulseParameters')

In the ‘analysePulseTrain’ function, add these lines after creating the table. That should result in a file with a matching name, for example:

DS0007_REDUCED PulseParameters.mat

I thought of that later.

.