Script to Find Repeating Data within Tolerance
4 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
Hello,
I have a table that has over 6,000 rows of data with 8 columns, with each column representing a different data.
Many of the rows of data are similar but not exactly the same as other rows.
In other words, plotting column 1 vs column 2, and column 1 vs column 3, etc. gives a sinusoidal shape with a periodic behavior, but it is not exactly repeating, but slightly increasing.
With that in mind, I wrote a script such that it finds similar data within a tolerance, and removes that row of data if it is deemed as similar.
In short, the code is something like this:
for J=1:Length
for I=1:Length
if I~=J
if abs(table.colA(J)-table.colA(I))<=tolerance && abs(table.colB(J)-table.colB(I))<=tolerance && % Repeat for other columns
table.colA(I)=-1;
table.colB(I)=-1;
% Repeat for other columns
else
end
else
end
end
end
% Then I keep only the rows whose values are greater than -1
What I have works, I got it to go from over 6,000 rows to about 150 rows
However it is very computationally expenisve, meaning it takes several hours to run
Does anyone have any advice to speed this up or make it more efficient?
Or even just a better way to do this task?
I know there is an easy function that removes duplicate rows in a table, but since none of the rows are exactly the same, this won't work.
Is it possible to do this to a tolerance, like two rows not exactly duplicates, but "close enough" to each other.
I was looking into it here and the in maual but couldn't find any easier way to do it than what I did above.
Any help or suggestions is appreciated, thanks.
0 Commenti
Risposta accettata
Più risposte (0)
Vedere anche
Categorie
Scopri di più su Logical in Help Center e File Exchange
Prodotti
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!