Simplifying for-find loop functions to speed up processing

Hi all,

A pretty basic question, but I'm trying to find a more elegant solution to search through a 750,000x5 list to remove entries from a corresponding list when more than three value in any respective row are above a threshold of 10. The long way of doing so that I have is

    % mD is a 750000 x 5 matrix containing distances to the nearest 5
    % neighbors of the r,c,v point (from knnsearch) in each row
    
    for i = size(r,1):-1:1
        if size(find(mD(i,:)>10),2) > 3
            r(i) = [];
            c(i) = [];
            v(i) = [];
        end
    end

[Dyuman Joshi]

How does the condition checked depend on the for loop index?

If it does not, then you are effectively deleting the elements (with indices - size(r,1):-1:1)) or not, which can be done directly or not done at all.

[Cameron]

This code goes line by to check on a per-point basis which I'm trying to speed up, it goes in -1 steps to not run out of indexes since it deletes as it goes. It depends on the for loop because the corresponding values in the distance list and the r,c,v list are the same index. In pseudo-code terms, its:

Are there more than 3 values in mD row (i) above 10?
If yes, delete row(i) from r, c, v

[Torsten]

Are there more than 3 values in mD row (i) above 10?

But you don't refer to row i of mD in your loop - you refer to the complete matrix mD with your find-command.

[Cameron]

Oops, good catch, that's my mistake, that should be mD(i,:)

Answer 1

[Dyuman Joshi]

I assume r, c and v have the same number of rows -

%Check which rows from the given range in mD have more than 3 values greater than 10
idx = sum(mD(1:size(r,1),:)>10, 2)>3

%perform deletion
r(idx) = [];
c(idx) = [];
v(idx) = [];

If all the variables have the same number or rows then you can remove the indices used and just use mD.

Answer 2

[Cameron]

I've developed a better solution to the above, which is

    [~,mD] = knnsearch(rcv,rcv,'K',5);

    mD_thresh = find(mD>10);
    [row,~] = ind2sub(size(mD),mD_thresh);
    for i = 1:size(row,1)
        rcv(i,:) = [];
    end

This only takes a few seconds to run, but it sacrifices testing whether more than 3 values in the row are over the 10 threshold, and just deletes any row with a single value over the limit (preferable to a 200 hour run though). I also still feel there's a way to eliminate that for loop.