Find the certain amount of values between NaNs and make another variable from these values

Hi!

I have a matrix A which has 50 rows and 15000 columns with some numbers and sometimes NaNs between them. In each of these rows I want to find periods of time when the continuous length of non-NaN values was not shorter then 270 (but it can 300, 500 or longer, whatever). I want to take 270 first values out from each of these epochs and make another variable from it. Is it somehow possible?

Just an example:

A =       1     2     3   NaN   NaN   NaN   NaN   NaN     4     5   NaN   NaN   NaN     6     7     8     9    10   NaN   NaN
         NaN   NaN     1     2     3     4   NaN   NaN   NaN   NaN     5   NaN   NaN   NaN     6     7     8     9    10  NaN
          1   NaN   NaN   NaN     2     3     4   NaN   NaN     5     6   NaN   NaN     7     8     9    10    11   NaN   NaN

I want to find the epochs when the number of values between NaN is 3 or more (for example). And then I take the first 3 elements of the epoch and put it to the new line of the new variable. In the end my new variable should look like this:

result =  1     2     3   
          6     7     8     
          1     2     3
          6     7     8
          2     3     4   
          7     8     9 

I hope I explained clearly.

Thanks for any help!

Answer 1

[Saurav Chaudhary]

Yes, it is possible. You can go through the code given below to get an idea, but again modify it according to your application:

Input matrix:

A =      [ 1     2     3   NaN   NaN   NaN   NaN   NaN     4     NaN   NaN   NaN   NaN     6     7     8     9    10   NaN   NaN
         NaN   NaN     1     2     3     4   NaN   NaN   NaN   NaN     5   NaN   NaN   NaN     6     7     8     9    10  NaN
          1   NaN   NaN   NaN     2     3     4   5   NaN     5     6   NaN   NaN     7     8     9    10    11   3 4];

Converting the matrices into 1-D vector, so as to traverse it easily:

[r,c] = size(A);
n = r * c;
reshape(A,[1,n]);
A= A';

Assigning left(L) and right(R) handles:

L= 1 ;
R= 2 ;

Specify limit you want between two NaN's:

Lim= 3 ;

Main body of function:

while R <= n
    
    %it takes care of the case when there are two consecutive number but NaN is next
    if isnan( A( R ) )
        L = R;
        R = R + 1;
    end
    
    %when there are consecutive numbers between NaN's 
    if R- L + 1 == Lim 
        Tem = zeros(1, Lim);
        i = 1;
        while i <= Lim
            Tem(i) = A(L);
            L = L + 1;
            i = i + 1;
        end
        while R<=n && ~isnan( A( R ) )
            R = R + 1;
        end
        L = R;
        R = R + 1;
        disp(Tem)
        
    %When either of the L or R is NaN we can skip them    
    elseif isnan( A( L ) ) || isnan( A( R ) )
        L = R;
        R = R + 1;
   
    %Otherwise we can move ahead
    else
        R = R + 1;
    end
end                                        

Answer 2

[Peter Perkins]

There's no need to loop over the 750K elements. Do this on each row x of your data, i.e. loop from 1:50.

Try stepping through this code in the debugger with these test rows

x = [1     2     3   NaN   NaN   NaN   NaN   NaN     4     5   NaN   NaN   NaN     6     7     8     9    10   NaN   NaN]
%x = [1     2     3   NaN   NaN   NaN   NaN   NaN     4     5   NaN   NaN   NaN     6     7     8     9    10   11   12]
%x = [NaN   NaN     1     2     3     4   NaN   NaN   NaN   NaN     5   NaN   NaN   NaN     6     7     8     9    10  NaN]
%x = [NaN   NaN     1     2     3     4   NaN   NaN   NaN   NaN     5   NaN   NaN   NaN     6     7     8     9    10  11]

and you will have learned a useful MATLAB idiom.

startsStops = find(diff(isnan(x)))+1; % <= the key
if ~isnan(x(1))
    startsStops = [1 startsStops];
end
if ~isnan(x(end))
    startsStops = [startsStops length(x)+1];
end
starts = startsStops(1:2:end);
stops = startsStops(2:2:end);

runLengths = stops - starts;
minLength = 3;
longRuns = (runLengths >= minLength);
starts = starts(longRuns);
runLengths = runLengths(longRuns);
numLongRuns = length(starts);

runs = starts + (0:minLength-1)';
xruns = reshape(x(runs),length(starts),minLength)