Zeroing matrix elements outside of a certain range.

edit: I changed my data example input because it was a bad example that accepted some solution that are not going to work in general

Hi guys, I would like to zero the outer elements of a matrix so that they are not counted in a sum / average.

Basically the matrix is experimental data and each column comes from a different experiment and the first and last datapoints in each experiment are likely to be artefact.

for example:

data = rand(5,2)

data = 5×2

0.3193 0.3313 0.1014 0.5895 0.2794 0.4797 0.6875 0.1108 0.6199 0.3224

and I would like to keep the points 1:4 in the first column (experiment 1) and 2:5 in the second column (experiment 2).

Ideally, I would like to input a matrix of bounds of valid range:

range = [1,4;2,5]

range = 2×2

1 4 2 5

and I would like the data for the index not in that range to be zeroed and ideally without a loop

like

data(index<range(1,:) or index>range(2,:)) = 0

but of course this is not the right line...

A long way would be:

remove = ones(5,2)

remove = 5×2

1 1 1 1 1 1 1 1 1 1

remove(range(1,1):range(1,2),1) = 0

remove = 5×2

0 1 0 1 0 1 0 1 1 1

remove(range(2,1):range(2,2),2) = 0

remove = 5×2

0 1 0 0 0 0 0 0 1 0

data(remove)=0

Array indices must be positive integers or logical values.

data appears to be both a function and a variable. If this is unintentional, use 'clear data' to remove the variable 'data' from the workspace.

ah no, even that doesn't work. But even if it would, that's way too inefficient. Sorry, really struggling...

Answer 1

[Dyuman Joshi]

A simple approach via indexing -

data = [1;2;3;4;5].*ones(5,2).*[0.5,0.4]

data = 5×2

0.5000 0.4000 1.0000 0.8000 1.5000 1.2000 2.0000 1.6000 2.5000 2.0000

out = [data(1:4,1) data(2:5,2)]

out = 4×2

0.5000 0.8000 1.0000 1.2000 1.5000 1.6000 2.0000 2.0000

[The Gogo]

Thank you!

That works for this example, but I would like to have this line embedded in a script, where I would give the range as a matrix. like for each column (i) of my data, I need to keep (range(i,1) : range (i,2)) of it. Well.. I could do it precisely like that in a loop, treating column by column. But I'm sure there's a more efficient way. Something like

data(function(range)) 

Function definition are not supported in this context. Functions can only be created as local or nested functions in code files.

that will give me direcly

[0.5 1 1.5 2 0; 0 0.8 1.2 1.6 2]'

as a result.

basically I need some boolean function that says "index is not in range". That sound extremely simple. yet I can't find it. I find only some "value is not in range" but not "index is not in range' :/

hmmm. Would maybe

data(~(1:4),1;~(2:5),2) = 0

would work? But again, it doesn't help, because I want to handle all the columns at once not one by one.

Something like:

data(~in(range'),:) = 0

but of course that would not work.

Also your example work because here the ranges 1:4 and 2:5 have the same length so they can be put together in a matrix but actually it is not necessarily the case for me. For example if the range are 1:4 and 2:4 let's say. That's why I need to convert the values I don't want to zeros instead of removing them completely, so that each columns keep having the same length as the others.

There's an other reason: each line of my data column i associated to certain time coordinates. If I totally remove some elements, it will not correspond anymore. For example data(3,1) and data(3,2) are data that are measured for the same time condition. But in your example this is not the case anymore for out(3,1) and out(3,2).

Anyway, thanks again!

[The Gogo]

So at the moment what I'm doing is:

data = [1;2;3;4;5].*ones(5,2).*[0.5,0.4]

data = 5×2

0.5000 0.4000 1.0000 0.8000 1.5000 1.2000 2.0000 1.6000 2.5000 2.0000

range = [1,4;2,5]

range = 2×2

1 4 2 5

zeroness = zeros(size(data))

zeroness = 5×2

0 0 0 0 0 0 0 0 0 0

for i = 1:size(data,2)
    zeroness(range(i,1):range(i,2),i) = 1;
end

data = data.*zeroness

data = 5×2

0.5000 0 1.0000 0.8000 1.5000 1.2000 2.0000 1.6000 0 2.0000

That works. But this function is going to be called at each iteration of an optimization algorithm. So I can take the most powerfull optimization algorithm I want: if the function to evaluate already has a for loop in it. It's not going to be very efficient.

I'd like to replace the for loop by matrix operations using the matrix "range" on the indexes of "data".

... but I can't figure out how...

[Dyuman Joshi]

@The Gogo, will there always be a single particular value missing from the range values?

[The Gogo]

No, it could be any number from 0 to several tens. Both at the begining and at the end of the range.

Answer 2

[Catalytic]

Zeroing the data will not exclude it from the average. You should use NaNs,

data = [1;2;3;4;5].*ones(5,2).*[0.5,0.4]

data = 5×2

0.5000 0.4000 1.0000 0.8000 1.5000 1.2000 2.0000 1.6000 2.5000 2.0000

range = [1,0;
        inf,1.7]

range = 2×2

1.0000 0 Inf 1.7000

data(range(1,:)>data  | data>range(2,:))=nan

data = 5×2

NaN 0.4000 1.0000 0.8000 1.5000 1.2000 2.0000 1.6000 2.5000 NaN

Average=mean(data,1,'omitnan')

Average = 1×2

1.7500 1.0000

[The Gogo]

Ahh no, sorry, I though it works but no. It's really a coincidence that the data I want to excude are the smallest of the first column and the largest of the last column. Sorry, I chose my example wrong. For my data, consider it's

data = rand(5,2)

data = 5×2

0.1537 0.4883 0.8801 0.3087 0.2036 0.1882 0.4103 0.6093 0.9977 0.5740

instead. Ok, correcting my post, now. Sorry, your case works for the example I put, but it's because I chose a wrong example.

Also sorry, my wrong, I'm not really making an average, I'm making a sum hence the zero.

+ ohhhhhhh there is a omitnan option!!!! I always do like

mean(data(~isnan(data)),2)

ans = 10×1

0.1537 0.8801 0.2036 0.4103 0.9977 0.4883 0.3087 0.1882 0.6093 0.5740

(not here as it's a sum, but on other occasions)

[Dyuman Joshi]

You are assuming that the comparison is made w.r.t some values.

However, the selection of data here is to be done w.r.t values being treated as indices.

[The Gogo]

Yes I think so. But I do not know how to treat those values as indices.

again, in a loop it would be easy by looping on indices and decide by comparing with range which values has to be zeroed.

Like for each column i,

data(j<range(i,1) | j> range(i,2),i) = 0 .

But to define j I need a for loop (plus another one for i).

I believe there is a way to make this without a loop, for all indexes at once with a matrix operation, but I do not find how.

[The Gogo]

Yes exactly, and how can I treat values as indices? that's my problem. Again in a loop it works fine, but I don't find how to do it trhough a matrix operarion