Matlab returns non-zero values for std deviation when n=1

I have a data set with 58 observations and 17 variables.

17 of the observations have replicates, the others do not.

site contains the full list of observations, and

list = unique(site,'rows');

returns the list of sites without the replication.

for A = 1:length(list);
    idx = strcmp(list(A),site);
    mVar(A,:) = mean(data(idx,:),1);
end

calculates the mean for those observations that do have replicates. If n=1 the single value is returned.

However, when I try to calculate the standard deviation with,

for A = 1:length(list);
    idx = strcmp(list(A),site);
    mStdV(A,:) = std(data(idx,:),0);
end

standard deviation for observations with replicates is fine, but for observations with n=1, I get the same non-zero value for all variables. I was expecting a return of NaN for these.

[the cyclist]

Would it be possible for you to include a distilled dataset, with examples of the variables "site" and "data", that allow this code to run and exhibit the problem. I am not fully understanding the issue.

Answer 1

[the cyclist]

See my comment, which will help diagnose, but my guess is the following:

It seems that the variable "site" has multiple columns. You are taking care, when calculating the mean, to act on the first dimension (down the column) of the result. However, when you calculate the standard deviation, it looks like you are not doing this. So, I am guessing that you are inadvertently taking the standard deviation across a single (non-replicated) row of your variable.

Just a guess.

[Rhys Leeming]

I think your answer is correct. I checked the stddev across an entire row in Excel and didn't get quite the same answer, but close and each other row (where n=1)gave close answers. BUT why? 1. the code is the same as for calculating the means AND Matlab correctly calculates the stdev on any other observations where n > 1.

[Rhys Leeming]

Sure, but how do I upload the files? I read in two Excel files and my Matlab file - grand total about 40 kb.

[Walter Roberson]

MATLAB works along columns normally, but for many of the routines, if it has not been specifically told which dimension to work on and the data is a row vector, it automatically proceeds across the row instead. You can avoid this by specifically providing the dimension number to mean() or std()

[the cyclist]

As I mentioned in my answer, and Walter has reiterated, your code is actually NOT the same for the means and the standard deviations. The syntax you have used for the mean command has explicitly told it to always work down the columns, but you have not done that with the standard deviation command. For a 2-d (or higher) array, "std" will implicitly work down the column (1st dimension). But for a row vector, "std" assumes that you mean the standard deviation along that row, UNLESS you add the dimension parameter to the function call. Use the syntax "std(data(idx,:),0,1)" to do that.

[If you found this answer helpful, please accept it, as an indication to future questioners that this solved the issue.]

Answer 2

[Rhys Leeming]

I had the flags for std set wrong!

Should be;

for A = 1:length(list);
    idx = strcmp(list(A),site);
    mVar(A,:) = mean(data(idx,:),1);
    mStdV(A,:) = std(data(idx,:),0,1);
end