Problem in statistical test

I tried to perform statistical test like below (T-test about two samples)

[h,p,ci,stats] = ttest2(Conf(:,2), Affn(:,2), 'tail', 'both', 'alpha', 0.05)

The results are

h = 0

p = 0.9510

ci = -0.0044

0.0042

stats =

tstat: -0.0615

df: 4798

sd: 0.0758

According to 'h' and 'p-value', hypothesis is not rejected, but statistics('tstat') is not in range of confidence interval('ci').

(As I know, statistics is must be in the range of confidence interval not to reject the test.)

In this case, how do I determine my opinion about the result of the statistical test?? (rejection or not)

Answer 1

[Star Strider]

The terminology can be confusing. From the Wikipedia article Null hypothesis (Basic definitions):

"The statement being tested in a test of statistical significance is called the null hypothesis. The test of significance is designed to assess the strength of the evidence against the null hypothesis. Usually, the null hypothesis is a statement of 'no effect' or 'no difference'."

and from the ttest2 documentation:

‘If h = 0, this indicates a failure to reject the null hypothesis at the Alpha significance level.’

So the hypothesis that there is no difference (null hypothesis) is accepted (i.e. not rejected).

There is no difference between the two groups at the chosen significance level, and assuming that they conform to the conditions of the t-test.

At least that is the way I have always read it.

.

[성현 윤]

Thank you for your nice answer.

As you said, 'h = 0' means null hypothesis is failed to reject.

This is the same with that the statistics(=tstat) is in the range of confidence interval(=ci).

But, in my result, 'tstat(-0.0615)' is out of range of 'ci(-0.0044 ~ 0.0042)'.

It is opposite the meaning of 'h = 0'.

So that's the reason that I'm confused.

[Star Strider]

My pleasure!

Note that:

tstat: -0.0615

is the t-statistic derived from your data.

From the ttest2 documentation on ‘ci’:

‘Confidence interval for the difference in population means of x and y, returned as a two-element vector containing the lower and upper boundaries of the 100 × (1 – Alpha)% confidence interval.’

It has nothing directly to do with the t-statistic that you are ccomparing it to. It is an entirely different value, with an entirely different interpreatation.

Also note that confidence intervals of opposite signs mean that they include 0, so a difference between population means of 0 (implying no difference) is within that range.

[성현 윤]

Thank you for answer.

You helped me clearly understand.

[Star Strider]

As always, my pleasure!