Fisher's z transform with atanh() to rho goes to infinity

I have run Spearman's correlations usign the following:

[rho, pval] = corr(A,B, 'type', 'Spearman', 'rows', 'complete'); 

I would like to conduct a fisher's z transform on my rho values so that I can compare them later on.

Doing

atanh(rho)

sometimes outputs Inf or -Inf (particularly when the output of rho is 1 / -1). In cases where the output of rho is 1.000 or -1.000 this doesn't occur.

I'm not entirely sure what I have missed or how I could fix this issue. Any help will be very much appreciated.

Answer 1

[Sam Chak]

Hi @Alba Peris, If you make sure that rho stays in the range , then will be finite.

rho = -0.999:0.001:0.999;
y   = atanh(rho);
plot(rho, y), grid on, xlabel('\rho'), ylabel('y')

[Alba Peris]

Thanks @Sam Chak.

I have realized it is because the output of my corr (the rho) is sometimes exactly 1 (that's when it goes to infinity and my double shows up as rho = 1) whereas in other cases it must be very close to 1 and my double is rho = 1.000

If I do rho == 1, in the first instance it does equal 1 but in the second one it doesn't.

Do you know how I could fix this within my corr() so that my rho does not go exactly to 1?

Thanks very much for the help!!

[Sam Chak]

Hi @Alba Peris

You can probably find the elements equal to one and then replace them by 0.999.

X           = magic(3);
Y           = X';
[rho, pval] = corr(X, Y)

rho = 3×3

0.8386 -0.7559 -0.4193 -0.2774 1.0000 -0.2774 -0.4193 -0.7559 0.8386

pval = 3×3

0.3667 0.4544 0.7245 0.8211 0.0000 0.8211 0.7245 0.4544 0.3667

idx         = find(abs(rho - 1) < eps)  % find the position

idx = 5

rho(idx)    = 0.999

rho = 3×3

0.8386 -0.7559 -0.4193 -0.2774 0.9990 -0.2774 -0.4193 -0.7559 0.8386

[Alba Peris]

Thanks very much! Indded this is what I did to solve the issue.

[Sam Chak]

You are welcome, @Alba Peris. I'm glad that it works our for you. If you find the solution helpful, please consider clicking 'Accept' ✔ on the answer and voting 👍 for it.

[MByk]

I am trying to average the correlation coefficients using the Fisher's transform and having the same problem.

yOut = squareform((CCMat - diag(diag(CCMat))), 'tovector');
RtoZ = mean(atanh(yOut));   
ZtoR = tanh(RtoZ);          

Columns 1 through 13
0.2313    0.6249    0.8157    0.8157       Inf    0.8157    0.8157       Inf    0.8157    0.8157    0.6249    0.6249    0.2313
Columns 14 through 26
0.6249    0.6249    0.2313    0.6249    0.2313    0.2313    0.6249    0.2313    0.2313    0.6249    0.2313       Inf    0.8157
Columns 27 through 39
Inf       Inf    0.8157       Inf    0.8157       Inf       Inf    0.8157       Inf    0.8157    0.8157       Inf    0.8157
Columns 40 through 45
Inf    0.8157       Inf    0.8157       Inf    0.8157

[Sam Chak]

@MByk, Try this:

idx       = find(abs(yOut - 1) < eps)  % find the position
yOut(idx) = 0.9999

[MByk]

Thank you very much.