Is a matrix with a condition number of 1e20 definitely more ill-conditioned than a matrix with a condition number of 1e19？

I am working with some ill-conditioned matrices, trying to find the relationship between the matrix's ill-conditioning and the results. However, I have noticed that the condition numbers of some matrices are extremely large, with the smallest singular values even smaller than eps. Does this mean that the condition numbers I got might be inaccurate? Or is a matrix with a condition number of 1e20 definitely more ill-conditioned than a matrix with a condition number of 1e19？

[dpb]

Read <Cleve's blog> first...

Answer 1

[John D'Errico]

NO. There is effectively no significant difference between the two. At that point, you are into what are essentially random bit flips in the least significant bits. (Not truly random. Perhaps arbitrary is a more descriptive word.)

Had you computed the condition number of a symbolic matrix, where ALL elements were computed exactly, then there is SOME difference. Is it a meaningful difference? Still, probably not. For example, a symbolic hilbert matrix (I've not used hilb for a reason that may not be obvious, even to me.)

myhilb = @(n) 1./(sym(1:n) + (1:n).' - 1);
A15 = myhilb(15)

A15 = 

A16 = myhilb(16);
double(cond(A15))

ans = 6.1166e+20

double(cond(A16))

ans = 2.0223e+22

So two matrices that have both relatively huge condition numbers. Both large enough that any solves done in double precision will be effectively meaningless. There is a predictable and computational difference here in the condition number, but I had to go into the symbolic domain to see that.

cond(hilb(18))

ans = 2.5208e+18

cond(hilb(19))

ans = 3.1551e+18

cond(hilb(20))

ans = 3.3066e+18

cond(hilb(21))

ans = 5.5719e+18

cond(hilb(22))

ans = 1.1771e+18

At some point, we start to get into what is again, virtually random garbage. It is very important to see that the condition number (as computed using double precision, versus the symbolicly computed condition number) actually decreased from n=21 to 22, yet we know it must increase as the matrix size increases here. The true condition number of that 22x22 hilbert matrix is in fact much larger.

double(cond(myhilb(22)))

ans = 2.7184e+31

What does a large condition number mean? One way of interpreting it is that the condition number MAY amplify any noise in the least significant bits of the right hand side of a solve by the condition number. So unless you have something more than 20 digits of precision in your data and your array, then ANY result you see from such a solve, using either matrix will potentially be complete and total garbage. A single bit flip of any of those numbers could generate an unpredictably different result.

How often do you have data that is accurate to more than 20 significant digits? Do you frequently generate experiments where your measurements are that accurate? Condition numbers greater then 1/eps are essentially the same as an infinite condition number. The difference beyond that point is not a meaningful one, and even close to that point, I'd not trust it.

[HAOJIE haojie]

Thank you very much for your detailed answer！