Rayleigh quotients

Theorem

Fo a symmetric matrix A, we can express the smallest and largest eigenvalues, lambda_{rm min}(A) and lambda_{rm max}(A), as
 begin{array}{l} lambda_{rm min}(A)  =  displaystylemin_{x} : left{ x^TAx ~:~ x^Tx = 1 right} ,  lambda_{rm max}(A)  =  displaystylemax_{x} : left{ x^TAx ~:~ x^Tx = 1 right} . end{array}

Proof: The proof of the expression above derives from the SED of the matrix, and the invariance of the Euclidean norm constraint under orthogonal transformations. We show this only for the largest eigenvalue; the proof for the expression for the smallest eigenvalue follows similar lines. Indeed, with A = U Lambda U^T, we have
 max_{x} : left{ x^TAx ~:~ x^Tx = 1 right} = max_{x} : left{ x^TU Lambda U^Tx ~:~ x^Tx = 1 right}.
Now we can define the new variable tilde{x} = U^Tx, so that x = Utilde{x}, and express the problem as
 begin{array}{rcl} max_{x} : left{ x^TAx ~:~ x^Tx = 1 right} &=& displaystylemax_{tilde{x}} : left{ tilde{x}^TLambda tilde{x} ~:~ tilde{x}^Ttilde{x} = 1 right}   &=& displaystylemax_{tilde{x}} : left{ sum_{i=1}^n lambda_i tilde{x}_i^2 ~:~ sum_{i=1}^n tilde{x}_i^2 = 1 right}. end{array}
Clearly, the maximum is less than lambda_{rm max}. That upper bound is attained, with tilde{x}_i = 1 for an index i such that lambda_i = lambda_{rm max}, and tilde{x}_j = 0 for j ne i. This proves the result. This corresponds to setting x = Utilde{x} = u_i, where u_i is the eigenvector corresponding to lambda_i = lambda_{rm max}.