Documentation

Singular Values

A singular value and corresponding singular vectors of a rectangular matrix A are, respectively, a scalar σ and a pair of vectors u and v that satisfy

$\begin{array}{l}Av=\sigma u\\ {A}^{H}u=\sigma v,\end{array}$

where ${A}^{H}$ is the Hermitian transpose of A. The singular vectors u and v are typically scaled to have a norm of 1. Also, if u and v are singular vectors of A, then -u and -v are singular vectors of A as well.

The singular values σ are always real and nonnegative, even if A is complex. With the singular values on the diagonal of a diagonal matrix Σ and the corresponding singular vectors forming the columns of two orthogonal matrices U and V, you obtain the equations

$\begin{array}{l}AV=U\Sigma \\ {A}^{H}U=V\Sigma .\end{array}$

Since U and V are unitary matrices, multiplying the first equation by ${V}^{H}$ on the right yields the singular value decomposition equation

$A=U\Sigma {V}^{H}.$

The full singular value decomposition of an m-by-n matrix involves an m-by-m U, an m-by-n Σ, and an n-by-n V. In other words, U and V are both square, and Σ is the same size as A. If A has many more rows than columns (m > n), then the resulting m-by-m matrix U is large. However, most of the columns in U are multiplied by zeros in Σ. In this situation, the economy-sized decomposition saves both time and storage by producing an m-by-n U, an n-by-n Σ and the same V: The eigenvalue decomposition is the appropriate tool for analyzing a matrix when it represents a mapping from a vector space into itself, as it does for an ordinary differential equation. However, the singular value decomposition is the appropriate tool for analyzing a mapping from one vector space into another vector space, possibly with a different dimension. Most systems of simultaneous linear equations fall into this second category.

If A is square, symmetric, and positive definite, then its eigenvalue and singular value decompositions are the same. But, as A departs from symmetry and positive definiteness, the difference between the two decompositions increases. In particular, the singular value decomposition of a real matrix is always real, but the eigenvalue decomposition of a real, nonsymmetric matrix might be complex.

For the example matrix

A =
9     4
6     8
2     7

the full singular value decomposition is

[U,S,V] = svd(A)

U =

0.6105   -0.7174    0.3355
0.6646    0.2336   -0.7098
0.4308    0.6563    0.6194

S =

14.9359         0
0    5.1883
0         0

V =

0.6925   -0.7214
0.7214    0.6925

You can verify that U*S*V' is equal to A to within round-off error. For this small problem, the economy size decomposition is only slightly smaller.

[U,S,V] = svd(A,0)

U =

0.6105   -0.7174
0.6646    0.2336
0.4308    0.6563

S =

14.9359         0
0    5.1883

V =

0.6925   -0.7214
0.7214    0.6925

Again, U*S*V' is equal to A to within round-off error.

If the matrix A is large and sparse, then using svd to calculate all of the singular values and vectors is not always practical. For example, if you need to know just a few of the largest singular values, then calculating all of the singular values of a 5000-by-5000 sparse matrix is a lot of extra work. In cases where only a subset of the singular values and vectors are required, the svds function is preferred over svd.

For a 1000-by-1000 random sparse matrix with a density of about 30%,

n = 1000;
A = sprand(n,n,0.3);

the six largest singular values are

S = svds(A)

S =

130.2184
16.4358
16.4119
16.3688
16.3242
16.2838

Also, the six smallest singular values are

S = svds(A,6,'smallest')

S =

0.0740
0.0574
0.0388
0.0282
0.0131
0.0066

For smaller matrices that can fit in memory as a full matrix, full(A), using svd(full(A)) might still be quicker than svds. However, for truly large and sparse matrices, using svds becomes necessary.