The

covariance matrix of an

*n*-

dimensional vector random variable **X** is the

*n*×

*n* matrix of the covariances between the elements of the

vector. The covariance matrix is the

generalization of the

variance of a

scalar random variable to multiple

dimensions.

The covariance matrix is the measure of how spread out the probability distribution of **X** is in *n*-dimensional space. The 'larger' the elements of the covariance matrix, the more spread out **X** is.

The covariance matrix of **X**, often denoted Σ, is defined by the formula

Σ = E[(**X**-**μ**)(**X**-**μ**)^{T}],

where

**X** and

**μ** are

column vectors and

**μ** = E[

*X*] is the

mean of

*X*.

The element of the matrix at row *i* and column *j* is the covariance between the *i*^{th} and *j*^{th} elements of **X**. Specifically, the *i*^{th}diagonal element of Σ is the variance of the *i*^{th} element of **X**.

The covariance matrix is always symmetric and positive semi-definite. These facts can be proven easily from the definition.

The covariance matrix may be singular. This can happen if the variable *X* does not have any variation along one or more dimensions.

More generally, the singularity of the covariance matrix implies that the distribution of *X* is flat along one or more orthogonal directions in *n*-dimensional space. The distribution is not really *n*-dimensional - it's less than *n*-dimensional. Specifically, the distribution lives in an affine subspace of the *n*-dimensional space. The rank of Σ gives the dimensionality of that affine subspace.