Eigenvalues and eigenvectors — Mathematics

In this lesson you will learn

Explain the geometric meaning of Av = λv
Find eigenvalues from the characteristic equation and then the matching eigenvectors
Diagonalise a 2 × 2 matrix and use it to compute powers
Find the long-run state of a Markov chain as an eigenvector

Stretch a rubber sheet by pulling its edges. Almost every arrow drawn on it turns to a new direction, but a few special arrows only get longer or shorter and keep their direction. For a matrix these special directions are its eigenvectors, and the stretch factors are its eigenvalues. They describe the natural frequencies of a bridge, the long-run behaviour of a Markov chain and the idea behind the first Google ranking. The German word eigen means “own”.

Definition and geometric meaning

Definition

Eigenvector and eigenvalue

A non-zero vector v is an eigenvector of a square matrix A if Av = λv for some number λ; this λ is the eigenvalue belonging to v.

A · v = λ · v, v ≠ 0

where:

Aa square n × n matrix
vthe eigenvector (a non-zero column)
λthe eigenvalue (may be negative or complex)

Geometrically, A does not turn v; it only stretches it by the factor λ. If λ > 1 the vector grows, if 0 < λ < 1 it shrinks, if λ < 0 it also flips to the opposite direction, and λ = 0 means that A squashes v to the zero vector. Any non-zero multiple of an eigenvector is again an eigenvector: A(cv) = cAv = λ(cv). So what we are really looking for are eigen-directions.

Checking a guess

A = [4 1; 2 3]. Are v = [1; 1] and u = [1; −2] eigenvectors? What about w = [1; 0]?

Show solution

Av = [4·1 + 1·1; 2·1 + 3·1] = [5; 5] = 5·[1; 1], so v is an eigenvector with λ = 5.
Au = [4 − 2; 2 − 6] = [2; −4] = 2·[1; −2], so u is an eigenvector with λ = 2.
Aw = [4; 2], which is not a multiple of [1; 0]: w changes direction, so it is not an eigenvector.

The characteristic equation

How do we find eigenvalues without guessing? Rewrite Av = λv as Av − λIv = 0, that is (A − λI)v = 0. This homogeneous system has a non-zero solution v exactly when the matrix A − λI is not invertible, which means its determinant is zero.

det(A − λI) = 0

where:

Ithe identity matrix of the same size
λthe unknown eigenvalue
det(A − λI)the characteristic polynomial, of degree n

An n × n matrix has n eigenvalues counted with multiplicity; some of them may be complex.

λ² − (tr A) · λ + det A = 0, tr A = a + d

where:

A = [a b; c d]a 2 × 2 matrix
tr Athe trace: the sum of the diagonal elements
det Aad − bc

Where it comes from: det(A − λI) = (a − λ)(d − λ) − bc = λ² − (a + d)λ + (ad − bc).

Eigenvalues and eigenvectors of a 2 × 2 matrix

Find the eigenvalues and eigenvectors of A = [4 1; 2 3].

Show solution

tr A = 4 + 3 = 7, det A = 4·3 − 1·2 = 10.
λ² − 7λ + 10 = 0 ⇒ (λ − 5)(λ − 2) = 0 ⇒ λ₁ = 5, λ₂ = 2. Check: 5 + 2 = 7 ✓, 5·2 = 10 ✓.
λ₁ = 5: A − 5I = [−1 1; 2 −2]; both rows say −x + y = 0 ⇒ y = x ⇒ v₁ = [1; 1].
λ₂ = 2: A − 2I = [2 1; 2 1]; 2x + y = 0 ⇒ y = −2x ⇒ v₂ = [1; −2].
As expected, the rows of A − λI came out proportional: det(A − λI) = 0.

A symmetric matrix and a rotation

Find the eigenvalues of S = [2 1; 1 2] and of the rotation R = [0 −1; 1 0].

Show solution

S: tr = 4, det = 3 ⇒ λ² − 4λ + 3 = 0 ⇒ λ = 3 and λ = 1, with eigenvectors [1; 1] and [1; −1].
They are perpendicular: 1·1 + 1·(−1) = 0, which always happens for symmetric matrices.
R: tr = 0, det = 1 ⇒ λ² + 1 = 0: no real roots, λ = ±i.
That makes sense: a rotation by 90° turns every vector, so there is no real eigen-direction (complex numbers are the next lesson).

Diagonalisation

Suppose an n × n matrix A has n linearly independent eigenvectors v₁, …, vₙ. Put them as the columns of a matrix P and the eigenvalues on the diagonal of D. The equations Avᵢ = λᵢvᵢ, taken together, say AP = PD, hence A = PDP⁻¹. This is always possible when the n eigenvalues are distinct, and a real symmetric matrix can always be diagonalised with perpendicular eigenvectors.

A = P · D · P⁻¹, Aᵏ = P · Dᵏ · P⁻¹

where:

Pthe matrix whose columns are the eigenvectors
Dthe diagonal matrix of the eigenvalues, in the same order
Dᵏthe diagonal matrix with λ₁ᵏ, …, λₙᵏ
ka natural number

Powers become easy: in Aᵏ = PDP⁻¹ · PDP⁻¹ · … every P⁻¹P in the middle cancels.

The tenth power without ten multiplications

Diagonalise A = [4 1; 2 3] and find a formula for Aⁿ.

Show solution

From the example above: P = [1 1; 1 −2], D = [5 0; 0 2].
det P = 1·(−2) − 1·1 = −3, P⁻¹ = (1/3)·[2 1; 1 −1] (the 2 × 2 inverse shortcut).
Check: PDP⁻¹ = [5 2; 5 −4] · (1/3)[2 1; 1 −1] = (1/3)[12 3; 6 9] = [4 1; 2 3] ✓.
Aⁿ = P·[5ⁿ 0; 0 2ⁿ]·P⁻¹ = (1/3)·[2·5ⁿ + 2ⁿ 5ⁿ − 2ⁿ; 2·5ⁿ − 2·2ⁿ 5ⁿ + 2·2ⁿ].
n = 3: (1/3)·[258 117; 234 141] = [86 39; 78 47], the same as A·A·A.
For n = 10 the formula gives [6510758 3254867; 6509734 3255891] at once.

Applications: Markov chains, PageRank, PCA

A Markov chain describes a system that jumps between states with fixed probabilities. Suppose two cafés near a university campus, A and B, share the students: each week 80% of A's customers stay loyal and 20% move to B, while 30% of B's customers move to A and 70% stay. Writing the shares as a column xₖ = [aₖ; bₖ], we get xₖ₊₁ = Txₖ, and every column of T sums to 1.

xₖ₊₁ = T · xₖ, T · x* = 1 · x*

where:

Tthe transition matrix: tᵢⱼ is the probability of moving from state j to state i
xₖthe shares (probabilities) after k steps
x*the stationary distribution: the eigenvector for λ = 1 whose elements sum to 1

Where do the customers end up?

T = [0.8 0.3; 0.2 0.7]. Find the long-run shares of the two cafés and show how fast they are reached from x₀ = [0.5; 0.5].

Show solution

λ = 1: (T − I)x = 0 ⇒ −0.2a + 0.3b = 0 ⇒ a = 1.5b.
Since a + b = 1: 2.5b = 1 ⇒ b = 0.4, a = 0.6. In the long run: 60% and 40%.
The second eigenvalue: λ₁ + λ₂ = tr T = 1.5 ⇒ λ₂ = 0.5 (check: 1 · 0.5 = det T = 0.56 − 0.06 = 0.5 ✓).
λ₂ = 0.5 means the distance to equilibrium halves every week: x₁ = [0.55; 0.45], x₂ = [0.575; 0.425], x₃ = [0.5875; 0.4125] → [0.6; 0.4].

PageRank. Google's original idea treats the web as a giant Markov chain: a random surfer keeps clicking links, and the importance of a page is its share in the stationary distribution, the eigenvector of the “Google matrix” for λ = 1. With billions of pages this vector is found by power iteration: start with any vector and multiply it by the matrix again and again. PCA (principal component analysis). The covariance matrix of a data set is symmetric; its eigenvector with the largest eigenvalue points in the direction in which the data spread most. Keeping only a few such principal components compresses data, for example in face recognition.

Python· Edit the code and run it again

import numpy as np

A = np.array([[4, 1], [2, 3]])
values, vectors = np.linalg.eig(A)
print(values)
print(np.linalg.matrix_power(A, 10))

T = np.array([[0.8, 0.3], [0.2, 0.7]])
x = np.array([0.5, 0.5])
for week in range(30):
    x = T @ x
print(np.round(x, 4))

▸ Expected output

[5. 2.]
[[6510758 3254867]
 [6509734 3255891]]
[0.6 0.4]

eig returns the eigenvalues (and eigenvectors of length 1). matrix_power confirms A¹⁰ from the diagonalisation example. The loop is power iteration for the café chain: after 30 weeks the shares are 0.6 and 0.4.

Key points

Av = λv with v ≠ 0: A does not turn an eigenvector, it only stretches it by λ.
Eigenvalues are the roots of det(A − λI) = 0; for 2 × 2, λ² − (tr A)λ + det A = 0.
The sum of the eigenvalues is the trace and their product is the determinant; for a triangular matrix they sit on the diagonal.
Eigenvectors come from (A − λI)v = 0; the rows must turn out proportional.
With n independent eigenvectors, A = PDP⁻¹ and Aᵏ = PDᵏP⁻¹.
The stationary distribution of a Markov chain is the eigenvector for λ = 1; the next eigenvalue shows how fast it is reached.

Check yourself

10 questions. Every correct answer earns XP.

1 / 10

What are the eigenvalues of [5 2; 2 2]?