Eigenvalues and eigenvectors

The definition Av = λv, the characteristic equation det(A − λI) = 0, worked 2 × 2 examples, diagonalisation A = PDP⁻¹ and the ideas behind Markov chains, principal component analysis and Google's PageRank.

  • University
  • 30 min
  • 10 questions
  • Lesson 88 of 92

In this lesson you will learn

  • Explain the geometric meaning of Av = λv
  • Find eigenvalues from the characteristic equation and then the matching eigenvectors
  • Diagonalise a 2 × 2 matrix and use it to compute powers
  • Find the long-run state of a Markov chain as an eigenvector

Stretch a rubber sheet by pulling its edges. Almost every arrow drawn on it turns to a new direction, but a few special arrows only get longer or shorter and keep their direction. For a matrix these special directions are its eigenvectors, and the stretch factors are its eigenvalues. They describe the natural frequencies of a bridge, the long-run behaviour of a Markov chain and the idea behind the first Google ranking. The German word eigen means “own”.

Definition and geometric meaning

Eigenvector and eigenvalueDefinition

A non-zero vector v is an eigenvector of a square matrix A if Av = λv for some number λ; this λ is the eigenvalue belonging to v.

A · v = λ · v, v ≠ 0

Where

A
a square n × n matrix
v
the eigenvector (a non-zero column)
λ
the eigenvalue (may be negative or complex)

Geometrically, A does not turn v; it only stretches it by the factor λ. If λ > 1 the vector grows, if 0 < λ < 1 it shrinks, if λ < 0 it also flips to the opposite direction, and λ = 0 means that A squashes v to the zero vector. Any non-zero multiple of an eigenvector is again an eigenvector: A(cv) = cAv = λ(cv). So what we are really looking for are eigen-directions.

Checking a guess

Problem

A = [4 1; 2 3]. Are v = [1; 1] and u = [1; −2] eigenvectors? What about w = [1; 0]?

Show solutionHide solution

Solution

Av = [4·1 + 1·1; 2·1 + 3·1] = [5; 5] = 5·[1; 1], so v is an eigenvector with λ = 5.
Au = [4 − 2; 2 − 6] = [2; −4] = 2·[1; −2], so u is an eigenvector with λ = 2.
Aw = [4; 2], which is not a multiple of [1; 0]: w changes direction, so it is not an eigenvector.

Sign up free to keep reading

The rest of the lesson, the quiz and your notes open with an Educora account. Signing up takes a minute, and your progress is saved on all your devices.

The characteristic equation

How do we find eigenvalues without guessing? Rewrite Av = λv as Av − λIv = 0, that is (A − λI)v = 0. This homogeneous system has a non-zero solution v exactly when the matrix A − λI is not invertible, which means its determinant is zero.

det(A − λI) = 0

Where

I
the identity matrix of the same size
λ
the unknown eigenvalue
det(A − λI)
the characteristic polynomial, of degree n

An n × n matrix has n eigenvalues counted with multiplicity; some of them may be complex.

λ² − (tr A) · λ + det A = 0tr A = a + d

Where

A = [a b; c d]
a 2 × 2 matrix
tr A
the trace: the sum of the diagonal elements
det A
ad − bc

Where it comes from: det(A − λI) = (a − λ)(d − λ) − bc = λ² − (a + d)λ + (ad − bc).

Eigenvalues and eigenvectors of a 2 × 2 matrix

Problem

Find the eigenvalues and eigenvectors of A = [4 1; 2 3].

Show solutionHide solution

Solution

tr A = 4 + 3 = 7, det A = 4·3 − 1·2 = 10.
λ² − 7λ + 10 = 0 ⇒ (λ − 5)(λ − 2) = 0 ⇒ λ₁ = 5, λ₂ = 2. Check: 5 + 2 = 7 ✓, 5·2 = 10 ✓.
λ₁ = 5: A − 5I = [−1 1; 2 −2]; both rows say −x + y = 0 ⇒ y = x ⇒ v₁ = [1; 1].
λ₂ = 2: A − 2I = [2 1; 2 1]; 2x + y = 0 ⇒ y = −2x ⇒ v₂ = [1; −2].
As expected, the rows of A − λI came out proportional: det(A − λI) = 0.

A symmetric matrix and a rotation

Problem

Find the eigenvalues of S = [2 1; 1 2] and of the rotation R = [0 −1; 1 0].

Show solutionHide solution

Solution

S: tr = 4, det = 3 ⇒ λ² − 4λ + 3 = 0 ⇒ λ = 3 and λ = 1, with eigenvectors [1; 1] and [1; −1].
They are perpendicular: 1·1 + 1·(−1) = 0, which always happens for symmetric matrices.
R: tr = 0, det = 1 ⇒ λ² + 1 = 0: no real roots, λ = ±i.
That makes sense: a rotation by 90° turns every vector, so there is no real eigen-direction (complex numbers are the next lesson).

Diagonalisation

Suppose an n × n matrix A has n linearly independent eigenvectors v₁, …, vₙ. Put them as the columns of a matrix P and the eigenvalues on the diagonal of D. The equations Avᵢ = λᵢvᵢ, taken together, say AP = PD, hence A = PDP⁻¹. This is always possible when the n eigenvalues are distinct, and a real symmetric matrix can always be diagonalised with perpendicular eigenvectors.

A = P · D · P⁻¹Aᵏ = P · Dᵏ · P⁻¹

Where

P
the matrix whose columns are the eigenvectors
D
the diagonal matrix of the eigenvalues, in the same order
Dᵏ
the diagonal matrix with λ₁ᵏ, …, λₙᵏ
k
a natural number

Powers become easy: in Aᵏ = PDP⁻¹ · PDP⁻¹ · … every P⁻¹P in the middle cancels.

The tenth power without ten multiplications

Problem

Diagonalise A = [4 1; 2 3] and find a formula for Aⁿ.

Show solutionHide solution

Solution

From the example above: P = [1 1; 1 −2], D = [5 0; 0 2].
det P = 1·(−2) − 1·1 = −3, P⁻¹ = (1/3)·[2 1; 1 −1] (the 2 × 2 inverse shortcut).
Check: PDP⁻¹ = [5 2; 5 −4] · (1/3)[2 1; 1 −1] = (1/3)[12 3; 6 9] = [4 1; 2 3] ✓.
Aⁿ = P·[5ⁿ 0; 0 2ⁿ]·P⁻¹ = (1/3)·[2·5ⁿ + 2ⁿ 5ⁿ − 2ⁿ; 2·5ⁿ − 2·2ⁿ 5ⁿ + 2·2ⁿ].
n = 3: (1/3)·[258 117; 234 141] = [86 39; 78 47], the same as A·A·A.
For n = 10 the formula gives [6510758 3254867; 6509734 3255891] at once.

Applications: Markov chains, PageRank, PCA

A Markov chain describes a system that jumps between states with fixed probabilities. Suppose two cafés near a university campus, A and B, share the students: each week 80% of A's customers stay loyal and 20% move to B, while 30% of B's customers move to A and 70% stay. Writing the shares as a column xₖ = [aₖ; bₖ], we get xₖ₊₁ = Txₖ, and every column of T sums to 1.

xₖ₊₁ = T · xₖT · x* = 1 · x*

Where

T
the transition matrix: tᵢⱼ is the probability of moving from state j to state i
xₖ
the shares (probabilities) after k steps
x*
the stationary distribution: the eigenvector for λ = 1 whose elements sum to 1

Where do the customers end up?

Problem

T = [0.8 0.3; 0.2 0.7]. Find the long-run shares of the two cafés and show how fast they are reached from x₀ = [0.5; 0.5].

Show solutionHide solution

Solution

λ = 1: (T − I)x = 0 ⇒ −0.2a + 0.3b = 0 ⇒ a = 1.5b.
Since a + b = 1: 2.5b = 1 ⇒ b = 0.4, a = 0.6. In the long run: 60% and 40%.
The second eigenvalue: λ₁ + λ₂ = tr T = 1.5 ⇒ λ₂ = 0.5 (check: 1 · 0.5 = det T = 0.56 − 0.06 = 0.5 ✓).
λ₂ = 0.5 means the distance to equilibrium halves every week: x₁ = [0.55; 0.45], x₂ = [0.575; 0.425], x₃ = [0.5875; 0.4125] → [0.6; 0.4].

PageRank. Google's original idea treats the web as a giant Markov chain: a random surfer keeps clicking links, and the importance of a page is its share in the stationary distribution, the eigenvector of the “Google matrix” for λ = 1. With billions of pages this vector is found by power iteration: start with any vector and multiply it by the matrix again and again. PCA (principal component analysis). The covariance matrix of a data set is symmetric; its eigenvector with the largest eigenvalue points in the direction in which the data spread most. Keeping only a few such principal components compresses data, for example in face recognition.

Python
import numpy as np

A = np.array([[4, 1], [2, 3]])
values, vectors = np.linalg.eig(A)
print(values)
print(np.linalg.matrix_power(A, 10))

T = np.array([[0.8, 0.3], [0.2, 0.7]])
x = np.array([0.5, 0.5])
for week in range(30):
    x = T @ x
print(np.round(x, 4))
Expected output
[5. 2.]
[[6510758 3254867]
 [6509734 3255891]]
[0.6 0.4]
eig returns the eigenvalues (and eigenvectors of length 1). matrix_power confirms A¹⁰ from the diagonalisation example. The loop is power iteration for the café chain: after 30 weeks the shares are 0.6 and 0.4.

Remember

  • Av = λv with v ≠ 0: A does not turn an eigenvector, it only stretches it by λ.
  • Eigenvalues are the roots of det(A − λI) = 0; for 2 × 2, λ² − (tr A)λ + det A = 0.
  • The sum of the eigenvalues is the trace and their product is the determinant; for a triangular matrix they sit on the diagonal.
  • Eigenvectors come from (A − λI)v = 0; the rows must turn out proportional.
  • With n independent eigenvectors, A = PDP⁻¹ and Aᵏ = PDᵏP⁻¹.
  • The stationary distribution of a Markov chain is the eigenvector for λ = 1; the next eigenvalue shows how fast it is reached.
Spotted a mistake or something inaccurate in this lesson?

Check yourself

10 questions. Every correct answer earns XP.

Question 1 / 10

Question 1 / 10. What are the eigenvalues of [5 2; 2 2]?