When a character in a video game turns around, the computer rotates thousands of points at once: it multiplies each of them by the same small table of four numbers. A shop's weekly sales by product and day, the pixels of a photo and the coefficients of a system of equations are all such tables. In the lesson on systems of linear equations we solved two equations in two unknowns; matrices let us handle three, a hundred or a million unknowns with one systematic method.
Matrix notation and types
MatrixDefinition
A rectangular table of numbers with m rows and n columns is called an m × n matrix. The number in row i and column j is the element aᵢⱼ; we write A = (aᵢⱼ).
Matrices are named with capital letters. In running text we write a matrix row by row, separating the rows with semicolons: A = [2 −1 0; 4 3 5] is a 2 × 3 matrix whose first row is 2, −1, 0. The first index is always the row and the second the column: here a₁₂ = −1 and a₂₃ = 5.
| A | j = 1 | j = 2 | j = 3 |
|---|---|---|---|
| i = 1 | 2 | −1 | 0 |
| i = 2 | 4 | 3 | 5 |
- Square matrix: m = n; its main diagonal is a₁₁, a₂₂, …, aₙₙ.
- Row vector (1 × n) and column vector (m × 1); for example, [3; 1] is a column.
- Zero matrix O: every element is 0.
- Identity matrix I: 1 on the main diagonal, 0 elsewhere; I₂ = [1 0; 0 1].
- Symmetric matrix: aᵢⱼ = aⱼᵢ, that is, Aᵀ = A (the transpose is explained below).
Operations with matrices
Only matrices of the same size can be added: you add the elements in matching positions. To multiply a matrix by a number k, multiply every element by k. These operations behave like ordinary arithmetic: A + B = B + A and k(A + B) = kA + kB.
Where
- A, B
- matrices of the same size m × n
- k
- a number (scalar)
- aᵢⱼ, bᵢⱼ
- the elements in row i, column j
The matrix product AB is defined only when the number of columns of A equals the number of rows of B. To get the element in row i and column j of AB, run along row i of A and down column j of B, multiply matching elements and add the products.
Where
- cᵢⱼ
- an element of the product C = AB
- A
- an m × n matrix
- B
- an n × p matrix
- n
- the common inner size; the result C is m × p
Row times column: A (m × n) · B (n × p) = C (m × p).
A sum, a scalar multiple and two products
Problem
A = [1 2; 3 4], B = [0 1; 5 −2]. Find A + B, 3A, AB and BA.
Show solutionHide solution
Solution
3A = [3 6; 9 12].
AB, row 1 times the columns: 1·0 + 2·5 = 10, 1·1 + 2·(−2) = −3.
AB, row 2 times the columns: 3·0 + 4·5 = 20, 3·1 + 4·(−2) = −5.
AB = [10 −3; 20 −5].
BA: 0·1 + 1·3 = 3, 0·2 + 1·4 = 4, 5·1 + (−2)·3 = −1, 5·2 + (−2)·4 = 2.
BA = [3 4; −1 2], so AB ≠ BA.
Other familiar rules do hold: (AB)C = A(BC) (associativity), A(B + C) = AB + AC (distributivity) and AI = IA = A, so the identity matrix plays the role of the number 1. The transpose Aᵀ turns rows into columns: the transpose of [2 −1 0; 4 3 5] is the 3 × 2 matrix [2 4; −1 3; 0 5].
Where
- Aᵀ
- the transpose of A (n × m if A is m × n)
- aⱼᵢ
- the element of A in row j, column i
Transposing a product reverses the order of the factors, like socks and shoes: what goes on last comes off first.
Matrices as transformations of the plane
Multiplying a 2 × 2 matrix by a column vector [x; y] gives a new vector, so the matrix transforms the plane. Such a map is linear: the origin stays in place and straight lines go to straight lines. The key observation: the first column of the matrix is the image of [1; 0] and the second column is the image of [0; 1]. Once you know where the two basis vectors go, you know the whole transformation.
Where
- R(θ)
- rotation about the origin by the angle θ anticlockwise
- S
- scaling: stretching by k₁ along the x-axis and by k₂ along the y-axis
- k₁, k₂
- scale factors (a negative value also reflects)
Rotation by 90°
Problem
Rotate the point (3, 1) by 90° anticlockwise about the origin. Then find the matrix of two such rotations in a row.
Show solutionHide solution
Solution
R · [3; 1] = [0·3 + (−1)·1; 1·3 + 0·1] = [−1; 3]. The image is (−1, 3).
Check: the length is unchanged, √(3² + 1²) = √10 = √((−1)² + 3²).
Two rotations: R · R = [0·0 + (−1)·1 0·(−1) + (−1)·0; 1·0 + 0·1 1·(−1) + 0·0] = [−1 0; 0 −1].
This is R(180°), a rotation by 180°, just as expected.
Applying transformation B first and then A is the same as multiplying by the single matrix AB: a product is read from right to left. This is exactly why matrix multiplication is defined as row times column. It also explains non-commutativity geometrically: with S = [2 0; 0 1] we get SR = [0 −2; 1 0] but RS = [0 −1; 2 0]. Rotating and then stretching is not the same as stretching and then rotating.
Systems as Ax = b: Gaussian elimination and rank
Consider the system
2x + y − z = 8
−3x − y + 2z = −11
−2x + y + 2z = −3
We write it compactly as Ax = b, where A = [2 1 −1; −3 −1 2; −2 1 2] is the coefficient matrix, x = [x; y; z] is the column of unknowns and b = [8; −11; −3] is the column of right-hand sides. Adding b as an extra column gives the augmented matrix [A | b].
Where
- A
- the m × n coefficient matrix
- x
- the column of n unknowns
- b
- the column of right-hand sides
- [A | b]
- the augmented matrix
Gaussian elimination uses three elementary row operations, none of which changes the set of solutions:
- Swap two rows.
- Multiply a row by a non-zero number.
- Add a multiple of one row to another row.
The goal is row echelon form: each row starts with more zeros than the row above, and the first non-zero element of a row is called a pivot. Then the unknowns are found from the bottom up (back substitution). Below, Rᵢ denotes row i.
Gaussian elimination, 3 × 3
Problem
Solve the system above by Gaussian elimination.
Show solutionHide solution
Solution
R₁: 2 1 −1 | 8
R₂: −3 −1 2 | −11
R₃: −2 1 2 | −3
R₂ → 2R₂ + 3R₁: 0 1 1 | 2 (this way no fractions appear).
R₃ → R₃ + R₁: 0 2 1 | 5.
R₃ → R₃ − 2R₂: 0 0 −1 | 1.
Back substitution: −z = 1 ⇒ z = −1; y + z = 2 ⇒ y = 3; 2x + y − z = 8 ⇒ 2x + 4 = 8 ⇒ x = 2.
Answer: (x, y, z) = (2, 3, −1). Check in the second equation: −6 − 3 − 2 = −11 ✓.
| x | y | z | b |
|---|---|---|---|
| 2 | 1 | −1 | 8 |
| 0 | 1 | 1 | 2 |
| 0 | 0 | −1 | 1 |
Rank of a matrixDefinition
The number of non-zero rows (pivots) in a row echelon form of the matrix. It equals the maximal number of linearly independent rows and, remarkably, also of linearly independent columns. Notation: r(A) or rank A.
Where
- r(A)
- the rank of the coefficient matrix
- r(A | b)
- the rank of the augmented matrix
- n
- the number of unknowns
Rouché–Capelli theorem: the system has a solution exactly when the two ranks are equal. Then r(A) = n gives a unique solution and r(A) < n gives infinitely many (with n − r(A) free unknowns). If r(A) < r(A | b), a row of the form 0 = c with c ≠ 0 appears and there is no solution.
import numpy as np
A = np.array([[1, 2], [3, 4]])
B = np.array([[0, 1], [5, -2]])
print(A @ B)
print(B @ A)
M = np.array([[2, 1, -1], [-3, -1, 2], [-2, 1, 2]])
b = np.array([8, -11, -3])
print(np.linalg.solve(M, b))
print(np.linalg.matrix_rank(M))Expected output
[[10 -3] [20 -5]] [[ 3 4] [-1 2]] [ 2. 3. -1.] 3
@ is the matrix product, solve finds x in Ax = b and matrix_rank gives the rank. Careful: in numpy A * B multiplies element by element, which is not the matrix product.Remember
- An m × n matrix has m rows and n columns; aᵢⱼ is in row i, column j.
- AB exists when the columns of A match the rows of B; each element is row times column, and the result is m × p.
- In general AB ≠ BA, but (AB)C = A(BC), AI = IA = A and (AB)ᵀ = BᵀAᵀ.
- A 2 × 2 matrix is a linear transformation; its columns are the images of [1; 0] and [0; 1].
- Gaussian elimination brings [A | b] to row echelon form; the number of pivots is the rank.
- A system has a solution only if r(A) = r(A | b); it is unique when this rank equals the number of unknowns.