Matrices and matrix operations

Matrix notation, addition, multiplication by a number and by another matrix, the identity and the transpose; matrices as transformations of the plane; systems written as Ax = b, Gaussian elimination and the rank of a matrix.

  • University
  • 30 min
  • 10 questions
  • Lesson 86 of 92

In this lesson you will learn

  • Add matrices, multiply them by a number and by each other, and transpose them
  • Describe rotations and scalings of the plane with matrices
  • Write a linear system as Ax = b and solve it by Gaussian elimination
  • Find the rank of a matrix and use it to count the solutions of a system

When a character in a video game turns around, the computer rotates thousands of points at once: it multiplies each of them by the same small table of four numbers. A shop's weekly sales by product and day, the pixels of a photo and the coefficients of a system of equations are all such tables. In the lesson on systems of linear equations we solved two equations in two unknowns; matrices let us handle three, a hundred or a million unknowns with one systematic method.

Matrix notation and types

MatrixDefinition

A rectangular table of numbers with m rows and n columns is called an m × n matrix. The number in row i and column j is the element aᵢⱼ; we write A = (aᵢⱼ).

Matrices are named with capital letters. In running text we write a matrix row by row, separating the rows with semicolons: A = [2 −1 0; 4 3 5] is a 2 × 3 matrix whose first row is 2, −1, 0. The first index is always the row and the second the column: here a₁₂ = −1 and a₂₃ = 5.

Aj = 1j = 2j = 3
i = 12−10
i = 2435
The 2 × 3 matrix A = [2 −1 0; 4 3 5]: the element aᵢⱼ sits in row i and column j.
  • Square matrix: m = n; its main diagonal is a₁₁, a₂₂, …, aₙₙ.
  • Row vector (1 × n) and column vector (m × 1); for example, [3; 1] is a column.
  • Zero matrix O: every element is 0.
  • Identity matrix I: 1 on the main diagonal, 0 elsewhere; I₂ = [1 0; 0 1].
  • Symmetric matrix: aᵢⱼ = aⱼᵢ, that is, Aᵀ = A (the transpose is explained below).

Sign up free to keep reading

The rest of the lesson, the quiz and your notes open with an Educora account. Signing up takes a minute, and your progress is saved on all your devices.

Operations with matrices

Only matrices of the same size can be added: you add the elements in matching positions. To multiply a matrix by a number k, multiply every element by k. These operations behave like ordinary arithmetic: A + B = B + A and k(A + B) = kA + kB.

(A + B)ᵢⱼ = aᵢⱼ + bᵢⱼ(kA)ᵢⱼ = k · aᵢⱼ

Where

A, B
matrices of the same size m × n
k
a number (scalar)
aᵢⱼ, bᵢⱼ
the elements in row i, column j

The matrix product AB is defined only when the number of columns of A equals the number of rows of B. To get the element in row i and column j of AB, run along row i of A and down column j of B, multiply matching elements and add the products.

cᵢⱼ = aᵢ₁b₁ⱼ + aᵢ₂b₂ⱼ + … + aᵢₙbₙⱼ = ∑ₖ aᵢₖ · bₖⱼ

Where

cᵢⱼ
an element of the product C = AB
A
an m × n matrix
B
an n × p matrix
n
the common inner size; the result C is m × p

Row times column: A (m × n) · B (n × p) = C (m × p).

A sum, a scalar multiple and two products

Problem

A = [1 2; 3 4], B = [0 1; 5 −2]. Find A + B, 3A, AB and BA.

Show solutionHide solution

Solution

A + B = [1 3; 8 2].
3A = [3 6; 9 12].
AB, row 1 times the columns: 1·0 + 2·5 = 10, 1·1 + 2·(−2) = −3.
AB, row 2 times the columns: 3·0 + 4·5 = 20, 3·1 + 4·(−2) = −5.
AB = [10 −3; 20 −5].
BA: 0·1 + 1·3 = 3, 0·2 + 1·4 = 4, 5·1 + (−2)·3 = −1, 5·2 + (−2)·4 = 2.
BA = [3 4; −1 2], so AB ≠ BA.

Other familiar rules do hold: (AB)C = A(BC) (associativity), A(B + C) = AB + AC (distributivity) and AI = IA = A, so the identity matrix plays the role of the number 1. The transpose Aᵀ turns rows into columns: the transpose of [2 −1 0; 4 3 5] is the 3 × 2 matrix [2 4; −1 3; 0 5].

(Aᵀ)ᵢⱼ = aⱼᵢ(AB)ᵀ = Bᵀ · Aᵀ

Where

Aᵀ
the transpose of A (n × m if A is m × n)
aⱼᵢ
the element of A in row j, column i

Transposing a product reverses the order of the factors, like socks and shoes: what goes on last comes off first.

Matrices as transformations of the plane

Multiplying a 2 × 2 matrix by a column vector [x; y] gives a new vector, so the matrix transforms the plane. Such a map is linear: the origin stays in place and straight lines go to straight lines. The key observation: the first column of the matrix is the image of [1; 0] and the second column is the image of [0; 1]. Once you know where the two basis vectors go, you know the whole transformation.

R(θ) = [cos θ −sin θ; sin θ cos θ]S = [k₁ 0; 0 k₂]

Where

R(θ)
rotation about the origin by the angle θ anticlockwise
S
scaling: stretching by k₁ along the x-axis and by k₂ along the y-axis
k₁, k₂
scale factors (a negative value also reflects)

Rotation by 90°

Problem

Rotate the point (3, 1) by 90° anticlockwise about the origin. Then find the matrix of two such rotations in a row.

Show solutionHide solution

Solution

cos 90° = 0 and sin 90° = 1, so R = [0 −1; 1 0].
R · [3; 1] = [0·3 + (−1)·1; 1·3 + 0·1] = [−1; 3]. The image is (−1, 3).
Check: the length is unchanged, √(3² + 1²) = √10 = √((−1)² + 3²).
Two rotations: R · R = [0·0 + (−1)·1 0·(−1) + (−1)·0; 1·0 + 0·1 1·(−1) + 0·0] = [−1 0; 0 −1].
This is R(180°), a rotation by 180°, just as expected.

Applying transformation B first and then A is the same as multiplying by the single matrix AB: a product is read from right to left. This is exactly why matrix multiplication is defined as row times column. It also explains non-commutativity geometrically: with S = [2 0; 0 1] we get SR = [0 −2; 1 0] but RS = [0 −1; 2 0]. Rotating and then stretching is not the same as stretching and then rotating.

Systems as Ax = b: Gaussian elimination and rank

Consider the system
2x + y − z = 8
−3x − y + 2z = −11
−2x + y + 2z = −3
We write it compactly as Ax = b, where A = [2 1 −1; −3 −1 2; −2 1 2] is the coefficient matrix, x = [x; y; z] is the column of unknowns and b = [8; −11; −3] is the column of right-hand sides. Adding b as an extra column gives the augmented matrix [A | b].

A · x = b

Where

A
the m × n coefficient matrix
x
the column of n unknowns
b
the column of right-hand sides
[A | b]
the augmented matrix

Gaussian elimination uses three elementary row operations, none of which changes the set of solutions:

  1. Swap two rows.
  2. Multiply a row by a non-zero number.
  3. Add a multiple of one row to another row.

The goal is row echelon form: each row starts with more zeros than the row above, and the first non-zero element of a row is called a pivot. Then the unknowns are found from the bottom up (back substitution). Below, Rᵢ denotes row i.

Gaussian elimination, 3 × 3

Problem

Solve the system above by Gaussian elimination.

Show solutionHide solution

Solution

[A | b]:
R₁: 2 1 −1 | 8
R₂: −3 −1 2 | −11
R₃: −2 1 2 | −3
R₂ → 2R₂ + 3R₁: 0 1 1 | 2 (this way no fractions appear).
R₃ → R₃ + R₁: 0 2 1 | 5.
R₃ → R₃ − 2R₂: 0 0 −1 | 1.
Back substitution: −z = 1 ⇒ z = −1; y + z = 2 ⇒ y = 3; 2x + y − z = 8 ⇒ 2x + 4 = 8 ⇒ x = 2.
Answer: (x, y, z) = (2, 3, −1). Check in the second equation: −6 − 3 − 2 = −11 ✓.
xyzb
21−18
0112
00−11
Row echelon form of [A | b]: the pivots are 2, 1 and −1. There are three pivots, so r(A) = 3.

Rank of a matrixDefinition

The number of non-zero rows (pivots) in a row echelon form of the matrix. It equals the maximal number of linearly independent rows and, remarkably, also of linearly independent columns. Notation: r(A) or rank A.

r(A) = r(A | b)

Where

r(A)
the rank of the coefficient matrix
r(A | b)
the rank of the augmented matrix
n
the number of unknowns

Rouché–Capelli theorem: the system has a solution exactly when the two ranks are equal. Then r(A) = n gives a unique solution and r(A) < n gives infinitely many (with n − r(A) free unknowns). If r(A) < r(A | b), a row of the form 0 = c with c ≠ 0 appears and there is no solution.

Python
import numpy as np

A = np.array([[1, 2], [3, 4]])
B = np.array([[0, 1], [5, -2]])
print(A @ B)
print(B @ A)

M = np.array([[2, 1, -1], [-3, -1, 2], [-2, 1, 2]])
b = np.array([8, -11, -3])
print(np.linalg.solve(M, b))
print(np.linalg.matrix_rank(M))
Expected output
[[10 -3]
 [20 -5]]
[[ 3  4]
 [-1  2]]
[ 2.  3. -1.]
3
numpy checks the examples: @ is the matrix product, solve finds x in Ax = b and matrix_rank gives the rank. Careful: in numpy A * B multiplies element by element, which is not the matrix product.

Remember

  • An m × n matrix has m rows and n columns; aᵢⱼ is in row i, column j.
  • AB exists when the columns of A match the rows of B; each element is row times column, and the result is m × p.
  • In general AB ≠ BA, but (AB)C = A(BC), AI = IA = A and (AB)ᵀ = BᵀAᵀ.
  • A 2 × 2 matrix is a linear transformation; its columns are the images of [1; 0] and [0; 1].
  • Gaussian elimination brings [A | b] to row echelon form; the number of pivots is the rank.
  • A system has a solution only if r(A) = r(A | b); it is unique when this rank equals the number of unknowns.
Spotted a mistake or something inaccurate in this lesson?

Check yourself

10 questions. Every correct answer earns XP.

Question 1 / 10

Question 1 / 10. A = [1 2; 0 1], B = [3 0; 1 2]. What is AB?