Machine learning concepts

Types of machine learning, features and labels, data splits, loss functions, gradient descent, overfitting and the bias–variance trade-off — with runnable NumPy examples.

  • University
  • 25 min
  • 10 questions
  • Lesson 60 of 66

In this lesson you will learn

  • Distinguish supervised, unsupervised and reinforcement learning and explain features and labels
  • Compute MSE and cross-entropy losses by hand and with NumPy
  • Train a linear regression model with gradient descent and explain the effect of the learning rate
  • Diagnose overfitting and underfitting from training and validation losses

Imagine you must write a program that predicts the price of a flat in Baku from its area, number of rooms and district. Writing all the rules by hand is impossible: conditions like “if the district is Nasimi and the area is over 80 m²...” never end. Machine learning offers another way: we give the computer thousands of examples of flats that were sold, and it learns the rule from the data by itself. This lesson is the mathematical foundation of the whole module: neural networks, CNNs and transformers all rest on it.

Machine learningDefinition

A program learns if its performance on some task, measured by a chosen metric, improves with experience — that is, with data. A model is a function with parameters, and learning means choosing those parameters from data.

Types of learning, features and labels

Each example is described by a vector of features x: for a flat, its area, rooms and floor. The quantity we want to predict is the label y: the price. With n examples and d features, the data is stored as an n × d matrix X and a vector y of length n. If the label is a number, the task is regression; if it is a class (spam / not spam), it is classification.

TypeDataGoalExample
Supervised learning(x, y) pairspredict y for a new xflat prices, spam filters, finding disease on an X-ray
Unsupervised learningonly x, no labelsfind structure in the datagrouping customers (clustering), dimensionality reduction
Reinforcement learninginteraction with an environment and rewardsmaximise the total rewarda game-playing agent, a robot learning to walk

Sign up free to keep reading

The rest of the lesson, the quiz and your notes open with an Educora account. Signing up takes a minute, and your progress is saved on all your devices.

Splitting the data: training, validation and test sets

The real test of a model is data it has never seen. So we shuffle the data randomly and split it into three parts (a typical ratio is 70/15/15 or 80/10/10):

  • Training set — the parameters are fitted on this data only.
  • Validation set — used to choose hyperparameters (learning rate, model size) and to decide when to stop training.
  • Test set — used once, at the very end: an unbiased estimate of real-world performance.
Python
import numpy as np

rng = np.random.default_rng(42)
idx = rng.permutation(10)
train, val, test = idx[:6], idx[6:8], idx[8:]
print('train:', train)
print('val:  ', val)
print('test: ', test)
Expected output
train: [5 6 0 7 3 2]
val:   [4 9]
test:  [1 8]
We shuffle the indices of 10 examples and split them 60/20/20. The seed is fixed, so the split is the same every time.

Loss functions

We need a single number that measures how wrong the model is — this number is given by the loss function L. Learning means searching for the parameters that minimise the loss. Regression usually uses mean squared error; classification uses cross-entropy.

MSE = (1/n) · ∑ᵢ₌₁ⁿ (yᵢ − ŷᵢ)²MSE = (1/n) · ∑ᵢ₌₁ⁿ (yᵢ − ŷᵢ)²

Where

n
number of examples
yᵢ
true value (label) of example i
ŷᵢ
the model's prediction

Mean squared error. Squaring punishes large errors much more; the unit is the square of the label's unit.

CE = −∑ₖ₌₁ᴷ yₖ · ln pₖ = −ln pₜ

Where

K
number of classes
yₖ
1 if class k is the correct one, otherwise 0 (one-hot)
pₖ
probability the model gives to class k
pₜ
probability of the true class

Cross-entropy. For two classes: BCE = −[y · ln p + (1 − y) · ln(1 − p)]. As the probability of the true class approaches 1, the loss approaches 0.

Example 1: computing losses by hand

Problem

a) True values y = (3, 5, 2.5), predictions ŷ = (2.5, 5, 3.5). Find the MSE.
b) An image classifier gives p = (0.7, 0.2, 0.1) for the classes “cat, dog, bird”. Find the cross-entropy if the picture shows a cat, and if it shows a bird.

Show solutionHide solution

Solution

a) Errors: 0.5, 0, −1. Squares: 0.25, 0, 1. Sum 1.25.
MSE = 1.25 / 3 ≈ 0.4167.
b) Cat: CE = −ln 0.7 ≈ 0.357 — the model was confident and right, so the loss is small.
Bird: CE = −ln 0.1 ≈ 2.303 — only 10% went to the true class, and the loss is more than 6 times larger.
Python
import numpy as np

y = np.array([3.0, 5.0, 2.5])
y_hat = np.array([2.5, 5.0, 3.5])
mse = np.mean((y - y_hat) ** 2)
print(f'MSE = {mse:.4f}')

p = np.array([0.7, 0.2, 0.1])
print(f'CE (true class 0) = {-np.log(p[0]):.4f}')
print(f'CE (true class 2) = {-np.log(p[2]):.4f}')
Expected output
MSE = 0.4167
CE (true class 0) = 0.3567
CE (true class 2) = 2.3026

Gradient descent and the learning rate

Imagine walking down a mountain in fog: you cannot see far, but you can feel the slope under your feet. You take a small step in the steepest downhill direction and repeat. The gradient ∇L points in the direction in which the loss grows fastest, so we move against it. The step length is set by the learning rate η.

θ ← θ − η · ∇L(θ)

Where

θ
all parameters of the model (weights and biases)
η
learning rate, usually 0.0001–0.1
∇L(θ)
gradient: the vector of partial derivatives of the loss with respect to each parameter

The gradient descent update rule. Each repetition is one step; one full pass over the training set is called an epoch.

In linear regression ŷᵢ = w · xᵢ + b and L = MSE. By the chain rule, the derivative of (ŷᵢ − yᵢ)² with respect to w is 2(ŷᵢ − yᵢ) · xᵢ, and with respect to b it is 2(ŷᵢ − yᵢ). Averaging over all examples gives:

∂L/∂w = (2/n) · ∑ᵢ (ŷᵢ − yᵢ) · xᵢ∂L/∂b = (2/n) · ∑ᵢ (ŷᵢ − yᵢ)∂L/∂w = (2/n) · ∑ᵢ (ŷᵢ − yᵢ) · xᵢ∂L/∂b = (2/n) · ∑ᵢ (ŷᵢ − yᵢ)

Where

w, b
slope (weight) and intercept (bias) of the line
ŷᵢ − yᵢ
error on example i

Example 2: one step of gradient descent

Problem

Model ŷ = w · x, a single example (x = 2, y = 6), start with w = 1, L = (ŷ − y)². Take one step with η = 0.1 and with η = 0.3 and find the new loss.

Show solutionHide solution

Solution

ŷ = 2, L = (2 − 6)² = 16.
dL/dw = 2(ŷ − y) · x = 2 · (−4) · 2 = −16.
η = 0.1: w = 1 − 0.1 · (−16) = 2.6; ŷ = 5.2; L = 0.64 — the loss fell 25 times.
η = 0.3: w = 1 + 4.8 = 5.8; ŷ = 11.6; L = 31.36 — we jumped over the minimum (w = 3) and the loss grew!
Here L(w) = 4(w − 3)², whose second derivative is 8; for a quadratic loss, descent converges only if η < 2/8 = 0.25.

Now let us apply the same idea to 50 synthetic points. The data is generated from the line y = 3x + 4 plus noise; the model must find w and b starting from zero, knowing nothing. Run the code — the loss curve is shown automatically.

Python
import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng(42)
x = rng.uniform(0, 5, size=50)
y = 3 * x + 4 + rng.normal(0, 1, size=50)

w, b, lr = 0.0, 0.0, 0.05
losses = []
for epoch in range(500):
    error = (w * x + b) - y
    losses.append(np.mean(error ** 2))
    dw = 2 * np.mean(error * x)
    db = 2 * np.mean(error)
    w -= lr * dw
    b -= lr * db
    if epoch % 100 == 0:
        print(f'epoch {epoch:3d}  loss {losses[-1]:.4f}')

print(f'w = {w:.3f}, b = {b:.3f}')
plt.plot(losses)
plt.yscale('log')
plt.xlabel('epoch')
plt.ylabel('MSE loss')
plt.show()
Expected output
epoch   0  loss 158.9249
epoch 100  loss 0.6019
epoch 200  loss 0.5740
epoch 300  loss 0.5735
epoch 400  loss 0.5735
w = 3.024, b = 3.769
The model found w ≈ 3.02 and b ≈ 3.77. They are not exactly 3 and 4 because this is the best line (the least-squares solution) for these 50 noisy points; the loss also approaches the noise level, not 0.

In practice the gradient is not computed on the whole dataset at every step but on a small mini-batch of size B (for example B = 32). This is stochastic gradient descent (SGD): the steps are noisier but much cheaper, and the noise sometimes even helps escape poor local minima.

Overfitting, underfitting and the bias–variance trade-off

A model that is too simple cannot capture the pattern in the data — this is underfitting. A model that is too complex memorises the random noise together with the pattern: its training error is almost zero, but its error on new data is large — this is overfitting. The code below fits polynomials of different degrees to 15 noisy points from sin(2πx):

Python
import numpy as np
from numpy.polynomial import Polynomial

rng = np.random.default_rng(1)
x_train = np.sort(rng.uniform(0, 1, 15))
x_val = np.sort(rng.uniform(0, 1, 15))
f = lambda x: np.sin(2 * np.pi * x)
y_train = f(x_train) + rng.normal(0, 0.2, 15)
y_val = f(x_val) + rng.normal(0, 0.2, 15)

for deg in [1, 3, 9, 12]:
    model = Polynomial.fit(x_train, y_train, deg)
    train_mse = np.mean((model(x_train) - y_train) ** 2)
    val_mse = np.mean((model(x_val) - y_val) ** 2)
    print(f'degree {deg:2d}: train {train_mse:.3f}, validation {val_mse:.3f}')
Expected output
degree  1: train 0.264, validation 0.235
degree  3: train 0.032, validation 0.037
degree  9: train 0.008, validation 1.894
degree 12: train 0.005, validation 67.532
Degree 1 underfits (both losses are high). Degree 3 is the best. Degrees 9 and 12: the training loss keeps falling while the validation loss explodes.
E[(y − f̂(x))²] = Bias[f̂(x)]² + Var[f̂(x)] + σ²

Where

f̂(x)
prediction of a model trained on a random training set
Bias
bias: the average error caused by the model's too-simple assumptions
Var
variance: how much the prediction depends on the particular training set
σ²
irreducible noise in the data

The bias–variance decomposition of the MSE. As a model becomes more complex, bias falls and variance grows; the best model minimises the sum.

Training lossValidation lossDiagnosisWhat to do
highhighunderfitting (high bias)bigger model, more features, train longer
lowmuch higheroverfitting (high variance)more data, regularisation, early stopping, simpler model
lowclose to itgood fitevaluate once on the test set
J(θ) = L(θ) + λ · ∑ⱼ θⱼ²

Where

λ
regularisation strength (a hyperparameter chosen on the validation set)
∑ⱼ θⱼ²
sum of squared weights — penalises large weights

L2 regularisation (weight decay): the model prefers “smoother” functions and is less inclined to memorise noise.

Remember

  • Supervised learning learns from (x, y) pairs: regression if the label is a number, classification if it is a class.
  • The training set is for parameters, the validation set for hyperparameters; the test set is used only at the very end.
  • Regression uses MSE = (1/n)∑(y − ŷ)²; classification uses cross-entropy CE = −ln pₜ.
  • Gradient descent: θ ← θ − η · ∇L; a too-large η makes the loss grow, a too-small η makes learning crawl.
  • Low training loss with high validation loss means overfitting; both high means underfitting.
Spotted a mistake or something inaccurate in this lesson?

Check yourself

10 questions. Every correct answer earns XP.

Question 1 / 10

Question 1 / 10. Which task is supervised regression?