Image processing with Pillow and OpenCV

An image is a NumPy array of pixels: coordinates, channels and the grayscale formula; opening, resizing, cropping, drawing and saving with Pillow; BGR, colour spaces, blur, thresholds, edges and contours with OpenCV — with a coin-counting project.

  • University
  • 22 min
  • 12 questions
  • Lesson 59 of 66

In this lesson you will learn

  • Read an image as an (H, W, C) array, tell (x, y) from [y, x] and compute brightness
  • Open, shrink with the aspect ratio kept, crop, rotate, draw on and save images with Pillow
  • Convert colour spaces in OpenCV and explain the blur, threshold, edge and contour steps
  • Find, count and measure objects in an image

For a computer an image is a table of numbers: every point — a pixel — is a few numbers. Shrinking a hundred photos for a website, adding a watermark, counting coins in a picture — all of these are operations on those numbers. Pillow (PIL) is for opening, resizing, drawing on and saving files, while OpenCV is a computer vision library: filters, edges, contours, finding objects. Both can turn an image into a NumPy array, so what you learned in «NumPy basics: arrays» works here directly.

An image is an array of numbers

Pixel and channelDefinition

A pixel is the smallest point of an image. A colour pixel has three channels — red, green, blue (RGB), each a uint8 from 0 to 255. A grayscale image has one channel: 0 is black, 255 white. The array’s shape is (H, W, C): height, width, channels.

x — columns (width)xy(0, 0)[1, 4]y — rows (height), grows downwardsimg[1, 4]: row y = 1, column x = 4shape = (4, 6, 3): height, width, channels
Pillow writes coordinates as (x, y), while NumPy and OpenCV arrays are indexed as [row, column] = [y, x].
L = 0.299 · R + 0.587 · G + 0.114 · B

Where

R, G, B
the pixel’s red, green and blue values (0–255)
L
brightness in grayscale (0–255)

Pillow’s convert('L') and OpenCV’s COLOR_BGR2GRAY use this formula (ITU-R 601). The eye is most sensitive to green, so green has the largest weight.

Python
import numpy as np
import matplotlib.pyplot as plt

img = np.zeros((4, 6, 3), dtype=np.uint8)
img[:, :3] = [255, 0, 0]
img[:, 3:] = [0, 0, 255]
img[1, 4] = [255, 255, 255]
print(img.shape, img.dtype, img.size)
print(img[1, 4], img[0, 0])

r, g, b = img[..., 0], img[..., 1], img[..., 2]
gray = (0.299 * r + 0.587 * g + 0.114 * b).round().astype(np.uint8)
print(gray)

plt.imshow(img)
plt.title('6 × 4 pixels')
Expected output
(4, 6, 3) uint8 72
[255 255 255] [255   0   0]
[[ 76  76  76  29  29  29]
 [ 76  76  76  29 255  29]
 [ 76  76  76  29  29  29]
 [ 76  76  76  29  29  29]]
This code runs in the browser: the image is just a uint8 array, and matplotlib shows it. img[1, 4] is row 1, column 4; in grayscale red becomes 76, blue 29 and white 255.

Example 1: brightness by hand

Problem

Find the gray brightness of 1) red (255, 0, 0); 2) green (0, 255, 0); 3) orange (200, 120, 40).

Show solutionHide solution

Solution

1) 0.299 · 255 ≈ 76.2 → 76.
2) 0.587 · 255 ≈ 149.7 → 150 — of colours with the same intensity, green looks the lightest.
3) 59.8 + 70.44 + 4.56 = 134.8 → 135 (the Pillow code below also gives 135).

Sign up free to keep reading

The rest of the lesson, the quiz and your notes open with an Educora account. Signing up takes a minute, and your progress is saved on all your devices.

Pillow: open, change, draw, save

Image.open(path) opens a file and Image.new(mode, size, color) creates an empty image; the size is always (width, height). ImageDraw.Draw(img) draws rectangles, ellipses, lines and text on it. save picks the format from the extension: .png is lossless, .jpg is compressed according to quality. Pillow and OpenCV do not run in the browser: pip install pillow opencv-python.

Python
from PIL import Image, ImageDraw

img = Image.new('RGB', (400, 300), color=(240, 240, 255))
draw = ImageDraw.Draw(img)
draw.rectangle((40, 40, 200, 140), fill=(255, 159, 28), outline='black', width=3)
draw.ellipse((220, 60, 360, 200), fill=(31, 182, 255))
draw.line((0, 299, 399, 0), fill='red', width=2)
draw.text((40, 250), 'Educora', fill='black')
img.save('scene.png')

loaded = Image.open('scene.png')
print(loaded.format, loaded.size, loaded.mode)
print(loaded.getpixel((100, 100)), loaded.getpixel((290, 130)), loaded.getpixel((5, 5)))

Expected output

PNG (400, 300) RGB
(255, 159, 28) (31, 182, 255) (240, 240, 255)
A rectangle is given by its box (left, upper, right, lower). The point (100, 100) is inside the orange rectangle and (290, 130) inside the blue circle; the corner (5, 5) has the background colour.
h′ = h · w′/wk = min(Wmax/w, Hmax/h)h′ = h · w′/wk = min(Wmax/w, Hmax/h)

Where

w, h
original width and height (pixels)
w′, h′
the new size
k
the scale for thumbnail((W_max, H_max)): the image must fit into both limits

Resizing while keeping the aspect ratio: resize accepts any size (you compute the ratio), while thumbnail keeps the ratio itself and shrinks the image in place.

Python
from PIL import Image, ImageFilter

photo = Image.new('RGB', (1600, 1200), (200, 120, 40))
w, h = photo.size
small = photo.resize((800, h * 800 // w))
print(small.size)

thumb = photo.copy()
thumb.thumbnail((300, 300))
print(thumb.size)
print(photo.crop((100, 100, 500, 400)).size)
print(photo.rotate(90, expand=True).size, photo.rotate(30).size)

gray = photo.convert('L')
print(gray.mode, gray.getpixel((0, 0)))
soft = small.filter(ImageFilter.GaussianBlur(radius=2))
soft.save('small.jpg', quality=85)
print(Image.open('small.jpg').format, Image.open('small.jpg').size)

Expected output

(800, 600)
(300, 225)
(400, 300)
(1200, 1600) (1600, 1200)
L 135
JPEG (800, 600)
crop returns a new image (400 × 300). rotate(90, expand=True) enlarges the canvas so the dimensions swap; without expand the corners are cut off. GaussianBlur softens the picture, and the JPEG is saved with quality=85.

Example 2: new sizes

Problem

1) A 4000 × 3000 photo is shrunk to a width of 800. 2) A 1600 × 1200 image gets thumbnail((300, 300)). 3) A portrait 1200 × 1600 image gets thumbnail((300, 300)).

Show solutionHide solution

Solution

1) h′ = 3000 · 800 / 4000 = 600 → 800 × 600.
2) k = min(300/1600, 300/1200) = min(0.1875, 0.25) = 0.1875 → 300 × 225 (as the code printed).
3) k = min(0.25, 0.1875) = 0.1875 → 225 × 300.

OpenCV: colour spaces, filters, edges

In OpenCV an image is a NumPy array as well, but the channel order is BGR (blue, green, red). cv2.cvtColor changes the colour space: grayscale (GRAY), RGB, HSV (hue, saturation, value — handy for finding objects by colour). Drawing functions change the array in place, points are given as (x, y), and colours are BGR.

Python
import cv2
import numpy as np

img = np.full((300, 400, 3), 255, dtype=np.uint8)
cv2.rectangle(img, (40, 40), (160, 140), (0, 0, 255), -1)
cv2.circle(img, (280, 150), 60, (255, 0, 0), -1)
cv2.putText(img, 'OpenCV', (40, 260), cv2.FONT_HERSHEY_SIMPLEX, 1.2, (0, 0, 0), 2)
print(img.shape, img.dtype)
print(img[90, 100], img[150, 280])

rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
print(rgb[90, 100], gray.shape, gray[90, 100], hsv[90, 100])
cv2.imwrite('shapes.png', img)
print(cv2.imread('missing.png'))

Expected output

(300, 400, 3) uint8
[  0   0 255] [255   0   0]
[255   0   0] (300, 400) 76 [  0 255 255]
None
(0, 0, 255) is red in OpenCV: [0 0 255] in the array and [255 0 0] after conversion to RGB. In HSV red has hue 0. When a file is missing, cv2.imread raises no exception — it prints a warning and returns None.
Python
import cv2
import numpy as np

a = np.array([[200, 250]], dtype=np.uint8)
b = np.array([[100, 10]], dtype=np.uint8)
print(a + b)
print(cv2.add(a, b))
print(np.clip(a.astype(np.int16) + b, 0, 255).astype(np.uint8))

Expected output

[[44  4]]
[[255 255]]
[[255 255]]
200 + 100 = 300 = 256 + 44, so NumPy stores 44, and 250 + 10 = 260 becomes 4. cv2.add adds with saturation; for the same result in NumPy, switch to a wider type first and use np.clip.

Kernel and filterDefinition

A filter replaces each pixel with a weighted sum of its neighbours; the table of weights is the kernel. cv2.GaussianBlur(img, (5, 5), 0) smooths noise with a 5 × 5 Gaussian kernel (the size must be odd). Canny (cv2.Canny) finds places where brightness changes sharply — edges, and cv2.threshold turns an image into a black-and-white mask.

To see what these steps do, let us perform the same operations with scipy.ndimage, which runs in the browser: noisy image → blur → edges (Sobel) → mask and counting the objects.

Python
import numpy as np
import matplotlib.pyplot as plt
from scipy import ndimage

yy, xx = np.mgrid[0:120, 0:160]
img = np.where((xx - 60)**2 + (yy - 60)**2 < 35**2, 220.0, 60.0)
img[20:50, 110:150] = 220.0
noisy = img + np.random.default_rng(0).normal(0, 25, img.shape)
blurred = ndimage.gaussian_filter(noisy, sigma=2)
edges = np.hypot(ndimage.sobel(blurred, axis=0), ndimage.sobel(blurred, axis=1))
mask = blurred > 140
labels, n = ndimage.label(mask)
print('objects:', n)
print('sizes:', [int(s) for s in ndimage.sum(mask, labels, range(1, n + 1))])

fig, axes = plt.subplots(1, 4, figsize=(10, 2.6))
for ax, im, title in zip(axes, [noisy, blurred, edges, mask], ['noisy', 'blurred', 'edges', 'mask']):
    ax.imshow(im, cmap='gray')
    ax.set_title(title)
    ax.axis('off')
Expected output
objects: 2
sizes: [1190, 3830]
After blurring, the threshold (140) removes the noise and two objects remain: the rectangle (30 × 40 = 1200 pixels, 1190 found) and the circle (π · 35² ≈ 3848, 3830 found).

Mini project: counting coins

We draw four light coins on a dark table, imitate camera noise and find them with OpenCV: count, centre, radius, area. The advantage of a synthetic image is that we know the right answer and can check the algorithm.

  1. Gray and smooth

    cvtColor(…, COLOR_BGR2GRAY), then GaussianBlur — noise speckles disappear.

  2. Mask

    threshold with THRESH_OTSU — the threshold is chosen automatically from the histogram.

  3. Contours

    findContours(…, RETR_EXTERNAL, …) — the outer border of every white blob.

  4. Measure and draw

    contourArea, minEnclosingCircle, boundingRect; mark the result with rectangle.

Python
import cv2
import numpy as np

rng = np.random.default_rng(7)
img = np.full((300, 400, 3), 90, dtype=np.uint8)
for (x, y), r in zip([(80, 80), (220, 90), (320, 210), (130, 220)], [30, 45, 25, 38]):
    cv2.circle(img, (x, y), r, (200, 210, 220), -1)
img = np.clip(img + rng.normal(0, 12, img.shape), 0, 255).astype(np.uint8)

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (5, 5), 0)
t, mask = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
print('threshold:', t, 'objects:', len(contours))
for c in sorted(contours, key=cv2.contourArea, reverse=True):
    x, y, w, h = cv2.boundingRect(c)
    (cx, cy), radius = cv2.minEnclosingCircle(c)
    print(f'center ({cx:.0f}, {cy:.0f})  r = {radius:.0f}  area {cv2.contourArea(c):.0f}  box {w}x{h}')
    cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)
edges = cv2.Canny(blur, 50, 150)
print('edge pixels:', int((edges > 0).sum()))
cv2.imwrite('result.png', img)

Expected output

threshold: 148.0 objects: 4
center (220, 90)  r = 45  area 6230  box 89x89
center (130, 220)  r = 38  area 4402  box 75x75
center (80, 80)  r = 30  area 2732  box 59x59
center (320, 210)  r = 25  area 1886  box 49x49
edge pixels: 1024
Otsu chose the threshold 148 and all four coins were found; the radii match the ones we drew (45, 38, 30, 25). The contour area is slightly smaller than πr² (π · 45² ≈ 6362) because the contour runs through pixel centres.
TaskPillowOpenCV
Open / saveImage.open, img.savecv2.imread, cv2.imwrite
Sizeimg.size → (W, H)img.shape → (H, W, C)
Resizeresize, thumbnailcv2.resize(img, (W, H))
Grayscaleconvert('L')cvtColor(…, COLOR_BGR2GRAY)
DrawImageDraw.Drawcv2.rectangle, cv2.circle, cv2.putText
Convert between themnp.array(img), Image.fromarray(arr)cvtColor(arr, COLOR_RGB2BGR)

Exercise

Convert the 2 × 2 colour image to grayscale with the formula L = 0.299 · R + 0.587 · G + 0.114 · B, round it and print it as a uint8 array.

Exercise · Python
import numpy as np

img = np.array([[[255, 0, 0], [0, 255, 0]],
                [[0, 0, 255], [128, 128, 128]]], dtype=np.uint8)
# take the channels with img[..., 0], img[..., 1], img[..., 2]
Expected output
[[ 76 150]
 [ 29 128]]

Exercise

Do what cv2.boundingRect does with NumPy: find the pixels brighter than 128, then print their count and the enclosing rectangle as (x, y, w, h).

Exercise · Python
import numpy as np

img = np.zeros((8, 8), dtype=np.uint8)
img[3:6, 2:6] = 200
img[4, 3] = 90
mask = img > 128
# np.nonzero(mask) returns (rows, columns) = (ys, xs)
Expected output
pixels: 11
box: (2, 3, 4, 3)

Remember

  • An image is a uint8 array: shape (H, W, C), values 0–255; arrays use img[y, x], Pillow uses (x, y).
  • Grayscale: L = 0.299 R + 0.587 G + 0.114 B; Pillow convert('L'), OpenCV COLOR_BGR2GRAY.
  • Pillow: open, resize, thumbnail (keeps the ratio), crop((left, upper, right, lower)), rotate, ImageDraw, save(quality=…).
  • OpenCV uses BGR order; imread returns None on failure; uint8 wraps around in NumPy, while cv2.add stops at 255.
  • The object-finding chain: gray → GaussianBlur (odd kernel) → threshold (Otsu) → findContours → contourArea, boundingRect.
Spotted a mistake or something inaccurate in this lesson?

Check yourself

12 questions. Every correct answer earns XP.

Question 1 / 12

Question 1 / 12. A 640 × 480 colour photo is read with cv2.imread. What is img.shape?