The file system: pathlib and shutil

`Path` objects and the `/` operator, the parts of a path, searching with `exists`, `iterdir`, `glob` and `rglob`, reading and writing text and bytes, `mkdir` and `stat`, copying, moving, archiving and safely deleting with `shutil`, temporary folders and a folder-report project.

  • Advanced
  • 22 min
  • 12 questions
  • Lesson 32 of 66

In this lesson you will learn

  • Build paths with Path and / and take them apart with name, stem, suffix, parent, parts
  • Walk through folders with iterdir, glob, rglob and check files with exists, is_file, stat
  • Read and write text and bytes, create folders, copy, move and archive with shutil
  • Write a folder-report script in a temporary folder, with safe deletion habits

Murad’s «Projects» folder holds hundreds of files: photos, PDFs, old backups. Which files take the most space? How many JPG photos are there, and how big are they together? Counting by hand would take hours; Python answers in seconds. In the lesson «Files and a mini project: grade calculator» you opened a single file with open(), and in «Automation scripts» you saw the pathlib and shutil modules briefly. Now we study them in depth: building and taking apart paths, searching folders, reading and writing files, and copying, moving and safely deleting them.

Path objects: building and taking apart paths

Path and the Path objectDefinition

A path is the address of a file or folder in the file system: reports/2026/sales.csv. An absolute path starts at the root (/home/aysel/... or C:\Users\...), while a relative path is counted from the current working folder. pathlib.Path stores a path not as plain text but as an object with methods, and it picks the separator for the operating system: \ on Windows, / on Linux and macOS.

Paths are joined with the / operator: Path('reports') / '2026' / 'sales.csv'. This is safer than gluing strings with +: you cannot forget a separator or add one too many. The file name, the extension and the parent folder are properties, while with_suffix and with_stem return a new changed path, because a Path is immutable. PurePosixPath and PureWindowsPath only compute paths without touching the disk — handy for checking paths of another system:

Python
from pathlib import Path, PurePosixPath, PureWindowsPath

report = Path('reports') / '2026' / 'sales.final.csv'
print(report.name, '|', report.stem, '|', report.suffix, '|', report.suffixes)
print(report.parts)
print(report.parent.as_posix(), '|', report.parent.parent.name)
print(report.with_suffix('.xlsx').name, '|', report.with_stem('sales_v2').name)
print(report.as_posix(), '|', report.is_absolute())

print(PureWindowsPath('C:/Users/Aysel') / 'notes.txt')
print(PurePosixPath('/home/aysel') / 'notes.txt')
print(PurePosixPath('uploads') / '/etc/passwd')
Expected output
sales.final.csv | sales.final | .csv | ['.final', '.csv']
('reports', '2026', 'sales.final.csv')
reports/2026 | reports
sales.final.xlsx | sales_v2.csv
reports/2026/sales.final.csv | False
C:\Users\Aysel\notes.txt
/home/aysel/notes.txt
/etc/passwd
The last line is a trap: if the right-hand side is an absolute path (it starts with /), the / operator throws away everything before it. Check names sent by users before joining them to a path.
PropertyResultMeaning
namesales.final.csvthe full name
stemsales.finalthe name without the last extension
suffix / suffixes.csv / ['.final', '.csv']the last extension / all of them
parentreports/2026the parent folder
parts('reports', '2026', 'sales.final.csv')a tuple of the parts
with_suffix('.xlsx')reports/2026/sales.final.xlsxa new path with another extension
For the path reports/2026/sales.final.csv

Examples: parts of a path

Problem

1) What are suffix, suffixes and stem for Path('backup/archive.tar.gz')?
2) What is the parent of (Path('docs') / 'cv.pdf').with_name('cv_2026.pdf')?
3) What is PurePosixPath('media') / '/tmp/x.txt'?

Show solutionHide solution

Solution

1) .gz, ['.tar', '.gz'], archive.tar — stem drops only the last extension.
2) docs — with_name changes only the last part; the folder stays the same.
3) /tmp/x.txt — the absolute path on the right throws away the part before it (media).

Sign up free to keep reading

The rest of the lesson, the quiz and your notes open with an Educora account. Signing up takes a minute, and your progress is saved on all your devices.

Checking and searching: exists, iterdir, glob, rglob

Some Path methods ask the disk. exists() says whether the path exists, is_file() and is_dir() whether it is a file or a folder. iterdir() returns only the first level of a folder, glob(pattern) gives the paths in that folder that match a pattern, and rglob(pattern) searches all subfolders too (the same as glob('**/' + pattern)). In patterns * means any run of characters, ? one character and [abc] one character from the list. These methods return a generator, not a list, and the order of the results is not guaranteed — so always sort them with sorted().

Python
import tempfile
from pathlib import Path


def rel(paths):
    return sorted(p.relative_to(root).as_posix() for p in paths)


with tempfile.TemporaryDirectory() as tmp:
    root = Path(tmp)
    for name in ['notes.txt', 'photos/baku.jpg', 'photos/2025/shusha.JPG',
                 'photos/2025/plan.txt', 'data/sales.csv', 'data/old/sales_2024.csv']:
        file = root / name
        file.parent.mkdir(parents=True, exist_ok=True)
        file.write_text('demo', encoding='utf-8')

    print(rel(root.iterdir()))
    print(rel(root.glob('*.txt')))
    print(rel(root.glob('photos/*')))
    print(rel(root.rglob('*.csv')))
    print(rel(root.rglob('*.jpg', case_sensitive=True)))
    print(rel(p for p in root.rglob('*') if p.suffix.lower() == '.jpg'))
    print((root / 'data').is_dir(), (root / 'data/sales.csv').is_file(), (root / 'nope.txt').exists())
Expected output
['data', 'notes.txt', 'photos']
['notes.txt']
['photos/2025', 'photos/baku.jpg']
['data/old/sales_2024.csv', 'data/sales.csv']
['photos/baku.jpg']
['photos/2025/shusha.JPG', 'photos/baku.jpg']
True True False
tempfile.TemporaryDirectory() creates a temporary folder and deletes it with everything inside when the with block ends — the ideal place to experiment without touching your real files. relative_to(root).as_posix() cuts off the long temporary part and shows the path with / separators.

Reading, writing and creating folders

For small files one line is enough: read_text() and write_text() read and write text in one go, read_bytes() and write_bytes() do the same with bytes (images, archives). Always pass encoding='utf-8' for text: without it Windows may use an old code page. write_text overwrites the file; to append, or to read a big file line by line, use path.open() in an ordinary with block. mkdir(parents=True, exist_ok=True) creates every missing parent folder and does not fail if the folder already exists. stat().st_size gives the file size in bytes, unlink() deletes a file, and rename() renames it and returns the new path.

Python
import tempfile
from pathlib import Path

with tempfile.TemporaryDirectory() as tmp:
    root = Path(tmp)
    notes = root / 'notes' / 'day1.txt'
    notes.parent.mkdir(parents=True, exist_ok=True)
    notes.write_text('Salam, Bakı!', encoding='utf-8')
    text = notes.read_text(encoding='utf-8')
    print(text, len(text), notes.stat().st_size)

    logo = root / 'logo.bin'
    logo.write_bytes(bytes([137, 80, 78, 71]))
    print(logo.read_bytes(), logo.stat().st_size)

    with notes.open('a', encoding='utf-8') as f:
        f.write(' Necəsən?')
    print(notes.read_text(encoding='utf-8'))

    done = notes.rename(notes.with_name('day1_done.txt'))
    print(done.name, notes.exists(), sorted(p.name for p in notes.parent.iterdir()))
    (root / 'missing.txt').unlink(missing_ok=True)
    try:
        (root / 'notes').mkdir()
    except FileExistsError as e:
        print('FileExistsError:', Path(e.filename).name)
Expected output
Salam, Bakı! 12 13
b'\x89PNG' 4
Salam, Bakı! Necəsən?
day1_done.txt False ['day1_done.txt']
FileExistsError: notes
'Salam, Bakı!' has 12 characters but takes 13 bytes in the file: the letter ı is 2 bytes in UTF-8 (see the lesson «Unicode, encodings and bytes»). After rename the old notes object points to a path that no longer exists — take the new path from the return value.

with os.path (old style)

import os

folder = os.path.join('data', 'raw')
os.makedirs(folder, exist_ok=True)
path = os.path.join(folder, 'sales.csv')
name, ext = os.path.splitext(os.path.basename(path))
if os.path.exists(path):
    with open(path, encoding='utf-8') as f:
        text = f.read()

with pathlib

from pathlib import Path

folder = Path('data') / 'raw'
folder.mkdir(parents=True, exist_ok=True)
path = folder / 'sales.csv'
name, ext = path.stem, path.suffix
if path.exists():
    text = path.read_text(encoding='utf-8')
Both versions do the same job. You will often meet os.path in older code, but in new code pathlib is shorter and easier to read; open(), shutil and other functions accept Path objects directly.

shutil: copying, moving, archiving, deleting

Path works with single files; whole folders are the job of the shutil module (“shell utilities”). Its functions accept both strings and Path objects:

FunctionWhat it does
shutil.copy(src, dst)copies the content and permissions of a file; if dst is a folder, the file lands inside it under the same name
shutil.copy2(src, dst)the same, and it also keeps the modification time — best for backups
shutil.copytree(src, dst, ignore=...)copies a whole folder; ignore_patterns('*.tmp') skips unwanted files
shutil.move(src, dst)moves a file or a folder (even to another disk)
shutil.make_archive(base, 'zip', root_dir)makes a ZIP (or tar) archive of a folder and returns the archive’s path
shutil.rmtree(path)deletes a folder with everything in it permanently
Python
import shutil
import tempfile
import zipfile
from pathlib import Path


def files(root):
    return sorted(p.relative_to(root).as_posix() for p in root.rglob('*') if p.is_file())


with tempfile.TemporaryDirectory() as tmp:
    root = Path(tmp)
    src = root / 'project'
    (src / 'docs').mkdir(parents=True)
    (src / 'main.py').write_text('print("hi")', encoding='utf-8')
    (src / 'docs' / 'readme.md').write_text('# Project', encoding='utf-8')

    shutil.copy2(src / 'main.py', src / 'main_backup.py')
    shutil.copytree(src, root / 'copy', ignore=shutil.ignore_patterns('*_backup.py'))
    shutil.move(src / 'docs' / 'readme.md', src / 'README.md')
    print(files(root))

    archive = Path(shutil.make_archive(str(root / 'project'), 'zip', root_dir=src))
    print(archive.name, sorted(zipfile.ZipFile(archive).namelist()))
    shutil.rmtree(root / 'copy')
    print(sorted(p.name for p in root.iterdir()))
Expected output
['copy/docs/readme.md', 'copy/main.py', 'project/README.md', 'project/main.py', 'project/main_backup.py']
project.zip ['README.md', 'docs/', 'main.py', 'main_backup.py']
['project', 'project.zip']
copytree skipped main_backup.py, and move moved readme.md under the new name README.md. The archive still contains docs/ as an empty folder: moving a file does not delete its folder.
Python
import shutil
import tempfile
from pathlib import Path


def safe_rmtree(target, root, dry_run=True):
    target, root = Path(target).resolve(), Path(root).resolve()
    if target == root or not target.is_relative_to(root):
        raise ValueError(f'refusing to delete outside {root.name}: {target.name}')
    victims = sorted(p.relative_to(root).as_posix() for p in target.rglob('*'))
    if dry_run:
        print('would delete:', victims)
    else:
        shutil.rmtree(target)
        print('deleted', target.relative_to(root).as_posix())


with tempfile.TemporaryDirectory() as tmp:
    root = Path(tmp) / 'workspace'
    (root / 'cache' / 'img').mkdir(parents=True)
    (root / 'cache' / 'img' / 'a.png').write_bytes(b'png')
    safe_rmtree(root / 'cache', root)
    safe_rmtree(root / 'cache', root, dry_run=False)
    try:
        safe_rmtree(root / '..' / 'secrets', root, dry_run=False)
    except ValueError as e:
        print('ValueError:', e)
Expected output
would delete: ['cache/img', 'cache/img/a.png']
deleted cache
ValueError: refusing to delete outside workspace: secrets
resolve() expands .. parts and returns an absolute path, so workspace/../secrets is caught leaving the root. Deleting the root itself is refused too.

Mini project: a folder report

Now we put everything together. folder_report(root) walks through the whole folder and answers: how many files there are and their total size, how much space each extension takes, which three files are the largest and which folders are empty. We lowercase the extension so that .JPG and .jpg fall into one group; files without an extension go to the (none) group. For the test we create files of given sizes in a temporary folder (b'x' * size).

Python
import tempfile
from collections import defaultdict
from pathlib import Path


def folder_report(root, top=3):
    files = [p for p in root.rglob('*') if p.is_file()]
    stats = defaultdict(lambda: [0, 0])               # suffix -> [count, bytes]
    for p in files:
        s = stats[p.suffix.lower() or '(none)']
        s[0] += 1
        s[1] += p.stat().st_size
    print(f'{len(files)} files, {sum(v[1] for v in stats.values())} bytes')
    for suffix, (count, size) in sorted(stats.items(), key=lambda kv: kv[1][1], reverse=True):
        print(f'  {suffix:7} {count} file(s) {size:7} bytes')
    print('largest:')
    for p in sorted(files, key=lambda p: p.stat().st_size, reverse=True)[:top]:
        print(f'  {p.relative_to(root).as_posix():20} {p.stat().st_size:7}')
    empty = [d.relative_to(root).as_posix() for d in root.rglob('*') if d.is_dir() and not any(d.iterdir())]
    print('empty folders:', sorted(empty))


with tempfile.TemporaryDirectory() as tmp:
    root = Path(tmp)
    sizes = {'photos/baku.jpg': 52_000, 'photos/shusha.JPG': 48_500, 'docs/cv.pdf': 21_300,
             'docs/notes.txt': 1_200, 'music/song.mp3': 64_000, 'README': 300}
    for name, size in sizes.items():
        (root / name).parent.mkdir(parents=True, exist_ok=True)
        (root / name).write_bytes(b'x' * size)
    (root / 'docs' / 'old').mkdir()
    (root / 'tmp').mkdir()
    folder_report(root)
Expected output
6 files, 187300 bytes
  .jpg    2 file(s)  100500 bytes
  .mp3    1 file(s)   64000 bytes
  .pdf    1 file(s)   21300 bytes
  .txt    1 file(s)    1200 bytes
  (none)  1 file(s)     300 bytes
largest:
  music/song.mp3         64000
  photos/baku.jpg        52000
  photos/shusha.JPG      48500
empty folders: ['docs/old', 'tmp']
defaultdict(lambda: [0, 0]) creates a [count, bytes] pair for every new extension. not any(d.iterdir()) finds an empty folder: any stops at the first element, so it is fast even for big folders.

In a real script root comes from the command line (see «Command-line apps: argparse and click»), and the report goes to a log instead of print («Logging with the logging module»). For very large folders, compute each file’s size once and keep it in a dictionary so that stat() is not called several times per file.

Exercise

Count all files in the tree folder (subfolders included) by extension. Upper and lower case must not matter (.TXT = .txt). Print suffix count lines in alphabetical order of the extension.

Exercise · Python
from pathlib import Path

for name in ['a.txt', 'b.TXT', 'img/c.jpg', 'img/d.png', 'img/2026/e.JPG', 'docs/f.pdf']:
    p = Path('tree') / name
    p.parent.mkdir(parents=True, exist_ok=True)
    p.write_text('x', encoding='utf-8')

counts = {}
# go through every file under 'tree' with rglob, count it by its lower-case suffix,
# then print "suffix count" lines in alphabetical order of the suffix
Expected output
.jpg 2
.pdf 1
.png 1
.txt 2

Exercise

Write backup(file, backup_dir): if backup_dir is missing, create it together with its parents, copy the file there with shutil.copy2 under the name <name>.bak (report.txt → report.txt.bak) and return the new path.

Exercise · Python
import shutil
from pathlib import Path

Path('report.txt').write_text('Q3 revenue: 1250 AZN', encoding='utf-8')


def backup(file, backup_dir):
    # create backup_dir (with its parents) if it is missing,
    # copy the file there as <name>.bak and return the new path
    pass


copy = backup(Path('report.txt'), Path('archive') / '2026')
print(copy.as_posix())
print(copy.read_text(encoding='utf-8'))
print(sorted(p.as_posix() for p in Path('archive').rglob('*')))
Expected output
archive/2026/report.txt.bak
Q3 revenue: 1250 AZN
['archive/2026', 'archive/2026/report.txt.bak']

Remember

  • Path stores a path as an object: join with /, take apart with name, stem, suffix, parent, parts; with_suffix returns a new path.
  • iterdir() — one level, glob() — a pattern, rglob() — all subfolders; sort the results with sorted() and control case sensitivity explicitly.
  • read_text/write_text (always with encoding='utf-8') and read_bytes/write_bytes; mkdir(parents=True, exist_ok=True); size — stat().st_size.
  • shutil: copy2 — a copy that keeps the time, copytree, move, make_archive; rmtree — permanent deletion.
  • Experiment inside tempfile.TemporaryDirectory(); before deleting, do a dry run and check with resolve() + is_relative_to() that the path lies inside the root folder.
Spotted a mistake or something inaccurate in this lesson?

Check yourself

12 questions. Every correct answer earns XP.

Question 1 / 12

Question 1 / 12. What does Path('a') / 'b' / 'c.txt' create?