Logging with the logging module

Why `print` is not enough: levels, `basicConfig` and formats, loggers, handlers and formatters, the logger hierarchy with `getLogger(__name__)`, logging to a file and `RotatingFileHandler`, `logger.exception`, configuration with `dictConfig`, and what must never be logged.

  • Advanced
  • 22 min
  • 12 questions
  • Lesson 33 of 66

In this lesson you will learn

  • Choose the right level for a message and explain how level filtering works
  • Combine loggers, handlers and formatters: different levels for the console and a file
  • Log exceptions with logger.exception and rotate log files
  • Configure an application’s logging in one place with dictConfig

A report script runs on a server every night at 3 a.m. One morning the report is missing. What happened? The messages written with print went to a screen nobody watches and were lost. Had the script written a log, a file would contain a line like 2026-10-02 03:00:05 ERROR report: database is locked. The logging module is Python’s standard logging system; you met it briefly in the lesson «Automation scripts», and now we study it properly.

  • Levels: unimportant messages can be switched off without changing the code.
  • Context: the time, module, function and line number are added automatically.
  • Destinations: the same message can go to the screen, a file, e-mail or a server.
  • Central control: libraries write log messages, and the application decides how to show them.

Levels

LevelNumberWhen
DEBUG10details for developers: variables, steps
INFO20normal progress: “report started”, “copied 118 files”
WARNING30something unusual, but work goes on
ERROR40an operation failed
CRITICAL50the program cannot go on
record is shown ⇔ level(record) ≥ level(logger) and level(record) ≥ level(handler)

Where

level(record)
the message’s level: log.info → 20
level(logger)
the logger’s effective level (its own or the nearest parent’s)
level(handler)
the level of the handler that outputs the message

The lower the threshold, the more messages appear. The root logger’s default level is WARNING, so without configuration info messages are not shown.

Python
import logging
import sys

logging.basicConfig(stream=sys.stdout, format='%(levelname)-8s| %(message)s', force=True)
log = logging.getLogger('demo')

for level in (logging.DEBUG, logging.INFO, logging.WARNING):
    log.setLevel(level)
    print(f'--- logger level {logging.getLevelName(level)} ({level})')
    log.debug('details for developers')
    log.info('normal progress')
    log.warning('something unusual')
Expected output
--- logger level DEBUG (10)
DEBUG   | details for developers
INFO    | normal progress
WARNING | something unusual
--- logger level INFO (20)
INFO    | normal progress
WARNING | something unusual
--- logger level WARNING (30)
WARNING | something unusual
We write stream=sys.stdout so that the output shows up in the browser (by default logging goes to stderr), and force=True replaces the configuration left over from earlier runs.

Write messages with %s placeholders and separate arguments, not with f-strings: log.info('copied %d files', n). The text is built only when the message is really output, and log analysis tools can group messages with the same template:

Python
import logging
import sys

logging.basicConfig(stream=sys.stdout, format='%(levelname)s %(message)s', force=True)
log = logging.getLogger('lazy')
log.setLevel(logging.INFO)


class Report:
    def __str__(self):
        print('  (building a long text...)')
        return 'big report'


log.debug('data: %s', Report())        # below INFO: the text is never built
log.debug(f'data: {Report()}')         # the f-string is built anyway
log.info('data: %s', Report())         # shown, so it is built once
Expected output
  (building a long text...)
  (building a long text...)
INFO data: big report
The first debug never built the text, while the f-string built it even though the message was dropped. For big objects and frequently called code the difference is noticeable.

Sign up free to keep reading

The rest of the lesson, the quiz and your notes open with an Educora account. Signing up takes a minute, and your progress is saved on all your devices.

basicConfig and formats

logging.basicConfig(...) is the simplest setup: it adds one handler and formatter to the root logger. It works only once — if the root already has a handler, later calls do nothing (unless force=True). So call it at the program’s entry point, in the if __name__ == '__main__': block. The format string contains fields written as %(name)s:

FieldWhat it writes
%(asctime)sthe time (format set by datefmt)
%(levelname)sthe level name
%(name)sthe logger’s name
%(module)s, %(funcName)s, %(lineno)dmodule, function, line number
%(message)sthe message itself
Python
import logging

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s %(levelname)s [%(name)s] %(funcName)s:%(lineno)d %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S',
)
log = logging.getLogger('shop')


def checkout(total):
    log.info('checkout total=%.2f', total)


checkout(42.5)

Expected output

2026-10-03 17:38:15 INFO [shop] checkout:12 checkout total=42.50
No stream is given here, so the log goes to stderr. The time will show the moment you run it; line number 12 is the log.info line.

Loggers, handlers and formatters

Logger, handler and formatterDefinition

A logger is a named object your code sends messages to (logging.getLogger('shop.orders')); the same name always returns the same object. A handler delivers a message to a destination: StreamHandler — the console, FileHandler — a file, RotatingFileHandler — rotating files. A formatter turns a record into a line of text. Each handler can have its own level and formatter.

log.info('order 17')Logger 'shop.orders'is INFO ≥ the logger’s level?StreamHandlerlevel: WARNINGFileHandlerlevel: DEBUGFormatter→ screenFormatter→ file app.logthen the record goes up to the parents:shop → root (propagate)

Logger names build a hierarchy with dots: shop.orders is a child of shop, and shop is a child of the root. A logger without its own level (NOTSET) takes the level of its nearest parent. A record goes first to the logger’s own handlers and then (as long as propagate = True) to the handlers of its parents — the parents’ levels are not checked again on the way. That is why every module writes log = logging.getLogger(__name__): the logger gets the module’s name (shop.orders), and the hierarchy follows the package structure.

Python
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.WARNING,
                    format='%(name)-14s %(levelname)-7s %(message)s', force=True)
logging.getLogger('shop').setLevel(logging.INFO)

orders = logging.getLogger('shop.orders')
payments = logging.getLogger('shop.payments')
print(orders.parent.name, orders.getEffectiveLevel(), payments.getEffectiveLevel())

orders.info('order 17 created')
payments.debug('card check details')
logging.getLogger('emails').info('newsletter sent')
logging.getLogger('emails').warning('mailbox full')
Expected output
shop 20 20
shop.orders    INFO    order 17 created
emails         WARNING mailbox full
The root level is WARNING, yet the INFO message of shop.orders was printed by the root’s handler: the level is checked only at the logger where the message is created (effective level 20). emails inherits the root’s level (30), so its info was dropped.

Now let us build the handlers by hand: only warnings go to the console, and everything goes to a file. propagate = False stops the message from also reaching the root and being printed a second time:

Python
import logging
import sys

log = logging.getLogger('nightly')
log.setLevel(logging.DEBUG)
log.propagate = False
log.handlers.clear()                 # so that running the code again adds no duplicates

console = logging.StreamHandler(sys.stdout)
console.setLevel(logging.WARNING)
console.setFormatter(logging.Formatter('CONSOLE %(levelname)s: %(message)s'))

file = logging.FileHandler('nightly.log', mode='w', encoding='utf-8')
file.setLevel(logging.DEBUG)
file.setFormatter(logging.Formatter('%(levelname)-7s %(name)s %(message)s'))

log.addHandler(console)
log.addHandler(file)
log.debug('found 120 files')
log.info('copied 118 files')
log.warning('2 files skipped: permission denied')
file.close()

print('--- nightly.log:')
print(open('nightly.log', encoding='utf-8').read(), end='')
Expected output
CONSOLE WARNING: 2 files skipped: permission denied
--- nightly.log:
DEBUG   nightly found 120 files
INFO    nightly copied 118 files
WARNING nightly 2 files skipped: permission denied

Exceptions and log files

Inside an except block, log.exception('...') writes the message at ERROR level and appends the full traceback — later you can see in the log where the error happened. For other levels exc_info=True does the same: log.warning('...', exc_info=True). The program catches the exception and keeps working (see «Handling errors: try and except»):

Python
import logging

logging.basicConfig(level=logging.INFO, format='%(levelname)s %(name)s: %(message)s')
log = logging.getLogger('orders')


def average(total, count):
    return total / count


try:
    average(120, 0)
except ZeroDivisionError:
    log.exception('could not compute the average for order %d', 17)
log.info('the program keeps running')

Expected output

ERROR orders: could not compute the average for order 17
Traceback (most recent call last):
  File "orders.py", line 12, in <module>
    average(120, 0)
    ~~~~~~~^^^^^^^^
  File "orders.py", line 8, in average
    return total / count
           ~~~~~~^~~~~~~
ZeroDivisionError: division by zero
INFO orders: the program keeps running
The path in the File "..." lines depends on where the file is stored; in the browser the name is <exec>.

The log file of a program that runs all the time can fill the disk. logging.handlers.RotatingFileHandler renames the file to app.log.1 when it reaches maxBytes (older ones become .2, .3) and opens a new file; backupCount says how many old files to keep. TimedRotatingFileHandler rotates files by time instead, for example every night.

Python
import logging
import os
from logging.handlers import RotatingFileHandler

log = logging.getLogger('sensor')
log.setLevel(logging.INFO)
log.propagate = False
log.handlers.clear()
handler = RotatingFileHandler('sensor.log', maxBytes=300, backupCount=2, encoding='utf-8')
handler.setFormatter(logging.Formatter('%(levelname)s %(message)s'))
log.addHandler(handler)

for i in range(1, 41):
    log.info('reading %02d: temperature %.1f', i, 20 + i / 10)
handler.close()

print(sorted(name for name in os.listdir() if name.startswith('sensor.log')))
with open('sensor.log', encoding='utf-8') as f:
    print(f.readlines()[-1].strip())
Expected output
['sensor.log', 'sensor.log.1', 'sensor.log.2']
INFO reading 40: temperature 24.0
Forty lines did not fit into 300-byte files: the newest records are in sensor.log, only two files of older ones are kept, and anything older was deleted.

Configuration with dictConfig: a mini project

In a large application logging settings are not scattered through the code: all formatters, handlers and loggers are described in one dictionary and applied at the program’s entry point with logging.config.dictConfig(...). The dictionary can also be read from a JSON or YAML file, so changing a level needs no code change. Always write 'disable_existing_loggers': False: otherwise loggers created before the configuration (for example at import time) are silenced. The mini project processes orders: INFO goes to the console, and DEBUG with a more detailed format goes to a file:

Python
import logging
import logging.config

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'short': {'format': '%(levelname)-7s %(name)s: %(message)s'},
        'full': {'format': '%(levelname)s|%(name)s|%(funcName)s|%(message)s'},
    },
    'handlers': {
        'console': {'class': 'logging.StreamHandler', 'stream': 'ext://sys.stdout',
                    'level': 'INFO', 'formatter': 'short'},
        'file': {'class': 'logging.FileHandler', 'filename': 'store.log', 'mode': 'w',
                 'encoding': 'utf-8', 'level': 'DEBUG', 'formatter': 'full'},
    },
    'loggers': {'store': {'level': 'DEBUG', 'handlers': ['console', 'file'], 'propagate': False}},
}
logging.config.dictConfig(LOGGING)
log = logging.getLogger('store.orders')


def process(order):
    log.debug('processing %s', order)
    if order['qty'] <= 0:
        log.warning('order %s skipped: qty=%s', order['id'], order['qty'])
        return 0
    total = order['qty'] * order['price']
    log.info('order %s: %.2f AZN', order['id'], total)
    return total


orders = [{'id': 1, 'qty': 2, 'price': 4.5}, {'id': 2, 'qty': 0, 'price': 9.9},
          {'id': 3, 'qty': 1, 'price': 15.0}]
log.info('revenue %.2f AZN', sum(process(o) for o in orders))
print(open('store.log', encoding='utf-8').readline(), end='')
Expected output
INFO    store.orders: order 1: 9.00 AZN
WARNING store.orders: order 2 skipped: qty=0
INFO    store.orders: order 3: 15.00 AZN
INFO    store.orders: revenue 24.00 AZN
DEBUG|store.orders|process|processing {'id': 1, 'qty': 2, 'price': 4.5}
There is no DEBUG on the console, while the first line of the file is exactly a DEBUG record that also shows the function name. The logger store.orders has no handlers of its own — its records reach the handlers of the parent store.

Exercise

Configure logging so that only WARNING and higher messages are written to sys.stdout in the format LEVEL: message. Do not change the lines that log the messages.

Exercise · Python
import logging
import sys

# configure logging here: stdout, level WARNING, format "LEVEL: message"

log = logging.getLogger('quiz')
log.debug('loading questions')
log.info('12 questions loaded')
log.warning('question 7 has no answer')
log.error('cannot save the results')
Expected output
WARNING: question 7 has no answer
ERROR: cannot save the results

Exercise

safe_divide(a, b) must log 10 / 4 = 2.5 at INFO level and return the result on success, and when b is 0 it must log cannot divide 3 by zero at WARNING level and return None. Build the messages with %s placeholders.

Exercise · Python
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO,
                    format='%(levelname)s %(name)s: %(message)s', force=True)
log = logging.getLogger('calc')


def safe_divide(a, b):
    # on success: log INFO "10 / 4 = 2.5" and return the result
    # if b is 0: log WARNING "cannot divide 3 by zero" and return None
    return a / b


print([safe_divide(10, 4), safe_divide(3, 0)])
Expected output
INFO calc: 10 / 4 = 2.5
WARNING calc: cannot divide 3 by zero
[2.5, None]

Remember

  • Levels: DEBUG 10 < INFO 20 < WARNING 30 < ERROR 40 < CRITICAL 50; the root’s default level is WARNING.
  • Use logging.getLogger(__name__) in every module; names build a dotted hierarchy, and records propagate to the parents’ handlers.
  • A handler sets the destination and its own level; a formatter sets how the line looks.
  • log.exception inside except writes ERROR plus the traceback; RotatingFileHandler limits the size of log files.
  • Configure once, at the entry point: basicConfig or dictConfig (disable_existing_loggers: False); never log passwords or personal data.
Spotted a mistake or something inaccurate in this lesson?

Check yourself

12 questions. Every correct answer earns XP.

Question 1 / 12

Question 1 / 12. Which is the increasing order of the levels?