Aysel wrote a script that converts currencies. Every time, her colleagues open the code and change the lines AMOUNT = 100 and RATE = 1.7 — sooner or later someone will break the wrong line. If the script takes its parameters from the command line, typing python convert.py 100 USD --rate 1.7 is enough, and --help explains what to do by itself. In «Automation scripts» you met argparse briefly; now we study it in depth and then write the same tool with the popular click library.
sys.argv: arguments as a list of strings
Every word typed in the terminal reaches the program in the list sys.argv: sys.argv[0] is the script’s name, and the rest are the arguments — always strings. Parsing by hand quickly becomes fragile: a forgotten argument gives IndexError, a value of the wrong type gives a confusing Traceback, there is no help text, and you have to look for named parameters like --rate yourself every time.
import sys
print(sys.argv)
name = sys.argv[1] if len(sys.argv) > 1 else 'world'
times = int(sys.argv[2]) if len(sys.argv) > 2 else 1
for _ in range(times):
print(f'Hello, {name}!')greet.py$ python greet.py Aysel 2
['greet.py', 'Aysel', '2']
Hello, Aysel!
Hello, Aysel!
$ python greet.py
['greet.py']
Hello, world!
$ python greet.py Aysel two
['greet.py', 'Aysel', 'two']
Traceback (most recent call last):
...
ValueError: invalid literal for int() with base 10: 'two''2' arrives as a string and must be converted with int(); when the user types two, they see a Traceback meant for programmers instead of an error message (shortened here).argparse: positional and optional arguments
Positional argument, option and flagDefinition
A positional argument is recognised by its place and is usually required: convert 100 USD. An option (optional argument) starts with - or --, can be written in any order and has a default: --rate 1.7, short form -r 1.7. A flag is an option without a value: True if --verbose was typed, False if not.
| add_argument parameter | What it does |
|---|---|
type=int, type=float | converts the string to a type; if that fails, gives a clear error |
default=... | the value when the argument is not given |
choices=['USD', 'EUR'] | allows only these values |
action='store_true' | a flag: True/False |
action='count', action='append' | -vvv → 3; collects a repeated option into a list |
nargs='?', '*', '+' | 0 or 1, any number, at least 1 value |
help=, metavar= | the --help text and the name shown there |
In a real script parser.parse_args() is called without arguments and reads sys.argv[1:]. If we pass it a list, the same code also works in the browser and in tests — all examples below do this. The result is a Namespace object: every argument is an attribute, and in names like --dry-run the hyphen turns into an underscore (args.dry_run).
import argparse
parser = argparse.ArgumentParser(prog='convert', description='Convert AZN to another currency.')
parser.add_argument('amount', type=float, help='amount in AZN')
parser.add_argument('currency', choices=['USD', 'EUR', 'TRY'], help='target currency')
parser.add_argument('-r', '--rate', type=float, default=1.7, help='AZN per unit')
parser.add_argument('--digits', type=int, default=2, metavar='N', help='digits after the point')
parser.add_argument('-v', '--verbose', action='store_true', help='also print the formula')
args = parser.parse_args(['100', 'USD', '--rate', '1.7', '-v'])
print(args)
if args.verbose:
print(f'{args.amount} / {args.rate}')
print(f'{args.amount} AZN = {round(args.amount / args.rate, args.digits)} {args.currency}')
print(parser.parse_args(['50', 'EUR', '-r', '1.95', '--digits', '1']))Expected output
Namespace(amount=100.0, currency='USD', rate=1.7, digits=2, verbose=True) 100.0 / 1.7 100.0 AZN = 58.82 USD Namespace(amount=50.0, currency='EUR', rate=1.95, digits=1, verbose=False)
'100' became 100.0 automatically, and the -v flag gave verbose=True. In the second call the options that were not given took their defaults.import argparse
parser = argparse.ArgumentParser(prog='backup')
parser.add_argument('folders', nargs='+', help='one or more folders')
parser.add_argument('-x', '--exclude', action='append', default=[], help='a pattern to skip (repeatable)')
parser.add_argument('-v', '--verbose', action='count', default=0, help='-v, -vv, -vvv')
group = parser.add_mutually_exclusive_group()
group.add_argument('--zip', action='store_true')
group.add_argument('--tar', action='store_true')
print(parser.parse_args(['docs', 'photos', '-x', '*.tmp', '-x', '*.log', '-vv', '--zip']))
print(parser.parse_args(['docs']))Expected output
Namespace(folders=['docs', 'photos'], exclude=['*.tmp', '*.log'], verbose=2, zip=True, tar=False) Namespace(folders=['docs'], exclude=[], verbose=0, zip=False, tar=False)
nargs='+' collected several folders into a list, the repeated -x became a list through append, and -vv counted 2. add_mutually_exclusive_group does not allow --zip and --tar together.Examples: argparse rules
Problem
1) What name does the option --dry-run get in the Namespace?
2) What does -vvv give with action='count', default=0?
3) add_argument('output', nargs='?', default='out.txt') and parse_args([]) — what is args.output?
Show solutionHide solution
Solution
dry_run — the leading hyphens are dropped and the inner hyphen becomes an underscore.2) 3 — every
v adds 1 to the counter.3)
'out.txt' — nargs='?' makes the positional argument optional, and without a value default is used.Help text and errors
argparse adds the -h/--help option by itself and builds the help text from description and the help= texts; %(default)s inserts the default value. When the user makes a mistake, argparse writes a short usage line and a clear message to stderr and stops the program with code 2 (it raises SystemExit). In a test or in the browser you can catch it with except SystemExit.
import argparse
parser = argparse.ArgumentParser(prog='convert', description='Convert AZN to another currency.')
parser.add_argument('amount', type=float, help='amount in AZN')
parser.add_argument('currency', choices=['USD', 'EUR', 'TRY'], help='target currency')
parser.add_argument('-r', '--rate', type=float, default=1.7, help='AZN per unit (default: %(default)s)')
parser.add_argument('-v', '--verbose', action='store_true', help='also print the formula')
parser.print_help()Expected output
usage: convert [-h] [-r RATE] [-v] amount {USD,EUR,TRY}
Convert AZN to another currency.
positional arguments:
amount amount in AZN
{USD,EUR,TRY} target currency
options:
-h, --help show this help message and exit
-r, --rate RATE AZN per unit (default: 1.7)
-v, --verbose also print the formulaimport argparse
import contextlib
import sys
parser = argparse.ArgumentParser(prog='convert')
parser.add_argument('amount', type=float)
parser.add_argument('currency', choices=['USD', 'EUR', 'TRY'])
def run(argv):
with contextlib.redirect_stderr(sys.stdout): # show the error text in the output
try:
print(parser.parse_args(argv))
except SystemExit as e:
print('exit code:', e.code)
run(['100', 'USD'])
run(['100', 'GBP'])
run(['ten', 'USD'])
run(['100'])Expected output
Namespace(amount=100.0, currency='USD')
usage: convert [-h] amount {USD,EUR,TRY}
convert: error: argument currency: invalid choice: 'GBP' (choose from USD, EUR, TRY)
exit code: 2
usage: convert [-h] amount {USD,EUR,TRY}
convert: error: argument amount: invalid float value: 'ten'
exit code: 2
usage: convert [-h] amount {USD,EUR,TRY}
convert: error: the following arguments are required: currency
exit code: 2redirect_stderr(sys.stdout) sends the error text to the normal output so that it shows up here. The exact wording of the messages can differ a little between Python versions, but the code is always 2.Subcommands and exit codes: mini project
Big tools have subcommands: git commit, git push, pip install. In argparse this is done by add_subparsers(dest='command', required=True): every subcommand is a separate parser with its own arguments, and the chosen command’s name is stored in args.command. Let us build a small notes tool: the commands add, list and done, with the notes kept in notes.json.
Exit codeDefinition
The number a program returns to the operating system when it ends: 0 means success, anything else means an error (argparse uses 2 for wrong arguments). Scripts, schedulers and CI systems look exactly at this code: in bash, cmd1 && cmd2 runs the second command only if the first returned 0, and the last code is shown by $? in bash and $LASTEXITCODE in PowerShell. In Python you return it with sys.exit(code).
import argparse
import contextlib
import json
import shlex
import sys
from pathlib import Path
DB = Path('notes.json')
def main(argv=None):
parser = argparse.ArgumentParser(prog='notes')
sub = parser.add_subparsers(dest='command', required=True)
add = sub.add_parser('add', help='add a note')
add.add_argument('text')
add.add_argument('-p', '--priority', type=int, choices=[1, 2, 3], default=2)
sub.add_parser('list', help='show open notes').add_argument('--all', action='store_true')
sub.add_parser('done', help='mark a note as done').add_argument('number', type=int)
args = parser.parse_args(argv)
notes = json.loads(DB.read_text(encoding='utf-8')) if DB.exists() else []
if args.command == 'add':
notes.append({'text': args.text, 'priority': args.priority, 'done': False})
print(f'added #{len(notes)}')
elif args.command == 'list':
for i, n in enumerate(notes, start=1):
if args.all or not n['done']:
print(f"{i}. [{'x' if n['done'] else ' '}] p{n['priority']} {n['text']}")
elif not 1 <= args.number <= len(notes):
print(f'notes: no note #{args.number}', file=sys.stderr)
return 1
else:
notes[args.number - 1]['done'] = True
DB.write_text(json.dumps(notes), encoding='utf-8')
return 0
with contextlib.redirect_stderr(sys.stdout):
for argv in (['add', 'Buy bread'], ['add', 'Python homework', '-p', '1'],
['done', '1'], ['list'], ['list', '--all'], ['done', '7']):
print('$ notes', shlex.join(argv))
print(' exit code', main(argv))Expected output
$ notes add 'Buy bread' added #1 exit code 0 $ notes add 'Python homework' -p 1 added #2 exit code 0 $ notes done 1 exit code 0 $ notes list 2. [ ] p1 Python homework exit code 0 $ notes list --all 1. [x] p2 Buy bread 2. [ ] p1 Python homework exit code 0 $ notes done 7 notes: no note #7 exit code 1
main(argv) returns the exit code instead of calling sys.exit — that is why it can be called several times here or in a test. For a wrong number the message goes to stderr and the code is 1. shlex.join shows the arguments with quotes, as they would be typed in a terminal.$ python notes.py add "Buy bread"
added #1
$ python notes.py done 7
notes: no note #7
$ echo $?
1
$ python notes.py remove 1
usage: notes [-h] {add,list,done} ...
notes: error: argument command: invalid choice: 'remove' (choose from add, list, done)
$ echo $?
2
$ python notes.py list && echo "all good"
1. [ ] p2 Buy bread
all goodif __name__ == '__main__': sys.exit(main()) added at the end. There is no remove command, so argparse exited with code 2, and && ran echo because list succeeded.As the commands grow, the if/elif chain gets long. Then you can write a separate function for every subcommand and attach it to its parser with set_defaults(func=cmd_add), while main simply calls return args.func(args). Notice one more thing: results go to stdout, and error messages go to stderr with print(..., file=sys.stderr). When the user types python notes.py list > notes.txt, the file gets only the notes, and an error stays on the screen.
click: the same with decorators
click is a third-party library (pip install click); for example, Flask’s flask command is built on it. Instead of a parser object you write decorators above a function (see «Decorators and closures»): @click.command(), @click.argument for positional arguments and @click.option for options. The arguments reach the function as ordinary parameters, and the function’s docstring becomes the help text. click.Choice, click.IntRange, is_flag=True, show_default=True and envvar=, which takes a value from an environment variable, come ready; @click.group() builds subcommands.
import click
@click.command()
@click.argument('amount', type=float)
@click.argument('currency', type=click.Choice(['USD', 'EUR', 'TRY']))
@click.option('-r', '--rate', type=float, default=1.7, show_default=True, help='AZN per unit.')
@click.option('-v', '--verbose', is_flag=True, help='Also print the formula.')
def convert(amount, currency, rate, verbose):
"""Convert AZN to another currency."""
if verbose:
click.echo(f'{amount} / {rate}')
click.echo(f'{amount} AZN = {round(amount / rate, 2)} {currency}')
if __name__ == '__main__':
convert()convert.py$ python convert.py 100 USD -v
100.0 / 1.7
100.0 AZN = 58.82 USD
$ python convert.py --help
Usage: convert.py [OPTIONS] AMOUNT {USD|EUR|TRY}
Convert AZN to another currency.
Options:
-r, --rate FLOAT AZN per unit. [default: 1.7]
-v, --verbose Also print the formula.
--help Show this message and exit.
$ python convert.py 100 GBP
Usage: convert.py [OPTIONS] AMOUNT {USD|EUR|TRY}
Try 'convert.py --help' for help.
Error: Invalid value for '{USD|EUR|TRY}': 'GBP' is not one of 'USD', 'EUR', 'TRY'.[default: 1.7], and for a wrong value click exits with code 2.click’s strong point is testing: click.testing.CliRunner invokes a command without starting a separate process and captures its output and exit code. We write the tests as in «Testing with pytest»:
import click
from click.testing import CliRunner
@click.group()
def cli():
"""A tiny notes tool."""
@cli.command()
@click.argument('text')
@click.option('-p', '--priority', type=click.IntRange(1, 3), default=2, show_default=True)
def add(text, priority):
"""Add a note."""
click.echo(f'added: {text} (p{priority})')
def test_add():
result = CliRunner().invoke(cli, ['add', 'Buy bread', '-p', '1'])
assert result.exit_code == 0
assert result.output == 'added: Buy bread (p1)\n'
def test_bad_priority():
result = CliRunner().invoke(cli, ['add', 'Buy bread', '-p', '9'])
assert result.exit_code == 2
assert '9 is not in the range 1<=x<=3' in result.outputtest_notes_cli.py$ pytest -q test_notes_cli.py
.. [100%]
2 passed in 0.05sIntRange(1, 3) rejected 9; result.output holds the error message too. The time (0.05s) differs on every run.| argparse | click | |
|---|---|---|
| Installation | standard library | pip install click |
| Style | a parser object, add_argument | decorators above a function |
| Subcommands | add_subparsers | @click.group() |
| Testing | call main([...]) | CliRunner().invoke(...) |
| Extras | — | prompts, colours, envvar |
argparse for a small script without dependencies; click is often chosen for a big tool with many commands.Exercise
Add arguments to the resize parser: the positional image, -w/--width (an integer, default 800), the flag --keep-ratio and --format (jpg, png, webp, default jpg).
import argparse
parser = argparse.ArgumentParser(prog='resize')
# add: the positional argument image; -w/--width (int, default 800);
# the flag --keep-ratio; --format with choices jpg, png, webp (default jpg)
args = parser.parse_args(['photo.png', '-w', '1200', '--keep-ratio'])
print(args.image, args.width, args.keep_ratio, args.format)
args = parser.parse_args(['logo.png', '--format', 'webp'])
print(args.image, args.width, args.keep_ratio, args.format)Expected output
photo.png 1200 True jpg logo.png 800 False webp
Exercise
Finish the wordcount tool: --top N (default 3) sets how many words are shown, and the flag -i/--ignore-case makes it lowercase the words before counting.
import argparse
from collections import Counter
def main(argv):
parser = argparse.ArgumentParser(prog='wordcount')
parser.add_argument('text')
# add --top (int, default 3) and the flag -i/--ignore-case
args = parser.parse_args(argv)
words = args.text.split()
# with --ignore-case, lowercase every word
for word, n in Counter(words).most_common(3):
print(word, n)
return 0
main(['Tea tea coffee Tea milk coffee', '--top', '2'])
print('---')
main(['Tea tea coffee Tea milk coffee', '-i'])Expected output
Tea 2 coffee 2 --- tea 3 coffee 2 milk 1
Remember
sys.argvis a list of strings ([0]is the script name); parsing by hand is fragile, whileargparsehandles types, help and errors itself.- Positional arguments are required, options are written as
--nameand have defaults;type,choices,store_true,count,append,nargs. - On a wrong argument
argparsewrites to stderr and exits with code 2; in tests and in the browser pass a list toparse_args. - Subcommands:
add_subparsers; letmain(argv=None)return the code andsys.exit(main())hand it to the system: 0 means success. clickdoes the same job with decorators (command,argument,option,group) and is easy to test withCliRunner.