Web APIs and the requests library

How HTTP works: methods, status codes, headers and JSON. GET and POST requests with `requests`, `params`, `headers`, `timeout`, `raise_for_status`, error handling and keeping API keys safe.

  • Advanced
  • 18 min
  • 10 questions
  • Lesson 38 of 66

In this lesson you will learn

  • Explain the parts of an HTTP request and response, the main methods and the status codes
  • Send GET requests with parameters and POST requests with a JSON body using requests
  • Write robust code with timeouts, raise_for_status and exceptions, and keep API keys secret

The weather forecast on your phone, a bank's exchange rates, a route on a map — programs get all this data from servers through an API (Application Programming Interface). The Educora apps also load lessons from the website's API. A web API is a simple agreement: you send an HTTP request to a certain address, and the server usually returns a JSON response. In Python the most popular tool for this is the requests library.

HTTP in brief

A request has four parts: the method (what to do), the URL (where), the headers (extra information: format, key) and the body (the data being sent, for example JSON). A response consists of a status code, headers and a body. The part of the URL after ? holds the query parameters (the query string): ?city=Baku&units=metric.

Text
GET /v1/weather?city=Baku&units=metric HTTP/1.1
Host: api.example.com
Accept: application/json

HTTP/1.1 200 OK
Content-Type: application/json

{"city": "Baku", "temp": 24.5, "wind": 7.2, "description": "sunny"}
This is what a request and a response look like “from the inside”. api.example.com is an example address reserved for documentation — all the responses in this lesson are illustrative.
MethodMeaningExample
GETread dataGET /v1/tasks
POSTcreate a new objectPOST /v1/tasks
PUT / PATCHreplace / partly update an objectPATCH /v1/tasks/42
DELETEdelete an objectDELETE /v1/tasks/42
CodeMeaning
200, 201, 204success: OK, Created, No Content
301, 302redirect — look at another address
400, 401, 403, 404client error: bad request, not logged in, forbidden, not found
429too many requests — the limit is exceeded, wait
500, 503server error, service unavailable

Sign up free to keep reading

The rest of the lesson, the quiz and your notes open with an Educora account. Signing up takes a minute, and your progress is saved on all your devices.

Requests with requests

Install the library on your computer: pip install requests. Python in the browser cannot send network requests, so the code in this lesson is not runnable here — try it on your own computer. requests.get returns a response object: status_code, headers, text, json(). Do not glue parameters into the URL by hand: the params dictionary encodes them correctly, for example “Sumqayıt” becomes Sumqay%C4%B1t.

Python
import requests

url = 'https://api.example.com/v1/weather'
params = {'city': 'Baku', 'units': 'metric'}
headers = {'Accept': 'application/json'}

response = requests.get(url, params=params, headers=headers, timeout=10)
print(response.status_code)
print(response.url)
print(response.headers['Content-Type'])
data = response.json()
print(data['city'], data['temp'])

Expected output

200
https://api.example.com/v1/weather?city=Baku&units=metric
application/json
Baku 24.5
response.json() turns the JSON body into a Python dictionary. The output is an example: a real API would return a different temperature.

To create a new object we send a POST. The json= argument converts the dictionary to JSON and adds the Content-Type: application/json header by itself. On successful creation the server usually returns code 201 and the new object (with its identifier).

Python
import requests

new_task = {'title': 'Finish the pandas lesson', 'done': False}
r = requests.post('https://api.example.com/v1/tasks', json=new_task, timeout=10)
print(r.status_code)
print(r.json())
print(r.request.headers['Content-Type'])

Expected output

201
{'id': 42, 'title': 'Finish the pandas lesson', 'done': False}
application/json

Errors, timeouts and raise_for_status

Careful: a 404 or 500 response is not an error for requests — the code quietly continues. raise_for_status() raises an HTTPError for 4xx and 5xx codes. Network problems are separate exceptions: Timeout and ConnectionError; all of them derive from requests.exceptions.RequestException.

Python
import requests

def get_json(url, **params):
    try:
        r = requests.get(url, params=params, timeout=5)
        r.raise_for_status()
        return r.json()
    except requests.exceptions.Timeout:
        print('The server did not answer in time')
    except requests.exceptions.HTTPError as e:
        print('HTTP error:', e.response.status_code)
    except requests.exceptions.RequestException as e:
        print('Network problem:', type(e).__name__)
    return None

print(get_json('https://api.example.com/v1/weather', city='Atlantis'))

Expected output

HTTP error: 404
None
Specific exceptions come before the general one (RequestException); otherwise they would never be reached.

Keeping API keys safe

Most APIs identify you by a key (an API key or token). A key is like a password: anyone who knows it can send requests in your name, use up your limit and even run up charges. The rules:

  • Never write the key in your code or commit it to Git — keep it in an environment variable.
  • If you use a .env file, add it to .gitignore.
  • If the API allows it, send the key in a header, not in the URL: URLs end up in logs and browser history.
  • Do not put a secret key into browser or mobile app code — any user can extract it; send such requests through your own server.
  • If a key leaks, revoke it immediately and create a new one.
Terminal
# Linux / macOS
export WEATHER_API_KEY='paste-your-key-here'

# Windows PowerShell
$env:WEATHER_API_KEY = 'paste-your-key-here'
The variable lives only in the current terminal session; to keep it permanently, use your operating system's settings or a .env file.
Python
import os
import requests

api_key = os.environ.get('WEATHER_API_KEY')
if not api_key:
    raise SystemExit('Set the WEATHER_API_KEY environment variable first')

session = requests.Session()
session.headers.update({'Authorization': f'Bearer {api_key}'})
r = session.get('https://api.example.com/v1/weather', params={'city': 'Baku'}, timeout=10)
r.raise_for_status()
print(r.json()['description'])

Expected output

sunny
The key stays outside the code, and the Session adds it to the header of every request. The exact header name (Authorization, X-API-Key, etc.) is given in the API's documentation.

Remember

  • An HTTP request: method, URL, headers, body; a response: status code, headers, body (often JSON).
  • GET reads, POST creates, PUT/PATCH update, DELETE deletes; 2xx is success, 4xx a client error, 5xx a server error.
  • requests.get(url, params=..., headers=..., timeout=...) and requests.post(url, json=...); get the result with r.json().
  • Always pass timeout and call raise_for_status(); catch exceptions from specific to general.
  • An API key lives in an environment variable, travels in a header and never goes into Git; if it leaks, revoke it.
Spotted a mistake or something inaccurate in this lesson?

Check yourself

10 questions. Every correct answer earns XP.

Question 1 / 10

Question 1 / 10. The server returned 404. What will requests.get(...) do?