Skip to content
Educora
EN
AzərbaycancaEnglishРусскийTürkçe
My learningEducora StudioSettings
University25 min31 / 42

pandas basics: Series and DataFrame

Work with tabular data in pandas: Series and DataFrame, reading CSV, `head`, `info`, `describe`, selecting with `loc` and `iloc`, filtering, sorting and new columns.

Check yourself
In this lesson you will learn
  • Explain the structure of a Series and a DataFrame (values, index, columns, dtypes)
  • Read a CSV and get a first look at the data with head, info, describe and value_counts
  • Select and sort rows and columns with loc, iloc, masks and query
  • Create new columns in a vectorized way without chained assignment

Aysel runs a small online stationery shop. Her sales are in an Excel sheet: product, category, price, stock and units sold. To answer questions like “which category brings the most revenue?” or “what needs to be reordered?” across hundreds of rows, there is the pandas library. It adds column names, row labels and hundreds of ready-made table operations on top of NumPy arrays.

Series and DataFrame

Definition
Series

A labelled one-dimensional array: values (a NumPy array) plus a matching index (labels). It has a name (name) and a type (dtype).

Definition
DataFrame

A two-dimensional table: a collection of Series (columns) that share one index. Each column can have its own type — text, integer, float, date.

In a Series you can reach an element by label (s['Pen']) or by position (s.iloc[0]). Arithmetic is vectorized just as in NumPy. Below we add 18% VAT to the prices:

Python
import pandas as pd

prices = pd.Series([1.5, 0.6, 35.0, 49.9],
                   index=['Notebook', 'Pen', 'Backpack', 'Headphones'],
                   name='price')
print(prices)
print(prices['Pen'], prices.iloc[-1])
print(prices[prices > 10].index.tolist())
print((prices * 1.18).round(2).tolist())
▸ Expected output
Notebook       1.5
Pen            0.6
Backpack      35.0
Headphones    49.9
Name: price, dtype: float64
0.6 49.9
['Backpack', 'Headphones']
[1.77, 0.71, 41.3, 58.88]

The simplest way to build a DataFrame is from a dictionary: the keys are the column names and the values are the column contents. shape is a (rows, columns) tuple, and dtypes shows the type of every column. In pandas 3 text columns have the type str (in older versions you will see object).

Python
import pandas as pd

df = pd.DataFrame({
    'name': ['Aysel', 'Murad', 'Leyla'],
    'age': [19, 21, 20],
    'city': ['Baku', 'Ganja', 'Baku'],
})
print(df)
print(df.shape, df.columns.tolist())
print(df.dtypes)
▸ Expected output
    name  age   city
0  Aysel   19   Baku
1  Murad   21  Ganja
2  Leyla   20   Baku
(3, 3) ['name', 'age', 'city']
name      str
age     int64
city      str
dtype: object

Reading a CSV and a first look at the data

In real work a table is read from a file: pd.read_csv('sales.csv'). In the browser there are no files, so we turn text into a “file” with io.StringIO — read_csv does not notice the difference. Each code block runs on its own, so we create the table again in every block. The first step is always the same: head() shows the first rows, and info() shows the columns, their types and the number of non-empty values. Below we print the same information piece by piece: shape, dtypes and count().

Python
import io
import pandas as pd

data = '''product,category,price,stock,sold
Notebook,Stationery,1.5,120,340
Pen,Stationery,0.6,300,820
Backpack,Accessories,35.0,25,48
Headphones,Electronics,49.9,15,37
USB cable,Electronics,6.5,80,150
Mug,Home,8.0,40,95
Desk lamp,Home,27.5,12,20'''
df = pd.read_csv(io.StringIO(data))
print(df.head(3))
print(df.shape)
print(df.dtypes)
print(df.count().tolist())
▸ Expected output
    product     category  price  stock  sold
0  Notebook   Stationery    1.5    120   340
1       Pen   Stationery    0.6    300   820
2  Backpack  Accessories   35.0     25    48
(7, 5)
product         str
category        str
price       float64
stock         int64
sold          int64
dtype: object
[7, 7, 7, 7, 7]
read_csv detected the types by itself: numbers became int64 and float64, text became str. count() counts the non-empty values in each column — there are no gaps here.

describe() gives, for every numeric column, the count, mean, standard deviation (dividing by n − 1), minimum, quartiles and maximum in one table. For a categorical column, value_counts() counts how many times each value appears.

Python
import io
import pandas as pd

data = '''product,category,price,stock,sold
Notebook,Stationery,1.5,120,340
Pen,Stationery,0.6,300,820
Backpack,Accessories,35.0,25,48
Headphones,Electronics,49.9,15,37
USB cable,Electronics,6.5,80,150
Mug,Home,8.0,40,95
Desk lamp,Home,27.5,12,20'''
df = pd.read_csv(io.StringIO(data))
print(df.describe().round(2))
print(df['category'].value_counts())
▸ Expected output
       price   stock    sold
count   7.00    7.00    7.00
mean   18.43   84.57  215.71
std    19.16  102.74  288.06
min     0.60   12.00   20.00
25%     4.00   20.00   42.50
50%     8.00   40.00   95.00
75%    31.25  100.00  245.00
max    49.90  300.00  820.00
category
Stationery     2
Electronics    2
Home           2
Accessories    1
Name: count, dtype: int64
h = (n − 1) · q Q(q) = x₍ₖ₎ + (h − k) · (x₍ₖ₊₁₎ − x₍ₖ₎), k = ⌊h⌋
where:
  • qthe level: 0.25 (first quartile), 0.5 (median), 0.75 (third quartile)
  • x₍ₖ₎the k-th value after sorting in ascending order (counting from 0)
  • hthe “fractional position” in the sorted data

By default pandas and NumPy compute quantiles by linear interpolation between two neighbouring values.

Example 1: checking the quartiles from describe

Compute the 25% and 75% quartiles of the price column by hand and compare them with the describe() result.

Show solution
Sorted prices: 0.6, 1.5, 6.5, 8.0, 27.5, 35.0, 49.9 (n = 7).
q = 0.25: h = 6 · 0.25 = 1.5 → halfway between x₍₁₎ = 1.5 and x₍₂₎ = 6.5: 1.5 + 0.5 · 5 = 4.0.
q = 0.75: h = 6 · 0.75 = 4.5 → 27.5 + 0.5 · 7.5 = 31.25.
Median: h = 3 → x₍₃₎ = 8.0. All three match describe(). The mean (18.43) is much larger than the median — a few expensive products pull the mean up.

Selecting: columns, loc and iloc

SyntaxWhat it returns
df['price']one column — a Series
df[['product', 'price']]several columns — a DataFrame
df.loc[row_labels, col_labels]selection by labels; the last label of a slice is included
df.iloc[row_pos, col_pos]selection by positions; the end of a slice is excluded
Python
import io
import pandas as pd

data = '''product,category,price,stock,sold
Notebook,Stationery,1.5,120,340
Pen,Stationery,0.6,300,820
Backpack,Accessories,35.0,25,48
Headphones,Electronics,49.9,15,37
USB cable,Electronics,6.5,80,150
Mug,Home,8.0,40,95
Desk lamp,Home,27.5,12,20'''
df = pd.read_csv(io.StringIO(data))
print(df['price'].head(3).tolist())
print(df.loc[2, 'product'], '|', df.iloc[0, 2])
print(df.loc[1:3, 'product':'price'])
print(df.iloc[1:3, 0:2])
df = df.set_index('product')
print(df.loc['Mug', 'price'])
▸ Expected output
[1.5, 0.6, 35.0]
Backpack | 1.5
      product     category  price
1         Pen   Stationery    0.6
2    Backpack  Accessories   35.0
3  Headphones  Electronics   49.9
    product     category
1       Pen   Stationery
2  Backpack  Accessories
8.0
loc[1:3] returned three rows (1, 2, 3), while iloc[1:3] returned two (1, 2). After set_index('product') you can reach rows by product name.

Filtering and sorting

We filter rows with boolean masks, just as in NumPy: conditions are combined with &, | and ~ and put in parentheses. isin checks whether a value is in a list, and query lets you write the condition as a string. sort_values sorts by a column, and nlargest returns the n largest values directly.

Python
import io
import pandas as pd

data = '''product,category,price,stock,sold
Notebook,Stationery,1.5,120,340
Pen,Stationery,0.6,300,820
Backpack,Accessories,35.0,25,48
Headphones,Electronics,49.9,15,37
USB cable,Electronics,6.5,80,150
Mug,Home,8.0,40,95
Desk lamp,Home,27.5,12,20'''
df = pd.read_csv(io.StringIO(data))
cheap = df[df['price'] < 10]
print(cheap[['product', 'price']])
print(df[(df['category'] == 'Electronics') & (df['stock'] < 50)]['product'].tolist())
print(df[df['category'].isin(['Home', 'Accessories'])]['product'].tolist())
print(df.query('price > 20 and sold > 30')[['product', 'sold']])
print(df.sort_values('sold', ascending=False).head(3)[['product', 'sold']])
print(df.nlargest(2, 'price')[['product', 'price']])
▸ Expected output
     product  price
0   Notebook    1.5
1        Pen    0.6
4  USB cable    6.5
5        Mug    8.0
['Headphones']
['Backpack', 'Mug', 'Desk lamp']
      product  sold
2    Backpack    48
3  Headphones    37
     product  sold
1        Pen   820
0   Notebook   340
4  USB cable   150
      product  price
3  Headphones   49.9
2    Backpack   35.0

New columns

A new column is created by assignment: df['revenue'] = df['price'] * df['sold'] — no loop, the calculation runs over the whole column. For a conditional value use np.where; to change only some rows use df.loc[mask, 'column'] = value. rename and drop return a new DataFrame, so we assign the result back to the variable.

Python
import io
import numpy as np
import pandas as pd

data = '''product,category,price,stock,sold
Notebook,Stationery,1.5,120,340
Pen,Stationery,0.6,300,820
Backpack,Accessories,35.0,25,48
Headphones,Electronics,49.9,15,37
USB cable,Electronics,6.5,80,150
Mug,Home,8.0,40,95
Desk lamp,Home,27.5,12,20'''
df = pd.read_csv(io.StringIO(data))
df['revenue'] = (df['price'] * df['sold']).round(2)
df['band'] = np.where(df['price'] >= 20, 'high', 'low')
df['status'] = 'ok'
df.loc[df['stock'] < 20, 'status'] = 'reorder'
df = df.rename(columns={'sold': 'units'}).drop(columns=['category'])
print(df)
print(f"Total revenue: {df['revenue'].sum():.2f} AZN")
▸ Expected output
      product  price  stock  units  revenue  band   status
0    Notebook    1.5    120    340    510.0   low       ok
1         Pen    0.6    300    820    492.0   low       ok
2    Backpack   35.0     25     48   1680.0  high       ok
3  Headphones   49.9     15     37   1846.3  high  reorder
4   USB cable    6.5     80    150    975.0   low       ok
5         Mug    8.0     40     95    760.0   low       ok
6   Desk lamp   27.5     12     20    550.0  high  reorder
Total revenue: 6813.30 AZN
Example 2: the share of Electronics in revenue

According to the table above, what percentage of the total revenue comes from the Electronics category? Compute it by hand and write the pandas expression.

Show solution
Electronics: Headphones 49.9 · 37 = 1846.3 and USB cable 6.5 · 150 = 975.0; together 2821.3 manat.
Total revenue is 6813.3 manat.
Share = 2821.3 / 6813.3 ≈ 0.414, that is about 41.4%.
pandas: df.loc[df['category'] == 'Electronics', 'revenue'].sum() / df['revenue'].sum() (compute it before the drop, while the category column still exists).

pandas is an everyday tool for analysts, economists, banks, telecom companies and research labs: reports, sales analysis, survey results, sensor logs. In the next lesson we will handle missing values, grouping, joining tables and dates.

Exercise

Read the CSV into a DataFrame. 1) Print, as a list, the names of students with a score of 80 or more, sorted by score in descending order; 2) print the mean score of all students rounded to 1 decimal place.

Exercise · Python
import io
import pandas as pd

data = '''name,group,score
Aysel,A,91
Murad,B,76
Leyla,A,85
Elvin,B,64
Nigar,A,78
Tural,B,88'''
df = pd.read_csv(io.StringIO(data))
# 1) names with score >= 80, highest first
# 2) mean score, 1 decimal
▸ Expected output
['Aysel', 'Tural', 'Leyla']
80.3
Exercise

Add a column total = price * qty to the café's orders table. Then print on separate lines: 1) the name of the item with the largest total; 2) the list of items whose total is greater than 25; 3) the sum of all total values.

Exercise · Python
import pandas as pd

orders = pd.DataFrame({
    'item': ['tea', 'coffee', 'cake', 'juice'],
    'price': [2.5, 4.0, 5.5, 3.0],
    'qty': [12, 7, 4, 9],
})
# add the total column, then print the three results
▸ Expected output
tea
['tea', 'coffee', 'juice']
107.0

Key points

  • A Series is a labelled one-dimensional array; a DataFrame is a table of Series that share an index.
  • A first look at new data: head(), info(), describe(), value_counts().
  • loc works with labels and includes the end of a slice; iloc works with positions and excludes it.
  • Filter with masks (&, |, ~), isin and query; sort with sort_values and nlargest.
  • To change some rows, write df.loc[mask, 'column'] = value and avoid chained assignment.

Check yourself

10 questions. Every correct answer earns XP.

1 / 10
For a DataFrame with the index 0, 1, 2, …, how many rows does df.loc[2:4] return?