Encoding text

When you type “Salam” on a keyboard, the computer stores not letters but numbers: 83, 97, 108, 97, 109. The screen shows letters again because a program draws the sign that matches each number. Sometimes, though, a text from a friend shows strange signs like “É™” instead of the Azerbaijani letter ə. In this lesson we will see how a computer encodes text and where such errors come from.

In the lesson “Measuring information: bits, bytes and units” we learned the formula I = K · i; now we will apply it to real texts. The topic comes up often in the entrance exam: in the four entrance exams of 2025–2026, five tasks were about encoding text (ASCII and UNICODE volumes, the power of an alphabet, words in a 64-symbol alphabet, a UNICODE text sent over a channel), and one text-processing task chose words by their UNICODE volume.

Code tables: a number for every symbol

ASCII: one symbol, one byte

ASCII (American Standard Code for Information Interchange) was created in 1963. It has 128 codes (0–127): 0–31 are control characters (for example, a new line), 32 is the space, 48–57 are the digits, 65–90 the capital and 97–122 the small Latin letters. In a computer every ASCII symbol takes 1 byte (8 bits). Codes 128–255 were used for national letters: tables such as Windows-1251 and KOI8-R were made for Cyrillic and Windows-1254 for Turkish.

word = "INFO"
print([ord(c) for c in word])
print(chr(66) + chr(73) + chr(84))
print(ord("a") - ord("A"), ord("ə"))

[73, 78, 70, 79]
BIT
32 601

UNICODE: all languages in one table

The 8-bit tables of different countries did not agree: the same code meant one letter in one table and another letter in another. So in 1991 the UNICODE standard was created. It gives a separate number to the letters of all the world’s scripts, to math signs and even to emoji — more than 150,000 symbols today. The first 128 codes are the same as in ASCII: “A” is still 65, and the number of ə is U+0259, that is, 601.

In school problems and DİM tasks, “UNICODE” means 16 bits (2 bytes) per symbol: 2¹⁶ = 65,536 different symbols. This matches the basic case of the UTF-16 encoding. Real files and the web mostly use UTF-8, where a symbol takes from 1 to 4 bytes: Latin letters take 1, the special Azerbaijani letters (ə, ş, ç, ğ, ı, ö, ü) and Cyrillic letters 2, the € sign and most Chinese characters 3, and emoji 4 bytes.

1. 1
   Count the letters

   Count the letters of every word and add them up.

2. 2
   Add the spaces

   If there is one space between words, the number of spaces = the number of words − 1.

3. 3
   Do not forget punctuation

   Full stops, commas, question and exclamation marks, quotation marks, dashes and digits are all symbols.

4. 4
   Multiply and convert

   Multiply K by 8 (ASCII) or 16 (UNICODE) bits, then convert to the unit you need.

s = "Small steps lead to big results."
print(len(s))                    # spaces and the dot included
print(len(s) * 8, len(s) * 16)   # bits in ASCII and in UNICODE
for w in ["Salam", "Gəncə", "Баку", "€"]:
    print(w, len(w.encode("utf-8")))   # real size in UTF-8, bytes

32
256 512
Salam 5
Gəncə 7
Баку 8
€ 3

Documents, the power of an alphabet and transmission

In a document of several pages, the number of symbols is K = pages · lines · symbols per line. In exam problems the numbers are usually powers of two (32 lines, 64 symbols), so it is convenient to work with exponents. The other way round, if the volume is known, you can find the bits per symbol and the power of the alphabet.

When a text is sent over a communication channel, the formula I = v · t from the lesson “Measuring information: bits, bytes and units” applies: the volume of the text in bits = rate · time.

s = "Bakı is the capital of Azerbaijan."
# print the UNICODE volume in bits

544

In the next lesson, “Computer graphics: encoding raster and vector images”, we will see how pictures are encoded; the formula N = 2ⁱ plays the key role there too.