Summary of Unicode Support in Python

Unicode Tutorials - Herong's Tutorial Examples

∟Summary of Unicode Support in Python

This section provides a quick summary of Unicode support in Python language.

Unicode support in Python language can be summarized as below:

1. Full support of Unicode started in Python 3.0 - So if you are still using Python 2.x, you need to manage Unicode characters and code points in your own code.

2. Default encoding for Python source code is UTF-8 - This allows you to enter Unicode characters in your source code directly. If your Python script is stored in a non-UTF-8 encoding (e.g. latin-1, or gb2312), you need to add a special comment line to specify the encoding name like:

# -*- coding: latin-1 -*-
...

3. "str" data type stores Unicode characters as UTF-8 encoded bytes - The UTF-8 encoded bytes are internal storage structures with no public access.

4. Special escape sequences supported in "str" literals -

Using the character name like "\N{GREEK CAPITAL LETTER DELTA}".
Using a 8-bit hex value like "\xac", for code points that are small enough to be expressed in 2 hexsdecimal digits.
Using a 16-bit hex value like "\u0394", for code points that are small enough to be expressed in 4 hexsdecimal digits.
Using a 32-bit hex value like "\U00000394", for any code points.

5. chr(int) converts a code point to a character - A character is actually a "str" with 1 character.

6. ord(chat) converts a character to a code point

7. str.encode() converts "str" to "bytes" - It uses the default UTF-8 encoding or a given encoding.

8. bytes.decode() converts "bytes" to "str" - It uses the default UTF-8 encoding or a given encoding.

9. "unicodedata" module for Unicode properties - It provides a number of static methods to access varies properties of a given code point defined by the Unicode standard.