Python Language and Unicode Characters

Unicode Tutorials - Herong's Tutorial Examples

∟Python Language and Unicode Characters

This chapter provides notes and tutorial examples on Unicode support in Python language. Topics including using 'str' data type to store single and multiple Unicode characters; Unicode escape sequences in 'str' literals; default Unicode encoding used in Python; str.encode() and bytes.decode() methods for encoding conversions; 'unicodedata' Module to retrieve Unicode properties.

Summary of Unicode Support in Python

Python Source Code Encoding

Unicode Support on "str" Data Type

Unicode Character Encoding and Decoding

"unicodedata" Module for Unicode Properties

Conclusions:

Unicode is fully supported since Python 3.0.
Default encoding for Python source code and conversion methods is UTF-8.
"str" stores Unicode character strings as lists of Unicode code points.
len(str) returns the number of Unicode code points, not the number of human language characters.
\N{...}, \x.., \u...., and \U........ escape sequences in "str" literals can be used to specify Unicode code points.
str.encode() converts character strings to byte sequences.
bytes.decode() converts byte sequences to character strings.
"unicodedata" module offers static methods to retrieve properties associated with code points defined by the Unicode standard.