Features of UTF-8 Encoding

This section provides a quick summary of features of UTF-8 encoding. UTF-8 is very efficient for Western language characters.

Features of UTF-8 encoding:

1.UTF-8 is very efficient in storage for characters in Western languages. The code points for most of characters in Western languages are in the U+0000...U+007F range, which will be encoded with 1 byte per character. The rest of characters will be encoded by two bytes. Some reports says that any text data in Western language will be encoded with 1.1 bytes per character on average. This is a big saving comparing with any other Unicode encodings.

2. UTF-8 is compatible with the single byte ASCII encoding. In another word, ASCII encoding can be viewed as a sub set of the UTF-8 encoding. Any ASCII byte stream is a valid UTF-8 byte stream.

3. UTF-8 is backward compatible with Unicode 3.0 character set, which only contains characters in the U+0000...U+FFFF range.

4. UTF-8 is less efficient in storage for characters in CJK (Chinese, Japanese, and Korean) languages comparing to UTF-16 encoding. The code points for most of characters in CJK languages are in the U+000800...U+00FFFF range, which will be encoded in UTF-8 with 3 bytes per character. UTF-16 will encode all characters 2 bytes per character.

5. Processing a UTF-8 encoded text files is relatively easy. If the leading bit of the first byte is a 0, then 1 byte is used to encoding this character. If the leading bit of the first byte is not a 0, then the number of non-zero leading bits is total number bytes used to encoding this character.

6. Another nice nature of UTF-8 encoding is that all subsequent bytes in encoded multi-byte sequence are having the pattern of 0b10xxxxxx in binary format. If you are looking at one byte of an encoded character in the middle of the encoded stream, and want to find out the first byte of this encoded character, you just need to follow this simple logic:

   while (current byte matches the bit pattern '10xxxxxx') {
      Current byte = previous byte
   }

Last update: 2009.

Table of Contents

 About This Book

 Character Sets and Encodings

 ASCII Character Set and Encoding

 GB2312 Character Set and Encoding

 GB18030 Character Set and Encoding

 JIS X0208 Character Set and Encodings

 Unicode Character Set

UTF-8 (Unicode Transformation Format - 8-Bit)

 UTF-8 Encoding

 UTF-8 Encoding Algorithm

Features of UTF-8 Encoding

 UTF-16, UTF-16BE and UTF-16LE Encodings

 UTF-32, UTF-32BE and UTF-32LE Encodings

 Java Language and Unicode Characters

 Character Encoding in Java

 Character Set Encoding Maps

 Encoding Conversion Programs for Encoded Text Files

 Using Notepad as a Unicode Text Editor

 Using Microsoft Word as a Unicode Text Editor

 Using Microsoft Excel as a Unicode Text Editor

 Unicode Fonts

 Unicode Code Point Blocks - Code Charts

 Outdated Tutorials

 References

 PDF Printing Version