"token" Datatype Values and Representations

This section describes the built-in datatype 'token' derived from 'normalizedString'. Whitespace characters are collapsed to a single ' ' character. Leading and trailing whitespace characters are removed.

The next "string" derived dataype we are going to look is: "token".

"token" is a datatype derived from "normalizedString" datatype by collapsing whitespace characters to space characters and removing leading/trailing space characters.

"token" value set is smaller than "normalizedString" value set. It contains only unique "normalizedString" values after consecutive whitespace characters of '\t', '\r', '\n' and ' ' are collapsed to a single ' '; and leading and/or trailing ' ' are removed.

Note that repeating whitespace characters are still allowed in "token" lexical representations. But they are replaced by a single ' ' to obtain "token" values. For example, 5 XML elements below are all valid and represent the same "normalizedString" value:

  <Token>Herong Yang</Token>
  <Token>Herong  Yang</Token>
  <Token>Herong&#x20;Yang</Token>
  <Token>   Herong   Yang   </Token>
  <Token>   Herong
Yang</Token>

A close example of using "token" datatype in XML is the <P> element in HTML documents, where all whitespace characters are collapsed.

Here is a sample XSD document that defines a sub element <Token> to use "token" values:

<?xml version="1.1"?>
<!-- token_datatype_test.xsd
 - Copyright (c) 2002-2013 HerongYang.com. All Rights Reserved.
-->
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="Token_Datatype_Test">
  <xs:complexType>
    <xs:sequence>
      <xs:element name="Token" type="xs:token"
        maxOccurs="unbounded"/>
    </xs:sequence>
  </xs:complexType>
</xs:element>
</xs:schema>

Here is a sample XML document that contains <Token> elements conforming with that definition:

<?xml version="1.1"?>
<!-- token_datatype_test.xml
 - Copyright (c) 2002-2013 HerongYang.com. All Rights Reserved.
-->
<Token_Datatype_Test>

<!-- 5 valid "token" elements represent the same value -->
  <Token>Herong Yang</Token>
  <Token>Herong  Yang</Token>
  <Token>Herong&#x20;Yang</Token>
  <Token>   Herong   Yang   </Token>
  <Token>   Herong
Yang</Token>

<!-- 3 valid "token" elements represent different values -->
  <Token>Herong Yang</Token>
  <Token>Herong_Yang</Token>
  <Token>Herong.Yang</Token>
</Token_Datatype_Test>

You can modify this example to try other "token" lexical representations and values.

Table of Contents

 About This Book

 Introduction to XML Schema

 XML Editor and Schema Processor - XMLPad

 Java API for XML Processing - JAXP

 JAXP - XML Schema (XSD) Validation

 Xerces2 Java Parser - Java API of XML Parsers

 Using Xerces2 Java APIs

 XML Schema Language - Basics

 Introduction of XSD Built-in Datatypes

"string" and Its Derived Datatypes

 "string" Datatype Values and Representations

 "normalizedString" Datatype Values and Representations

"token" Datatype Values and Representations

 "language" Datatype Values and Representations

 "language" Datatype Values - Invalid Inputs

 "Name" Datatype Values and Representations

 "NMTOKEN" Datatype Values and Representations

 "NCName" Datatype Values and Representations

 "ENTITY" Datatype Values and Representations

 "ID" Datatype Values and Representations

 "IDREF" Datatype Values and Representations

 "decimal" and Its Derived Datatypes

 "dateTime" and Its Related Datatypes

 Miscellaneous Built-in Datatypes

 Facets, Constraining Facets and Restriction Datatypes

 "simpleType" - Defining Your Own Simple Datatypes

 Complex Element Declaration

 Identity-Constraints: unique, key and keyref

 Assertion as Custom Validation Rules

 XML Schema Location and Namespace in XML Documents

 Overriding Element Types in XML Documents

 Linking Multiple Schema Documents Together

 Glossary

 Archived Tutorials

 References

 Full Version in PDF/EPUB