The string tokenizer class allows an application to break a string into tokens. The tokenization method is much simpler than the one used by the <code>StreamTokenizer</code> class. The <code>StringTokenizer</code> methods do not distinguish among identifiers, numbers, and quoted strings, nor do they recognize and skip comments. <p> The set of delimiters (the characters that separate tokens) may be specified either at creation time or on a per-token basis. <p> An instance of <code>StringTokenizer</code> behaves in one of two ways, depending on whether it was created with the <code>returnDelims</code> flag having the value <code>true</code> or <code>false</code>: <ul> <li>If the flag is <code>false</code>, delimiter characters serve to separate tokens. A token is a maximal sequence of consecutive characters that are not delimiters. <li>If the flag is <code>true</code>, delimiter characters are themselves considered to be tokens. A token is thus either one delimiter character, or a maximal sequence of consecutive characters that are not delimiters. </ul><p> A <tt>StringTokenizer</tt> object internally maintains a current position within the string to be tokenized. Some operations advance this current position past the characters processed.<p> A token is returned by taking a substring of the string that was used to create the <tt>StringTokenizer</tt> object. <p> The following is one example of the use of the tokenizer. The code: <blockquote><pre> StringTokenizer st = new StringTokenizer("this is a test"); while (st.hasMoreTokens()) { System.out.println(st.nextToken()); } </pre></blockquote> <p> prints the following output: <blockquote><pre> this is a test </pre></blockquote>