    All Packages  Class Hierarchy  This Package  Previous  Next  Index

# Class xe.xe_Tokenizer

    java.lang.Object
       |
       +----xe.xe_Tokenizer

public class xe_Tokenizer
extends Object

Perform low-level parsing of an XML source and keep the state of the current parse.

The source is supplied via a Reader, which must be set by calling SetSource() prior to calling the range of methods which perform detailed parsing of various sorts. Most of the parsing methods pass back their results in the form of xe_Tokens.

##

xe_Tokenizer()
  Default constructor.

##

atEnd()
  Determine if we have reached the end of the input source.
getLineCount()
  Return the number of lines parsed so far.
getStopIfVerifyErrorFlag()
  Find out whether we check the semantics of the source.
getStrictXmlSyntaxFlag()
  Find out whether strict XML syntax is to be rigidly applied.
getTotalCharCount()
  Return the number of characters parsed so far.
getValidateFlag()
  Find out whether we validate the source.
parseContentToken()

  Parse the next piece of the content of an element, and return it as a token.

parseLiteralToken(boolean)
  Parse the next token as a literal.
parseLiteralToken(char)
  Parse the next token as a literal, assuming that its opening quote (either single or double) has already been parsed.
parseName()
  Parse a name.
parseName(StringBuffer)
  Parse a name, which must start with a character acceptable at the start of a name and whose remaining characters must be acceptable within a name.
parseNameToken()
  Parse the next token as a name.
parseNameToken(boolean)
  Parse the next token as a name.
parseNameTokenExpected(String, boolean)
  Parse the next token as a name, and check it has the expected value.
parseNextToken(boolean)
  Parse the next token.
parseNextTokenExpected(int, boolean)
  Parse the next token, and make sure it is of the expected type; an exception is thrown if it is not.
parseNumber()
  Parse a decimal integer number, which can only include 0-9.
parseUntil(char)
  Parse until we encounter the InputEndChar (or the end of the source).
parseUntil(String)
  Parse until we encounter the InputEndString (or the end of the source).
parseWhitespace(boolean)
  Parse all whitespace characters, until we find a non-whitespace character.
readNextChar()
  Read the next character from the source.
reportWarning(String)
  Report a warning message.
reportWarning(String, int)
  Report a warning message.
resetStatistics()
  Reset the counts of lines, etc., ready for another parse.
setSource(Reader)
  Set the reader from which the XML source is to be read.
setStopIfVerifyErrorFlag(boolean)
  Specify whether we check the semantics of the source.
setStrictXmlSyntaxFlag(boolean)
  Specify whether strict XML syntax is to be rigidly applied.
setValidateFlag(boolean)
  Specify whether we validate the source.
throwParseException(String)
  Throw a parse exception, using the InputErrorMessage.
throwParseException(String, int, int)
  Throw a parse exception, using the InputErrorMessage.

##

xe_Tokenizer

     public xe_Tokenizer()

  Default constructor.

##

parseContentToken

     public xe_Token parseContentToken() throws xm_ParseException, IOException

  Parse the next piece of the content of an element, and return it as a token. Content can comprise any combination of the following types of entity:

  Reference: &Name; or &#nn; or &#xnn;
  CDATA section:
  Comment:
  Processing instruction:
  Element: ...
  PCData: Text

  Returns:
    Token representing the piece of content parsed
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseNextTokenExpected

     public xe_Token parseNextTokenExpected(int InputExpectedTokenType,
                                            boolean InputConsumeWhitespaceFlag) throws xm_ParseException, IOException

  Parse the next token, and make sure it is of the expected type; an exception is thrown if it is not.

  Parameters:
    InputExpectedTokenType - The type of token we expect
    InputConsumeWhitespaceFlag - If true, all preceding whitespace is separated; otherwise, it is treated as part of the token itself
  Returns:
    Token representing the value parsed
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseNextToken

     public xe_Token parseNextToken(boolean InputConsumeWhitespaceFlag) throws xm_ParseException, IOException

  Parse the next token.

  Parameters:
    InputConsumeWhitespaceFlag - If true, all preceding whitespace is separated; otherwise, it is treated as part of the token itself
  Returns:
    Token representing the value parsed
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseNameTokenExpected

     public xe_Token parseNameTokenExpected(String InputExpectedValue,
                                            boolean InputConsumeWhitespaceFlag) throws xm_ParseException, IOException

  Parse the next token as a name, and check it has the expected value. It should only be called if the next token must be a name - and throws an exception if it is not. It also throws an exception if the name does not have the expected value.

  Parameters:
    InputExpectedValue - The value the token is expected to have
    InputConsumeWhitespaceFlag - If true, all preceding whitespace is separated; otherwise, it is not allowed (ie. is an error)
  Returns:
    NAME token
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseNameToken

     protected xe_Token parseNameToken(boolean InputConsumeWhitespaceFlag) throws xm_ParseException, IOException

  Parse the next token as a name. It should only be called if the next token must be a name - and throws an exception if it is not.

  Parameters:
    InputConsumeWhitespaceFlag - If true, all preceding whitespace is separated; otherwise, it is not allowed (ie. is an error)
  Returns:
    NAME token
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseNameToken

     protected xe_Token parseNameToken() throws xm_ParseException, IOException

  Parse the next token as a name. It should only be called if a valid name start character has already been found, or the next token must be a name.

  Returns:
    NAME token
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseName

     public String parseName() throws xm_ParseException, IOException

  Parse a name.

  Returns:
    A name as a string
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseName

     protected void parseName(StringBuffer InputNameStringBuffer) throws xm_ParseException, IOException

  Parse a name, which must start with a character acceptable at the start of a name and whose remaining characters must be acceptable within a name. (It's rather obvious, really!) The name which is parsed will be added to the InputNameStringBuffer (which doesn't have to be empty beforehand). Currently no namespace support is provided.

  Parameters:
    InputNameStringBuffer - Buffer to which the name parsed is added
  Returns:
    A name as a string
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseNumber

     protected String parseNumber() throws xm_ParseException, IOException

  Parse a decimal integer number, which can only include 0-9.

  Returns:
    The number as a string
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseWhitespace

     public String parseWhitespace(boolean InputWantWhitespaceFlag) throws xm_ParseException, IOException

  Parse all whitespace characters, until we find a non-whitespace character.

  Parameters:
    InputWantWhitespaceFlag - If true, we parse whitespace; if false we do nothing. While the latter seems pointless, it obviates checks elsewhere.
  Returns:
    String of the whitespace which was read, or a null String if there was no whitespace
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseLiteralToken

     public xe_Token parseLiteralToken(boolean InputConsumeWhitespaceFlag) throws xm_ParseException, IOException

  Parse the next token as a literal. We expect an opening quote (either single or double), some text and then a closing quote (the same as the opening one). The quotes themselves are not stored as part of the literal itself.

  Parameters:
    InputConsumeWhitespaceFlag - If true, all preceding whitespace is separated; otherwise, whitespace is treated as part of the main string
  Returns:
    Token representing the value parsed
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseLiteralToken

     public xe_Token parseLiteralToken(char InputStartQuoteChar) throws xm_ParseException, IOException

  Parse the next token as a literal, assuming that its opening quote (either single or double) has already been parsed. We expect some text and then a closing quote (the same as the opening one). The quotes themselves are not stored as part of the literal itself.

  Parameters:
    InputStartQuoteChar - The opening quote character
  Returns:
    Token representing the literal value parsed
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseUntil

     public String parseUntil(String InputEndString) throws xm_ParseException, IOException

  Parse until we encounter the InputEndString (or the end of the source). The InputEndString is omitted from the end of the resulting value.

  Parameters:
    InputEndString - The string to look for
  Returns:
    String representing the value parsed
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

parseUntil

     public String parseUntil(char InputEndChar) throws xm_ParseException, IOException

  Parse until we encounter the InputEndChar (or the end of the source). The InputEndChar is omitted from the end of the resulting value.

  Parameters:
    InputEndChar - The character to look for
  Returns:
    String representing the value parsed
  Throws: xm_ParseException
    XML wellformedness error
  Throws: IOException
    Error reading from source reader

throwParseException

     public void throwParseException(String InputErrorMessage,
                                     int InputStartPosition,
                                     int InputEndPosition) throws xm_ParseException

  Throw a parse exception, using the InputErrorMessage.

  Parameters:
    InputErrorMessage - Message text
    InputStartPosition - Start of area in source to highlight
    InputEndPosition - End of area in source to highlight
  Throws: xm_ParseException
    An exception whose throwing has been explicitly requested

throwParseException

     public void throwParseException(String InputErrorMessage) throws xm_ParseException

  Throw a parse exception, using the InputErrorMessage.

  Throws: xm_ParseException
    An exception whose throwing has been explicitly requested

reportWarning

     public void reportWarning(String InputMessage)

  Report a warning message.

  Parameters:
    InputMessage - The text of the main message to report

reportWarning

     public void reportWarning(String InputMessage,
                               int InputColumnOffset)

  Report a warning message.

  Parameters:
    InputMessage - The text of the main message to report
    InputColumnOffset - The number of characters back from where we are now at which the event occurred

setSource

     public void setSource(Reader InputSourceReader) throws IOException

  Set the reader from which the XML source is to be read.

  Parameters:
    InputSourceStream - The reader from which to read source to parse
  Throws: IOException
    Error reading first character from source reader

setValidateFlag

     public void setValidateFlag(boolean InputValidateFlag)

  Specify whether we validate the source.

  Parameters:
    InputValidateFlag - True means we validate

setStopIfVerifyErrorFlag

     public void setStopIfVerifyErrorFlag(boolean InputStopIfVerifyErrorFlag)

  Specify whether we check the semantics of the source.

  Parameters:
    InputStopIfVerifyErrorFlag - True means we check semantics

setStrictXmlSyntaxFlag

     public void setStrictXmlSyntaxFlag(boolean InputStrictXmlSyntaxFlag)

  Specify whether strict XML syntax is to be rigidly applied.

  Parameters:
    InputStrictXmlSyntaxFlag - True means apply strict XML syntax

resetStatistics

     public void resetStatistics()

  Reset the counts of lines, etc., ready for another parse.

getValidateFlag

     public boolean getValidateFlag()

  Find out whether we validate the source.

  Returns:
    s True means we validate

getStopIfVerifyErrorFlag

     public boolean getStopIfVerifyErrorFlag()

  Find out whether we check the semantics of the source.

  Returns:
    s True means we check semantics

getStrictXmlSyntaxFlag

     public boolean getStrictXmlSyntaxFlag()

  Find out whether strict XML syntax is to be rigidly applied.

  Returns:
    s True means we apply strict XML syntax

getLineCount

     public int getLineCount()

  Return the number of lines parsed so far.

  Returns:
    The number of lines parsed up to this point

getTotalCharCount

     public int getTotalCharCount()

  Return the number of characters parsed so far.

  Returns:
    The number of characters parsed up to this point

atEnd

     protected boolean atEnd()

  Determine if we have reached the end of the input source.

  Returns:
    True if at end of source

readNextChar

     protected void readNextChar() throws IOException

  Read the next character from the source. Input is not buffered.

  Throws: IOException
    Error reading from source reader

    All Packages  Class Hierarchy  This Package  Previous  Next  Index