Public Member Functions | |
TZToken | NextToken (TStream Stream, Char FirstChar, TZTokenizer Tokenizer) |
The tokenizer will typically have a character state table that decides which state to use, depending on an initial character. If a single character is insufficient, a state such as SlashState
will read a second character, and may delegate to another state, such as SlashStarState
. This prospect of delegation is the reason that the nextToken()
method has a tokenizer argument.
Definition at line 139 of file ZTokenizer.pas.
TZToken TZTokenizerState::NextToken | ( | TStream | Stream, | |
Char | FirstChar, | |||
TZTokenizer | Tokenizer | |||
) |
Reimplemented in TZExpressionNumberState, TZExpressionQuoteState, TZExpressionCommentState, TZExpressionWordState, TZNumberState, TZQuoteState, TZCommentState, TZCppCommentState, TZCCommentState, TZSymbolState, TZWhitespaceState, TZWordState, TZGenericSQLWordState, TZGenericSQLQuoteState, TZMySQLNumberState, TZMySQLQuoteState, TZMySQLCommentState, TZPostgreSQLNumberState, TZPostgreSQLQuoteState, TZPostgreSQLCommentState, TZSQLiteNumberState, TZSQLiteQuoteState, TZSQLiteCommentState, TZSybaseNumberState, TZSybaseQuoteState, and TZSybaseCommentState.