TZSymbolState Class Reference

The idea of a symbol is a character that stands on its own, such as an ampersand or a parenthesis. More...

Inheritance diagram for TZSymbolState:

TZTokenizerState TZExpressionSymbolState TZGenericSQLSymbolState TZInterbaseSymbolState TZMySQLSymbolState TZOracleSymbolState TZPostgreSQLSymbolState TZSQLiteSymbolState TZSybaseSymbolState

List of all members.

Public Member Functions

 Add (const string Value)
TZToken NextToken (TStream Stream, Char FirstChar, TZTokenizer Tokenizer)
 TZSymbolState ()
 ~TZSymbolState ()

Protected Attributes

TZSymbolRootNode Symbols

Private Attributes

TZSymbolRootNode FSymbols


Detailed Description

The idea of a symbol is a character that stands on its own, such as an ampersand or a parenthesis.

For example, when tokenizing the expression (isReady)& (isWilling) , a typical tokenizer would return 7 tokens, including one for each parenthesis and one for the ampersand. Thus a series of symbols such as )&( becomes three tokens, while a series of letters such as isReady becomes a single word token.

Multi-character symbols are an exception to the rule that a symbol is a standalone character. For example, a tokenizer may want less-than-or-equals to tokenize as a single token. This class provides a method for establishing which multi-character symbols an object of this class should treat as single symbols. This allows, for example, "cat <= dog" to tokenize as three tokens, rather than splitting the less-than and equals symbols into separate tokens.

By default, this state recognizes the following multi- character symbols: !=, :-, <=, >=

Definition at line 430 of file ZTokenizer.pas.


Constructor & Destructor Documentation

TZSymbolState::TZSymbolState (  ) 

TZSymbolState::~TZSymbolState (  ) 


Member Function Documentation

TZSymbolState::Add ( const string  Value  ) 

TZToken TZSymbolState::NextToken ( TStream  Stream,
Char  FirstChar,
TZTokenizer  Tokenizer 
)

Reimplemented from TZTokenizerState.


Member Data Documentation

Definition at line 435 of file ZTokenizer.pas.

See also:
FSymbols For reading

FSymbols For writing

Definition at line 439 of file ZTokenizer.pas.


The documentation for this class was generated from the following file:

Generated on Wed Dec 30 08:42:58 2009 for zeoslib by  doxygen 1.5.7.1