Description:

Implementation of the default parser of module ALib Expressions.

This internal class is not too well documented. Nevertheless, it is designed to be able to tweak its behavior slightly and in case of need, even derive and use a custom parser class. For doing so, please consult the source code of this class. A custom parser might be set to protected field Compiler::parserby a derived compiler type before compiling a first expression.

It is possible to define scannable custom unary and binary operators. Definitions of binary operators include a "precedence value" that allows aligning them with the built-in types. Also, built-in operators can be removed if wanted.

Definition at line 23 of file parser_impl.inl.

Inheritance diagram for alib::expressions::detail::ParserImpl:

[legend]

Collaboration diagram for alib::expressions::detail::ParserImpl:

[legend]

Public Method Index:
	ParserImpl (Compiler &compiler, MonoAllocator &allocator)

virtual	~ParserImpl () override
	Virtual destructor.

virtual ALIB_DLL detail::AST *	Parse (const String &exprString, NumberFormat *nf) override

Public Method Index: inherited from alib::expressions::detail::Parser
virtual	~Parser ()
	Virtual destructor.

Protected Type Index:
enum class	Tokens : char { EOT = '\0' , SymbolicOp = 'O' , AlphaUnOp = 'U' , AlphaBinOp = 'B' , LitString = 'S' , LitInteger = 'I' , LitFloat = 'F' , Identifier = 'A' , BraceOpen = '(' , BraceClose = ')' , Comma = ',' , SubscriptOpen = '[' , SubscriptClose = ']' }
	Types of tokens. More...

Protected Field Index:
StdVectorMono< AST * > *	ASTs

HashSet< MonoAllocator, String, alib::hash_string_ignore_case< character >, alib::equal_to_string_ignore_case< character > >	binaryOperators

Compiler &	compiler
	The compiler that this parser works for.

MonoAllocator &	compileTimeAllocator

String	expression
	The given expression to parse.

NumberFormat *	numberFormat
	Used for scanning literals. Provided to this class with each parse request.

BitSet< 256 >	operatorChars

Substring	scanner
	The rest of expression.

BitSet< 256 >	syntaxTokens

Tokens	token
	The actual token type.

double	tokFloat
	Float value of token (if applicable).

integer	tokInteger
	Integer value of token (if applicable).

ASTLiteral::NFHint	tokLiteralHint
	The actual token type.

integer	tokPosition
	The position of the token in expression.

String	tokString
	String value of token (if applicable).

HashSet< MonoAllocator, String, alib::hash_string_ignore_case< character >, alib::equal_to_string_ignore_case< character > >	unaryOperators

Protected Method Index:
ALIB_DLL String	getBinaryOp ()

ALIB_DLL String	getUnaryOp ()

void	NextToken ()
	This is the "scanner" or "lexer" method.

AST *	parseBinary ()

AST *	parseConditional ()

AST *	parseSimple ()

AST *	parseSubscript (AST *function)

AST *	pop ()

AST *	push (AST *ast)

AST *	replace (AST *ast)

AST *	top ()

Additional Inherited Members
Public Static Method Index: inherited from alib::expressions::detail::Parser
static ALIB_DLL Parser *	Create (Compiler &compiler)

Enumeration Details:

◆ Tokens

enum class alib::expressions::detail::ParserImpl::Tokens : char

strongprotected

Types of tokens.

Enumerator
EOT	End of tokens. (No next token available.)
SymbolicOp	A symbolic operator. Can be unary or binary.
AlphaUnOp	An alphabetic unary operator.
AlphaBinOp	An alphabetic binary operator.
LitString	a string literal.
LitInteger	a integer literal.
LitFloat	a float literal.
Identifier	An identifier.
BraceOpen	An opening brace.
BraceClose	A closing brace.
Comma	A comma.
SubscriptOpen	An opening subscript brace.
SubscriptClose	A closing subscript brace.

Definition at line 27 of file parser_impl.inl.

Field Details:

◆ ASTs

StdVectorMono<AST*>* alib::expressions::detail::ParserImpl::ASTs

protected

List of ASTs currently created in recursion.

Note: This vector is created in the monotonic allocator and never even deleted, as all inserted AST elements, exclusively allocate from the same temporary allocator.

Definition at line 121 of file parser_impl.inl.

◆ binaryOperators

HashSet< MonoAllocator, String, alib::hash_string_ignore_case <character>, alib::equal_to_string_ignore_case<character> > alib::expressions::detail::ParserImpl::binaryOperators

protected

Hash set of binary operators. The key of the table is the operator string, which usually consists of one to three characters, like '+' or '<<='.

This table is filled in the constructor of the class with the values stored in Compiler::BinaryOperators and used for testing of existence.

Definition at line 115 of file parser_impl.inl.

◆ compiler

Compiler& alib::expressions::detail::ParserImpl::compiler

protected

The compiler that this parser works for.

Definition at line 71 of file parser_impl.inl.

◆ compileTimeAllocator

MonoAllocator& alib::expressions::detail::ParserImpl::compileTimeAllocator

protected

Memory for temporary allocations, like AST objects or literal strings with converted escape sequences. Provided by the compiler with the method Parse.

Definition at line 50 of file parser_impl.inl.

◆ expression

String alib::expressions::detail::ParserImpl::expression

protected

The given expression to parse.

Definition at line 78 of file parser_impl.inl.

◆ numberFormat

NumberFormat* alib::expressions::detail::ParserImpl::numberFormat

protected

Used for scanning literals. Provided to this class with each parse request.

Definition at line 74 of file parser_impl.inl.

◆ operatorChars

BitSet<256> alib::expressions::detail::ParserImpl::operatorChars

protected

Lists single characters that got found in operator strings which have been registered with Compiler::AddUnaryOperator and Compiler::AddBinaryOperator.

Used by the internal token scanner (lexer) and by default will become something like "=+-*%/?:~!|&^!<>/%". when found in the expression string.

Definition at line 95 of file parser_impl.inl.

◆ scanner

Substring alib::expressions::detail::ParserImpl::scanner

protected

The rest of expression.

Definition at line 81 of file parser_impl.inl.

◆ syntaxTokens

BitSet<256> alib::expressions::detail::ParserImpl::syntaxTokens

protected

Lists single characters that get directly converted into tokens of corresponding type when found in the expression string. Tokens are "()[],".

Definition at line 85 of file parser_impl.inl.

◆ token

Tokens alib::expressions::detail::ParserImpl::token

protected

The actual token type.

Definition at line 53 of file parser_impl.inl.

◆ tokFloat

double alib::expressions::detail::ParserImpl::tokFloat

protected

Float value of token (if applicable).

Definition at line 62 of file parser_impl.inl.

◆ tokInteger

integer alib::expressions::detail::ParserImpl::tokInteger

protected

Integer value of token (if applicable).

Definition at line 59 of file parser_impl.inl.

◆ tokLiteralHint

ASTLiteral::NFHint alib::expressions::detail::ParserImpl::tokLiteralHint

protected

The actual token type.

Definition at line 56 of file parser_impl.inl.

◆ tokPosition

integer alib::expressions::detail::ParserImpl::tokPosition

protected

The position of the token in expression.

Definition at line 68 of file parser_impl.inl.

◆ tokString

String alib::expressions::detail::ParserImpl::tokString

protected

String value of token (if applicable).

Definition at line 65 of file parser_impl.inl.

◆ unaryOperators

HashSet< MonoAllocator, String, alib::hash_string_ignore_case <character>, alib::equal_to_string_ignore_case<character> > alib::expressions::detail::ParserImpl::unaryOperators

protected

Hash set of unary operators. The key of the table is the operator string, which usually consists of one character, like '-' or '!'.

This table is filled in the constructor of the class with the values stored in Compiler::UnaryOperators and used for testing of existence.

Definition at line 105 of file parser_impl.inl.

Constructor(s) / Destructor Details:

◆ ParserImpl()

alib::expressions::detail::ParserImpl::ParserImpl	(	Compiler &	compiler,
		MonoAllocator &	allocator )

Constructor.

Parameters

compiler	The compiler that this parser works for.
allocator	A monotonic allocator for permanent allocations.

Definition at line 31 of file parser_impl.cpp.

◆ ~ParserImpl()

virtual alib::expressions::detail::ParserImpl::~ParserImpl ( )

inlineoverridevirtual

Virtual destructor.

Definition at line 133 of file parser_impl.inl.

Method Details:

◆ getBinaryOp()

String alib::expressions::detail::ParserImpl::getBinaryOp ( )

protected

Tests if the actual token represents a known binary operator.

Returns: Returns the binary operator symbol, respectively a nulled string on failure.

Definition at line 605 of file parser_impl.cpp.

Here is the call graph for this function:

◆ getUnaryOp()

String alib::expressions::detail::ParserImpl::getUnaryOp ( )

protected

Tests if the actual token represents a known unary operator.

Returns: Returns the binary operator symbol, respectively a nulled string on failure.

Definition at line 569 of file parser_impl.cpp.

Here is the call graph for this function:

◆ NextToken()

void alib::expressions::detail::ParserImpl::NextToken ( )

protected

This is the "scanner" or "lexer" method.

Definition at line 105 of file parser_impl.cpp.

◆ Parse()

detail::AST * alib::expressions::detail::ParserImpl::Parse	(	const String &	exprString,
		NumberFormat *	nf )

overridevirtual

Parses the given expression string.

Note: The return value is hidden by using void*. This is to allow avoid flooding of boost header includes files to the code entities using module ALib Expressions.

Parameters

exprString	The string to parse.
nf	Used to scan number literals.

Returns: The abstract syntax tree representing the expression.

Implements alib::expressions::detail::Parser.

Definition at line 309 of file parser_impl.cpp.

◆ parseBinary()

AST * alib::expressions::detail::ParserImpl::parseBinary ( )

protected

Internal method that optionally parses a binary operator and levels (recursively) trees of such according to operator precedence and brackets given.

Returns: T.

Definition at line 374 of file parser_impl.cpp.

◆ parseConditional()

AST * alib::expressions::detail::ParserImpl::parseConditional ( )

protected

Internal method that optionally parses a conditional operator (Q ? T : F )

Returns: T.

Definition at line 340 of file parser_impl.cpp.

◆ parseSimple()

AST * alib::expressions::detail::ParserImpl::parseSimple ( )

protected

Parses unary ops, literals, identifiers, functions and expressions surrounded by brackets.

Returns: The abstract syntax tree node parsed.

Definition at line 430 of file parser_impl.cpp.

Here is the call graph for this function:

◆ parseSubscript()

AST * alib::expressions::detail::ParserImpl::parseSubscript ( AST * function )

protected

Invoked after an identifier or function was parsed. Tests for subscript operator, otherwise returns the given ast as is.

Parameters

function The identifier or function parsed.

Returns: Either the given node or a node of type ASTBinaryOp with lhs set to function, rhs to the parsed subscript arg and and operator set to '[]'.

Definition at line 539 of file parser_impl.cpp.

Here is the call graph for this function:

◆ pop()

AST * alib::expressions::detail::ParserImpl::pop ( )

inlineprotected

Simple shortcut popping and returning last ast from the current list.

Returns: Popped AST object.

Definition at line 197 of file parser_impl.inl.

◆ push()

AST * alib::expressions::detail::ParserImpl::push ( AST * ast )

inlineprotected

Simple shortcut pushing an ast to current list and returning it.

Parameters

ast	The AST node to push.

Returns: Popped AST object.

Definition at line 207 of file parser_impl.inl.

◆ replace()

AST * alib::expressions::detail::ParserImpl::replace ( AST * ast )

inlineprotected

Simple shortcut replacing the topmost ast.

Parameters

ast	The new AST node to replace the existing one with.

Returns: The given object.

Definition at line 223 of file parser_impl.inl.

◆ top()

AST * alib::expressions::detail::ParserImpl::top ( )

inlineprotected

Simple shortcut to the topmost AST.

Returns: The topmost AST object.

Definition at line 215 of file parser_impl.inl.

The documentation for this class was generated from the following files:

Description:

Public Method Index:

Protected Type Index:

Protected Field Index:

Protected Method Index:

Additional Inherited Members

Enumeration Details:

◆ Tokens

Field Details:

◆ ASTs

◆ binaryOperators

◆ compiler

◆ compileTimeAllocator

◆ expression

◆ numberFormat

◆ operatorChars

◆ scanner

◆ syntaxTokens

◆ token

◆ tokFloat

◆ tokInteger

◆ tokLiteralHint

◆ tokPosition

◆ tokString

◆ unaryOperators

Constructor(s) / Destructor Details:

◆ ParserImpl()

◆ ~ParserImpl()

Method Details:

◆ getBinaryOp()

◆ getUnaryOp()

◆ NextToken()

◆ Parse()

◆ parseBinary()

◆ parseConditional()

◆ parseSimple()

◆ parseSubscript()

◆ pop()

◆ push()

◆ replace()

◆ top()