Description:

Implementation of the default parser of module ALib Expressions.

This internal class is not too well documented. Nevertheless, it is designed to be able to tweak its behavior slightly and in case of need, even derive and use a custom parser class. For doing so, please consult the source code of this class. A custom parser might be set to protected field Compiler::parserby a derived compiler type before compiling a first expression.

It is possible to define scannable custom unary and binary operators. Definitions of binary operators include a "precedence value" that allows aligning them with the built-in types. Also, built-in operators can be removed if wanted.

Definition at line 40 of file parser_impl.hpp.

#include <parser_impl.hpp>

Inheritance diagram for ParserImpl:

[legend]

Collaboration diagram for ParserImpl:

[legend]

Public Method Index:
	ParserImpl (Compiler &compiler, MonoAllocator &allocator)

virtual	~ParserImpl () override
	Virtual destructor.

virtual ALIB_API detail::AST *	Parse (const String &exprString, NumberFormat *nf) override

Public Method Index: inherited from Parser
virtual	~Parser ()
	Virtual destructor.

Protected Type Index:
enum class	Tokens : char { EOT = '\0' , SymbolicOp = 'O' , AlphaUnOp = 'U' , AlphaBinOp = 'B' , LitString = 'S' , LitInteger = 'I' , LitFloat = 'F' , Identifier = 'A' , BraceOpen = '(' , BraceClose = ')' , Comma = ',' , SubscriptOpen = '[' , SubscriptClose = ']' }
	Types of tokens. More...

Protected Field Index:
StdVectorMono< AST * > *	ASTs

HashSet< MonoAllocator, String, alib::hash_string_ignore_case< character >, alib::equal_to_string_ignore_case< character > >	binaryOperators

Compiler &	compiler
	The compiler that this parser works for.

MonoAllocator &	compileTimeAllocator

String	expression
	The given expression to parse.

NumberFormat *	numberFormat
	Used for scanning literals. Provided to this class with each parse request.

std::bitset< 256 >	operatorChars

Substring	scanner
	The rest of expression.

std::bitset< 256 >	syntaxTokens

Tokens	token
	The actual token type.

double	tokFloat
	Float value of token (if applicable).

integer	tokInteger
	Integer value of token (if applicable).

ASTLiteral::NFHint	tokLiteralHint
	The actual token type.

integer	tokPosition
	The position of the token in expression.

String	tokString
	String value of token (if applicable).

HashSet< MonoAllocator, String, alib::hash_string_ignore_case< character >, alib::equal_to_string_ignore_case< character > >	unaryOperators

Protected Method Index:
ALIB_API String	getBinaryOp ()

ALIB_API String	getUnaryOp ()

void	NextToken ()
	This is the "scanner" or "lexer" method.

AST *	parseBinary ()

AST *	parseConditional ()

AST *	parseSimple ()

AST *	parseSubscript (AST *function)

AST *	pop ()

AST *	push (AST *ast)

AST *	replace (AST *ast)

AST *	top ()

Additional Inherited Members
Public Static Method Index: inherited from Parser
static ALIB_API Parser *	Create (Compiler &compiler)

Enumeration Details:

◆ Tokens

enum class Tokens : char

strongprotected

Types of tokens.

Definition at line 44 of file parser_impl.hpp.

Field Details:

◆ ASTs

StdVectorMono<AST*>* ASTs

protected

List of ASTs currently created in recursion.

Note: This vector is created in the monotonic allocator and never even deleted, as all inserted AST elements, exclusively allocate from the same temporary allocator.

Definition at line 138 of file parser_impl.hpp.

◆ binaryOperators

HashSet< MonoAllocator, String, alib::hash_string_ignore_case <character>, alib::equal_to_string_ignore_case<character> > binaryOperators

protected

Hash set of binary operators. The key of the table is the operator string, which usually consists of one to three characters, like '+' or '<<='.

This table is filled in the constructor of the class with the values stored in Compiler::BinaryOperators and used for testing of existence.

Definition at line 132 of file parser_impl.hpp.

◆ compiler

Compiler& compiler

protected

The compiler that this parser works for.

Definition at line 88 of file parser_impl.hpp.

◆ compileTimeAllocator

MonoAllocator& compileTimeAllocator

protected

Memory for temporary allocations, like AST objects or literal strings with converted escape sequences. Provided by the compiler with method Parse.

Definition at line 67 of file parser_impl.hpp.

◆ expression

String expression

protected

The given expression to parse.

Definition at line 95 of file parser_impl.hpp.

◆ numberFormat

NumberFormat* numberFormat

protected

Used for scanning literals. Provided to this class with each parse request.

Definition at line 91 of file parser_impl.hpp.

◆ operatorChars

std::bitset<256> operatorChars

protected

Lists single characters that got found in operator strings which have been registered with Compiler::AddUnaryOperator and Compiler::AddBinaryOperator.

Used by the internal token scanner (lexer) and by default will become something like "=+-*%/?:~!|&^!<>/%". when found in the expression string.

Definition at line 112 of file parser_impl.hpp.

◆ scanner

Substring scanner

protected

The rest of expression.

Definition at line 98 of file parser_impl.hpp.

◆ syntaxTokens

std::bitset<256> syntaxTokens

protected

Lists single characters that get directly converted into tokens of corresponding type when found in the expression string. Tokens are "()[],".

Definition at line 102 of file parser_impl.hpp.

◆ token

Tokens token

protected

The actual token type.

Definition at line 70 of file parser_impl.hpp.

◆ tokFloat

double tokFloat

protected

Float value of token (if applicable).

Definition at line 79 of file parser_impl.hpp.

◆ tokInteger

integer tokInteger

protected

Integer value of token (if applicable).

Definition at line 76 of file parser_impl.hpp.

◆ tokLiteralHint

ASTLiteral::NFHint tokLiteralHint

protected

The actual token type.

Definition at line 73 of file parser_impl.hpp.

◆ tokPosition

integer tokPosition

protected

The position of the token in expression.

Definition at line 85 of file parser_impl.hpp.

◆ tokString

String tokString

protected

String value of token (if applicable).

Definition at line 82 of file parser_impl.hpp.

◆ unaryOperators

HashSet< MonoAllocator, String, alib::hash_string_ignore_case <character>, alib::equal_to_string_ignore_case<character> > unaryOperators

protected

Hash set of unary operators. The key of the table is the operator string, which usually consists of one character, like '-' or '!'.

This table is filled in the constructor of the class with the values stored in Compiler::UnaryOperators and used for testing of existence.

Definition at line 122 of file parser_impl.hpp.

Constructor(s) / Destructor Details:

◆ ParserImpl()

ParserImpl	(	Compiler &	compiler,
		MonoAllocator &	allocator )

Constructor.

Parameters

compiler	The compiler that this parser works for.
allocator	A monotonic allocator for permanent allocations.

Definition at line 23 of file parser_impl.cpp.

◆ ~ParserImpl()

virtual ~ParserImpl ( )

inlineoverridevirtual

Virtual destructor.

Definition at line 150 of file parser_impl.hpp.

Method Details:

◆ getBinaryOp()

String getBinaryOp ( )

protected

Tests if the actual token represents a known binary operator.

Returns: Returns the binary operator symbol, respectively a nulled string on failure.

Definition at line 606 of file parser_impl.cpp.

Here is the call graph for this function:

◆ getUnaryOp()

String getUnaryOp ( )

protected

Tests if the actual token represents a known unary operator.

Returns: Returns the binary operator symbol, respectively a nulled string on failure.

Definition at line 568 of file parser_impl.cpp.

Here is the call graph for this function:

◆ NextToken()

void NextToken ( )

protected

This is the "scanner" or "lexer" method.

Definition at line 98 of file parser_impl.cpp.

Here is the call graph for this function:

◆ Parse()

detail::AST * Parse	(	const String &	exprString,
		NumberFormat *	nf )

overridevirtual

Parses the given expression string.

Note: The return value is hidden by using void*. This is to allow avoid flooding of boost header includes files to the code entities using module ALib Expressions.

Parameters

exprString	The string to parse.
nf	Used to scan number literals.

Returns: The abstract syntax tree representing the expression.

Implements Parser.

Definition at line 308 of file parser_impl.cpp.

Here is the call graph for this function:

◆ parseBinary()

AST * parseBinary ( )

protected

Internal method that optionally parses a binary operator and levels (recursively) trees of such according to operator precedence and brackets given.

Returns: T.

Definition at line 373 of file parser_impl.cpp.

Here is the call graph for this function:

◆ parseConditional()

AST * parseConditional ( )

protected

Internal method that optionally parses a conditional operator (Q ? T : F )

Returns: T.

Definition at line 339 of file parser_impl.cpp.

Here is the call graph for this function:

◆ parseSimple()

AST * parseSimple ( )

protected

Parses unary ops, literals, identifiers, functions and expressions surrounded by brackets.

Returns: The abstract syntax tree node parsed.

Definition at line 429 of file parser_impl.cpp.

Here is the call graph for this function:

◆ parseSubscript()

AST * parseSubscript ( AST * function )

protected

Invoked after an identifier or function was parsed. Tests for subscript operator, otherwise returns the given ast as is.

Parameters

function The identifier or function parsed.

Returns: Either the given node or a node of type ASTBinaryOp with lhs set to function, rhs to the parsed subscript arg and and operator set to '[]'.

Definition at line 538 of file parser_impl.cpp.

Here is the call graph for this function:

◆ pop()

AST * pop ( )

inlineprotected

Simple shortcut popping and returning last ast from the current list.

Returns: Popped AST object.

Definition at line 214 of file parser_impl.hpp.

◆ push()

AST * push ( AST * ast )

inlineprotected

Simple shortcut pushing an ast to current list and returning it.

Parameters

ast	The AST node to push.

Returns: Popped AST object.

Definition at line 224 of file parser_impl.hpp.

◆ replace()

AST * replace ( AST * ast )

inlineprotected

Simple shortcut replacing the topmost ast.

Parameters

ast	The new AST node to replace the existing one with.

Returns: The given object.

Definition at line 240 of file parser_impl.hpp.

◆ top()

AST * top ( )

inlineprotected

Simple shortcut to the topmost AST.

Returns: The topmost AST object.

Definition at line 232 of file parser_impl.hpp.

The documentation for this class was generated from the following files:

Description:

Public Method Index:

Protected Type Index:

Protected Field Index:

Protected Method Index:

Additional Inherited Members

Enumeration Details:

◆ Tokens

Field Details:

◆ ASTs

◆ binaryOperators

◆ compiler

◆ compileTimeAllocator

◆ expression

◆ numberFormat

◆ operatorChars

◆ scanner

◆ syntaxTokens

◆ token

◆ tokFloat

◆ tokInteger

◆ tokLiteralHint

◆ tokPosition

◆ tokString

◆ unaryOperators

Constructor(s) / Destructor Details:

◆ ParserImpl()

◆ ~ParserImpl()

Method Details:

◆ getBinaryOp()

◆ getUnaryOp()

◆ NextToken()

◆ Parse()

◆ parseBinary()

◆ parseConditional()

◆ parseSimple()

◆ parseSubscript()

◆ pop()

◆ push()

◆ replace()

◆ top()