lexer

package
v0.0.0-...-72c54ed Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 31, 2015 License: MIT Imports: 4 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type LexFn

type LexFn func(*Lexer) LexFn

LexFn is an interface that lexing functions have to implement

func LexAnd

func LexAnd(lexer *Lexer) LexFn

LexAnd emits TokenAnd

func LexBegin

func LexBegin(lexer *Lexer) LexFn

LexBegin skips all whitespaces and returns a function that can lex the remaining input

func LexComma

func LexComma(lexer *Lexer) LexFn

LexComma emits TokenComma

func LexDoubleQuotedString

func LexDoubleQuotedString(lexer *Lexer) LexFn

LexDoubleQuotedString emits TokenDoubleQuotedStringLiteral

func LexLeftParenthesis

func LexLeftParenthesis(lexer *Lexer) LexFn

LexLeftParenthesis emits TokenLeftParenthesis

func LexNot

func LexNot(lexer *Lexer) LexFn

LexNot emits TokenNot

func LexOr

func LexOr(lexer *Lexer) LexFn

LexOr emits TokenOr

func LexRegexEqual

func LexRegexEqual(lexer *Lexer) LexFn

LexRegexEqual emits TokenRegexEqual

func LexRightParenthesis

func LexRightParenthesis(lexer *Lexer) LexFn

LexRightParenthesis emits TokenRightParenthesis

func LexSha1

func LexSha1(lexer *Lexer) LexFn

LexSha1 emits TokenSha1

func LexSingleQuotedString

func LexSingleQuotedString(lexer *Lexer) LexFn

LexSingleQuotedString emits TokenSingleQuotedStringLiteral

func LexStringEqual

func LexStringEqual(lexer *Lexer) LexFn

LexStringEqual emits TokenStringEqual

type Lexer

type Lexer struct {
	Input                string
	Tokens               []Token
	State                LexFn
	Errors               []error
	TokenStart           int
	Position             int
	OpenParenthesisCount int
}

Lexer is a structure that contains current state of the lexer

func New

func New(input string) *Lexer

New returns a new lexer for the given input

func (*Lexer) EatWhitespaces

func (lexer *Lexer) EatWhitespaces() rune

EatWhitespaces skips all whitespaces

func (*Lexer) Emit

func (lexer *Lexer) Emit(tokenType TokenType)

Emit appends given token to the lexer tokens slice

func (*Lexer) Errorf

func (lexer *Lexer) Errorf(err string) LexFn

Errorf appends error with the given error message to the list of lexer errors

func (*Lexer) HasTokenTypeAt

func (lexer *Lexer) HasTokenTypeAt(pos int, tokenType TokenType) bool

HasTokenTypeAt returns true if the lexer recognized given token type at the given position

func (*Lexer) IsEOF

func (lexer *Lexer) IsEOF() bool

IsEOF returns true if the lexer has reached the end of the input

func (*Lexer) Lex

func (lexer *Lexer) Lex() []error

Lex performs lexing

func (*Lexer) Read

func (lexer *Lexer) Read() rune

Read returns current rune

func (*Lexer) RemainingInput

func (lexer *Lexer) RemainingInput() string

RemainingInput returns a string of an unread remainder input string

type Token

type Token struct {
	Type  TokenType
	Value string
}

Token is a structure that contains type of recognized token and it's value if applicable

type TokenType

type TokenType int

TokenType represents a type of recognized token

const (
	// TokenEOF is an End of File token
	TokenEOF TokenType = iota

	// TokenLeftParenthesis is a left parenthesis token
	TokenLeftParenthesis

	// TokenRightParenthesis is a right parenthesis token
	TokenRightParenthesis

	// TokenComma is a comma token
	TokenComma

	// TokenRegexEqual is a regex equal operator token
	TokenRegexEqual

	// TokenStringEqual is a string equal operator token
	TokenStringEqual

	// TokenNot is a negation logical operator token
	TokenNot

	// TokenAnd is an and logical operator token
	TokenAnd

	// TokenOr is an or logical operator token
	TokenOr

	// TokenSingleQuotedStringLiteral is a single quoted string literal token
	TokenSingleQuotedStringLiteral

	// TokenDoubleQuotedStringLiteral is a double quoted string literal token
	TokenDoubleQuotedStringLiteral

	// TokenSha1 is a sha1 function token
	TokenSha1
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL