lexer

package

v0.0.0-...-2e2f811 Latest Latest Go to latest Published: May 5, 2020 License: MIT Imports: 5 Imported by: 1

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/CuberL/spar

Links

Open Source Insights

Documentation ¶

Overview ¶

Copy from https://github.com/benbjohnson/sql-parser

Index ¶

Variables
type LexerImpl
- func NewLexerImpl(r io.Reader, t Tokenizer) *LexerImpl
- func (li *LexerImpl) Lex(lastToken int) (*LexerResult, error)
type LexerResult
type Scanner
- func NewScanner(r io.Reader) *Scanner
- func (s *Scanner) Scan() (tok Token, lit string)
type Token
type Tokenizer

Constants ¶

This section is empty.

Variables ¶

View Source

var (
	UnexpectedTokenErr = fmt.Errorf("unexpected token")
)

Functions ¶

This section is empty.

Types ¶

type LexerImpl ¶

type LexerImpl struct {
	Result interface{}
	// contains filtered or unexported fields
}

func NewLexerImpl ¶

func NewLexerImpl(r io.Reader, t Tokenizer) *LexerImpl

func (*LexerImpl) Lex ¶

func (li *LexerImpl) Lex(lastToken int) (*LexerResult, error)

type LexerResult ¶

type LexerResult struct {
	Token   int
	Literal string
}

type Scanner ¶

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner represents a lexical scanner.

func NewScanner ¶

func NewScanner(r io.Reader) *Scanner

NewScanner returns a new instance of Scanner.

func (*Scanner) Scan ¶

func (s *Scanner) Scan() (tok Token, lit string)

Scan returns the next token and literal value.

type Token ¶

type Token int

Token represents a lexical token.

const (
	// Special tokens
	ILLEGAL Token = iota
	EOF
	WS

	// Literals
	IDENT   // main
	NUMBER  // number literal
	COMMENT // comment

	// Misc characters
	ASTERISK          // *
	COMMA             // ,
	LEFT_PARENTHESIS  // (
	RIGHT_PARENTHESIS // )
	SEMICOLON         // ;
	EQUAL             // =
	ANGLE_LEFT        // <
	ANGLE_RIGHT       // >
)

type Tokenizer ¶

type Tokenizer interface {
	FromStrLit(lit string, lastToken int) int
}

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL