lexer

package
v0.0.0-...-2934bfa Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 2, 2015 License: MIT Imports: 6 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type LexFn

type LexFn func(*Lexer) LexFn

func LexBegin

func LexBegin(lexer *Lexer) LexFn

This lexer function starts everything off. It determines if we are beginning with a key/value assignment or a section.

func LexEqualSign

func LexEqualSign(lexer *Lexer) LexFn

This lexer function emits a TOKEN_EQUAL_SIGN then returns the lexer for value.

func LexKey

func LexKey(lexer *Lexer) LexFn

This lexer function emits a TOKEN_KEY with the name of an key that will be assigned a value.

func LexLeftBracket

func LexLeftBracket(lexer *Lexer) LexFn

This lexer function emits a TOKEN_LEFT_BRACKET then returns the lexer for a section header.

func LexRightBracket

func LexRightBracket(lexer *Lexer) LexFn

This lexer function emits a TOKEN_RIGHT_BRACKET then returns the lexer for a begin.

func LexSection

func LexSection(lexer *Lexer) LexFn

This lexer function emits a TOKEN_SECTION with the name of an INI file section header.

func LexValue

func LexValue(lexer *Lexer) LexFn

This lexer function emits a TOKEN_VALUE with the value to be assigned to a key.

type Lexer

type Lexer struct {
	Name   string
	Input  string
	Tokens chan lexertoken.Token
	State  LexFn

	Start int
	Pos   int
	Width int
}

Lexer object contains the state of our parser and provides a stream for accepting tokens.

Based on work by Rob Pike http://cuddle.googlecode.com/hg/talk/lex.html#landing-slide

func (*Lexer) Backup

func (this *Lexer) Backup()

Backup to the beginning of the last read token.

func (*Lexer) CurrentInput

func (this *Lexer) CurrentInput() string

Returns a slice of the current input from the current lexer start position to the current position.

func (*Lexer) Dec

func (this *Lexer) Dec()

Decrement the position

func (*Lexer) Emit

func (this *Lexer) Emit(tokenType lexertoken.TokenType)

Puts a token onto the token channel. The value of this token is read from the input based on the current lexer position.

func (*Lexer) Errorf

func (this *Lexer) Errorf(format string, args ...interface{}) LexFn

Returns a token with error information.

func (*Lexer) Ignore

func (this *Lexer) Ignore()

Ignores the current token by setting the lexer's start position to the current reading position.

func (*Lexer) Inc

func (this *Lexer) Inc()

Increment the position

func (*Lexer) InputToEnd

func (this *Lexer) InputToEnd() string

Return a slice of the input from the current lexer position to the end of the input string.

func (*Lexer) IsEOF

func (this *Lexer) IsEOF() bool

Returns the true/false if the lexer is at the end of the input stream.

func (*Lexer) IsWhitespace

func (this *Lexer) IsWhitespace() bool

Returns true/false if then next character is whitespace

func (*Lexer) Next

func (this *Lexer) Next() rune

Reads the next rune (character) from the input stream and advances the lexer position.

func (*Lexer) NextToken

func (this *Lexer) NextToken() lexertoken.Token

Return the next token from the channel

func (*Lexer) Peek

func (this *Lexer) Peek() rune

Returns the next rune in the stream, then puts the lexer position back. Basically reads the next rune without consuming it.

func (*Lexer) Run

func (this *Lexer) Run()

Starts the lexical analysis and feeding tokens into the token channel.

func (*Lexer) Shutdown

func (this *Lexer) Shutdown()

Shuts down the token stream

func (*Lexer) SkipWhitespace

func (this *Lexer) SkipWhitespace()

Skips whitespace until we get something meaningful.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL