token

package
v0.0.9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 23, 2021 License: Apache-2.0 Imports: 4 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var TokMap = TokenMap{
	// contains filtered or unexported fields
}

Functions

This section is empty.

Types

type Context

type Context interface{}

Context allows user-defined data to be associated with the lexer/scanner to be associated with each token that lexer produces.

type Pos

type Pos struct {
	Offset  int
	Line    int
	Column  int
	Context Context
}

func (Pos) String

func (p Pos) String() string

type Sourcer

type Sourcer interface {
	Source() string
}

Sourcer is a Context interface which presents a Source() method identifying e.g the filename for the current code.

type Token

type Token struct {
	Type
	Lit []byte
	Pos
}

func (*Token) CharLiteralValue

func (t *Token) CharLiteralValue() string

CharLiteralValue returns the string value of the char literal.

func (*Token) Equals

func (t *Token) Equals(rhs interface{}) bool

Equals returns returns true if the token Type and Lit are matches.

func (*Token) Float32Value

func (t *Token) Float32Value() (float32, error)

Float32Value returns the float32 value of the token or an error if the token literal does not denote a valid float32.

func (*Token) Float64Value

func (t *Token) Float64Value() (float64, error)

Float64Value returns the float64 value of the token or an error if the token literal does not denote a valid float64.

func (*Token) IDValue

func (t *Token) IDValue() string

IDValue returns the string representation of an identifier token.

func (*Token) Int32Value

func (t *Token) Int32Value() (int32, error)

Int32Value returns the int32 value of the token or an error if the token literal does not denote a valid float64.

func (*Token) Int64Value

func (t *Token) Int64Value() (int64, error)

Int64Value returns the int64 value of the token or an error if the token literal does not denote a valid float64.

func (*Token) StringValue

func (t *Token) StringValue() string

StringValue returns the string value of the token literal.

func (*Token) UTF8Rune

func (t *Token) UTF8Rune() (rune, error)

UTF8Rune decodes the UTF8 rune in the token literal. It returns utf8.RuneError if the token literal contains an invalid rune.

type TokenMap

type TokenMap struct {
	// contains filtered or unexported fields
}

func (TokenMap) Id

func (m TokenMap) Id(tok Type) string

func (TokenMap) StringType

func (m TokenMap) StringType(typ Type) string

func (TokenMap) TokenString

func (m TokenMap) TokenString(tok *Token) string

func (TokenMap) Type

func (m TokenMap) Type(tok string) Type

type Type

type Type int
const (
	INVALID Type = iota
	EOF
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL