etoken

package
v0.0.0-...-f54e8e0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 7, 2024 License: MPL-2.0, BSD-3-Clause Imports: 4 Imported by: 0

Documentation

Overview

Package token defines constants representing the lexical tokens of the Go programming language and basic operations on tokens (printing, predicates).

Index

Constants

This section is empty.

Variables

View Source
var GENERICS = GENERICS_NONE

can be changed at runtime. useful to enable them by in gomacro, without affecting packages that depend on gomacro, as Gophernotes

Functions

func IsKeyword

func IsKeyword(tok Token) bool

IsKeyword returns true for tokens corresponding to keywords; it returns false otherwise.

func IsLiteral

func IsLiteral(tok Token) bool

IsLiteral returns true for tokens corresponding to identifiers and basic type literals; it returns false otherwise.

func IsMacroKeyword

func IsMacroKeyword(tok Token) bool

IsMacroKeyword returns true for tokens corresponding to macro-related keywords; it returns false otherwise.

func IsOperator

func IsOperator(tok Token) bool

IsOperator returns true for tokens corresponding to operators and delimiters; it returns false otherwise.

func String

func String(tok Token) string

Types

type File

type File struct {
	*token.File
	// contains filtered or unexported fields
}

A File is a handle for a file belonging to a FileSet. A File has a name, size, line offset table and optionally source code.

func (*File) Position

func (f *File) Position(p token.Pos) (pos token.Position)

Position returns the Position value for the given file position p. Calling f.Position(p) is equivalent to calling f.PositionFor(p, true).

func (*File) PositionFor

func (f *File) PositionFor(p token.Pos, adjusted bool) (pos token.Position)

PositionFor returns the Position value for the given file position p. If adjusted is set, the position may be adjusted by position-altering //line comments; otherwise those comments are ignored. p must be a Pos value in f or NoPos.

func (*File) SetSource

func (f *File) SetSource(source []string)

SetSource sets the source code for the given file.

func (*File) SetSourceForContent

func (f *File) SetSourceForContent(content []byte)

SetSourceForContent computes and sets the source code for the given file.

func (*File) Source

func (f *File) Source(p token.Pos) (line string, pos token.Position)

Source returns the source code for the given file position p, if available.

type FileSet

type FileSet struct {
	token.FileSet
	// contains filtered or unexported fields
}

A FileSet represents a set of source files. This is a wrapper for go/token.FileSet that adds a starting line offset to each file in the set

func NewFileSet

func NewFileSet() *FileSet

NewFileSet creates a new file set.

func (*FileSet) AddFile

func (s *FileSet) AddFile(filename string, base, size, line int) *File

AddFile adds a new file with a given filename, base offset, and file size

func (*FileSet) File

func (s *FileSet) File(p token.Pos) (f *File)

File returns the file that contains the position p. If no such file is found (for instance for p == NoPos), the result is nil.

func (*FileSet) Position

func (s *FileSet) Position(p token.Pos) (pos token.Position)

Position converts a Pos p in the fileset into a Position value. Calling s.Position(p) is equivalent to calling s.PositionFor(p, true).

func (*FileSet) PositionFor

func (s *FileSet) PositionFor(p token.Pos, adjusted bool) (pos token.Position)

PositionFor converts a Pos p in the fileset into a Position value. If adjusted is set, the position may be adjusted by position-altering //line comments; otherwise those comments are ignored. p must be a Pos value in s or NoPos.

func (*FileSet) Source

func (s *FileSet) Source(p token.Pos) (line string, pos token.Position)

Source converts a Pos p in the fileset into a line of source code (if available) and a Position value.

type Generics

type Generics int
const (
	GENERICS_NONE Generics = iota
	// enables C++ style generics
	GENERICS_V1_CXX
	// enables generics "contracts are interfaces"
	GENERICS_V2_CTI
)

func (Generics) V1_CXX

func (g Generics) V1_CXX() bool

func (Generics) V2_CTI

func (g Generics) V2_CTI() bool

type Token

type Token = token.Token
const (
	QUOTE Token = (token.VAR+127)&^127 + iota
	QUASIQUOTE
	UNQUOTE
	UNQUOTE_SPLICE
	MACRO
	FUNCTION
	LAMBDA
	TYPECASE
	TEMPLATE // template
	HASH     // #

	// the following are never used by go/scanner
	// they are returned by ast2/Ast.Op() for corresponding AST nodes
	E_ALIASTYPE
	E_CALL
	E_DECLSTMT
	E_FIELDDECL
	E_FUNCDECL
	E_FUNCTYPE
	E_KEYVALUE
	E_LABELEDSTMT
	E_SENDSTMT
	E_TYPEASSERT
	E_TYPESWITCH
)

func Lookup

func Lookup(lit string) Token

Lookup maps a identifier to its keyword token.

func LookupSpecial

func LookupSpecial(lit string) Token

LookupSpecial maps a identifier starting with '~' to its keyword token.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL