lexer

package

v0.0.0-...-2f62ff3 Latest Latest Go to latest Published: Jun 2, 2022 License: BSD-3-Clause Imports: 4 Imported by: 1

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/Kingsford-Group/biblint

Documentation ¶

Overview ¶

Constants ¶

View Source

const (
	ILLEGAL TokenType = "ILLEGAL"
	EOF               = "EOF"
	IDENT             = "IDENT"
	STRING            = "STRING"
	AT                = "@"
	COMMA             = ","
	LBRACE            = "{"
	RBRACE            = "}"
	HASH              = "#"
	EQUALS            = "="
)

The types of tokens that the lexer can return.

Variables ¶

View Source

var EOFToken = &Token{Type: EOF}

Functions ¶

This section is empty.

Types ¶

type Lexer ¶

type Lexer struct {
	// contains filtered or unexported fields
}

func New ¶

func New(f io.Reader) *Lexer

New returns a new lexer than will return a stream of tokens in the bibtex language.

func (*Lexer) Err ¶

func (l *Lexer) Err() error

Err returns the last recorded error.

func (*Lexer) NextToken ¶

func (l *Lexer) NextToken(braceStrings bool) (*Token, error)

NextToken produces the next token. Assumes curRune will give the next unprocessed character. We must maintain that invariant after New and nextToken. If braceStrings is true, treats {}-delimitated regions as a string (requiring balanced {} strings).

func (*Lexer) Position ¶

func (l *Lexer) Position() (int, int)

func (*Lexer) SkipToNewLine ¶

func (l *Lexer) SkipToNewLine() error

SkipToNewLine skips until the current run is '\n'.

type Token ¶

type Token struct {
	Type    TokenType
	Literal string
	// contains filtered or unexported fields
}

Token is returned by NextToken(). Literal is the string corresponding to the token.

func (*Token) Position ¶

func (t *Token) Position() (int, int)

type TokenType ¶

type TokenType string

Source Files ¶

View all Source files

lexer.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL