lexmachine: github.com/timtadh/lexmachine/machines Index | Files

package machines

import "github.com/timtadh/lexmachine/machines"

Package machines implements the lexing algorithms.

Index

Package Files

dfa_machine.go machine.go

type DFAAccepting Uses

type DFAAccepting map[int]int

DFAAccepting represents maps from accepting DFA states to match identifiers. These both identify which states are accepting states and which matches they belong to from the AST.

type DFATrans Uses

type DFATrans [][256]int

DFATrans represents a Deterministic Finite Automatons state transition table

type EmptyMatchError Uses

type EmptyMatchError struct {
    TC      int
    Line    int
    Column  int
    MatchID int
}

EmptyMatchError is returned when a pattern would have matched the empty string

func (*EmptyMatchError) Error Uses

func (e *EmptyMatchError) Error() string

type Match Uses

type Match struct {
    PC          int
    TC          int
    StartLine   int
    StartColumn int
    EndLine     int
    EndColumn   int
    Bytes       []byte // the actual bytes matched during scanning.
}

A Match represents the positional and textual information from a match.

func (*Match) Equals Uses

func (m *Match) Equals(other *Match) bool

Equals checks two matches for equality

func (Match) String Uses

func (m Match) String() string

String formats the match for humans

type Scanner Uses

type Scanner func(int) (int, *Match, error, Scanner)

Scanner is a functional iterator returned by the LexerEngine. See http://hackthology.com/functional-iteration-in-go.html

func DFALexerEngine Uses

func DFALexerEngine(startState, errorState int, trans DFATrans, accepting DFAAccepting, text []byte) Scanner

DFALexerEngine does the actual tokenization of the byte slice text using the DFA state machine. If the lexing process fails the Scanner will return an UnconsumedInput error.

func LexerEngine Uses

func LexerEngine(program inst.Slice, text []byte) Scanner

LexerEngine does the actual tokenization of the byte slice text using the NFA bytecode in program. If the lexing process fails the Scanner will return an UnconsumedInput error.

type UnconsumedInput Uses

type UnconsumedInput struct {
    StartTC     int
    FailTC      int
    StartLine   int
    StartColumn int
    FailLine    int
    FailColumn  int
    Text        []byte
}

UnconsumedInput error type

func (*UnconsumedInput) Error Uses

func (u *UnconsumedInput) Error() string

Error implements the error interface

Package machines imports 4 packages (graph) and is imported by 4 packages. Updated 2018-03-05. Refresh now. Tools for package owners.