Package machines implements the lexing algorithms.
DFAAccepting represents maps from accepting DFA states to match identifiers. These both identify which states are accepting states and which matches they belong to from the AST.
DFATrans represents a Deterministic Finite Automatons state transition table
EmptyMatchError is returned when a pattern would have matched the empty string
A Match represents the positional and textual information from a match.
Equals checks two matches for equality
String formats the match for humans
Scanner is a functional iterator returned by the LexerEngine. See http://hackthology.com/functional-iteration-in-go.html
func DFALexerEngine(startState, errorState int, trans DFATrans, accepting DFAAccepting, text byte) Scanner
DFALexerEngine does the actual tokenization of the byte slice text using the DFA state machine. If the lexing process fails the Scanner will return an UnconsumedInput error.
LexerEngine does the actual tokenization of the byte slice text using the NFA bytecode in program. If the lexing process fails the Scanner will return an UnconsumedInput error.
UnconsumedInput error type
Error implements the error interface