Documentation
¶
Index ¶
- Variables
- type Buffer
- type Position
- type Stream
- func (l *Stream) Advance() (r rune, err error)
- func (l *Stream) Collect() string
- func (l *Stream) Index() int64
- func (l *Stream) Peek() (rune, error)
- func (l *Stream) PeekN(n int) (s string, err error)
- func (l *Stream) Position() (line, column int)
- func (l *Stream) Retreat() (r rune, err error)
- func (l *Stream) String() string
- type StringRuneReader
Constants ¶
This section is empty.
Variables ¶
var ErrStartOfFile = errors.New("SOF")
ErrStartOfFile is the error used when we've retreated to the start of the file and can't retreat any further.
Functions ¶
This section is empty.
Types ¶
type Buffer ¶
type Buffer struct {
// contains filtered or unexported fields
}
func NewBufferFromString ¶
func (*Buffer) CollectSlice ¶
type Position ¶
Position represents the character position within a text file.
func NewPosition ¶
NewPosition creates a Position to represent the character position within a text file.
func (*Position) Advance ¶
Advance advances the position by a line if the rune is'\n', does nothing if the rune is '\r' and advances by a col character if the rune is anything else.
type Stream ¶
type Stream struct {
// Input holds the Reader being scanned.
Input io.RuneReader
// Buffer is the space currently being searched for tokens to avoid seeking the input stream.
// When a token match is found, the buffer is emptied.
Buffer *Buffer
// Start represents the start position of the lexer against the Input, e.g.
// if we've lexed 124 runes already, that's where we're starting from.
Start int64
// Current represents the current position of the lexer. We might have lexed
// 124 runes and have read 3 more runes without yet emitting a token, so current would
// be 124+3=127
Current int64
// CurrentRune is the current rune at the cursor.
CurrentRune rune
// contains filtered or unexported fields
}
Stream defines a lexical scanner over a stream.
func New ¶
func New(input io.RuneReader) *Stream
New creates a new parser input from a buffered reader.
func NewFromString ¶
NewFromString creates a new parser input from an input string.
func NewWithBufferSize ¶ added in v0.0.48
func NewWithBufferSize(input io.RuneReader, size int) *Stream
NewWithBufferSize allows the buffer to be sized appropriately for the input. There's no need to allocate more than the length of the input as the buffer.
func (*Stream) Collect ¶
Collect returns the value of the consumed buffer and updates the position of the stream to the current position.
func (*Stream) PeekN ¶
PeekN reads a number of runes from the Input, then sets the current position back.
type StringRuneReader ¶
type StringRuneReader struct {
// contains filtered or unexported fields
}
StringRuneReader allows a string to be read rune-by-rune. It allocates slightly less variables than NewBufferString or NewReader.
Source Files