Documentation
¶
Overview ¶
Package token implements tokenization of BIT code.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrSyntax = errors.New("bit/token: syntax error")
ErrSyntax is returned by Parser methods when a token is truncated or invalid.
Functions ¶
This section is empty.
Types ¶
type ErrInvalidChar ¶
type ErrInvalidChar struct {
Char rune
}
ErrInvalidChar is returned by Parser methods when a character that is not an uppercase ASCII letter or a space is written.
func (ErrInvalidChar) Error ¶
func (err ErrInvalidChar) Error() string
Error implements the error interface.
type Parser ¶
type Parser struct {
// Tokens are appended to the slice by writing to the parser. The
// Parser does not change or access this slice in any other way.
Tokens []Token
// contains filtered or unexported fields
}
Parser converts a stream of bytes or runes into a []Token.
It is not safe for concurrent use across multiple goroutines; however, Parser does not modify any global or shared state, so multiple independent instances of Parser are safe to use concurrently.
Methods that return error can return ErrSyntax or ErrInvalidChar. If ErrSyntax is returned by a Write* method, the current token is discarded to assist with tokenization error recovery.
func (*Parser) Done ¶
Done returns ErrSyntax if a partial token is buffered. Otherwise, it returns nil.
func (*Parser) WriteByte ¶
WriteByte adds a byte to the Parser input. If the byte is not an uppercase ASCII letter or a space character as defined by Unicode, this returns ErrInvalidChar. If an invalid token would be created by adding this byte, ErrSyntax is returned and the current token is discarded.
Source Files