Documentation
¶
Overview ¶
Package scan implements a scanner for Goal source text.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func GetLineCol ¶
GetLineCol returns the line text, line number and column number of a given position in a source string. This can be used to retrieve human-readable position information given a token Pos field.
func IsAlphaNum ¶
IsAlphaNum returns true if the rune is a letter or a digit (any non-first character of an identifier).
Types ¶
type IdentType ¶
type IdentType int
identType represents the different kinds of special roles for alphanumeric identifiers that act as keywords.
type Scanner ¶
type Scanner struct {
Comments bool // output Comment tokens too (default: false)
// contains filtered or unexported fields
}
Scanner represents the state of the scanner.
func New ¶
New returns a scanner for the given source string, with the given monadic and dyadic keyword mappings. If shebang is true, the scanner ignores the starting line if it starts with #!.
func (*Scanner) AfterNewline ¶
AfterNewline returns true if last read non-comment token was a newline.
func (*Scanner) AfterOpen ¶
AfterOpen returns true if last read non-comment token was an opening brace, bracket or paren.
func (*Scanner) AtExprStart ¶
AtExprStart returns true if the scanner expects next a new (sub)expression. This means last non-comment token was an opening brace, bracket or paren, a newline, semicolon or special token.
type Token ¶
type Token struct {
Type Type // token type
Pos int // offset of token's first character
End int // offset of first character after token
// Content text. It may differ from the exact token text in the source:
// delimiter token types have an empty Text field, and Ident, String
// and Regexp types can go through some source processing (like
// handling some escapes or changing/removing delimiters) before the
// content is stored in the Text field.
Text string
}
Token represents a token information returned from the scanner.
type Type ¶
type Type int
tokenType represents the different kinds of tokens.
const ( EOF Type = iota Error // syntax error: Text contains message Adverb // ' / \ ´ ` (alone or following tightly expression, potentially followed by :) Comment // comments /.* or multi-line /.*\ (without final newline) Dyad // : + * - or dyadic keyword DyadColon // +: *: -: (only for builtin dyadic symbols) Var // alphanumeric variable var (also from $var ${var}) VarField // var..field or var.. or .. (but not ..field) LeftBrace // { LeftBraceArgs // {[ LeftBracket // [ (sequences and lambda arguments) LeftBracketIdx // [ (indexing) LeftParen // ( Monad // monadic keyword Newline // \n (except duplicates and after opening delimiter) Number // literal number (integer or float) QqEnd // " or / (or other delimiters) QqStart // " or qq/ (or other delimiters) Regexp // escaped PATTERN in rx/PATTERN/ (or other delimiters) RightBrace // } RightBracket // ] RightParen // ) Semicolon // ; Special // non-adverbial ' \ (return early on error & rt.log) String // `string` or properly escaped "string" (from rq// too) )
These constants describe the possible kinds of tokens.
Source Files