goparsec: github.com/prataprc/goparsec Index | Examples | Files | Directories

package parsec

import "github.com/prataprc/goparsec"

Package parsec provides a library of parser-combinators. The basic idea behind parsec module is that, it allows programmers to compose basic set of terminal parsers, a.k.a tokenizers and compose them together as a tree of parsers, using combinators like: And, OrdChoice, Kleene, Many, Maybe.

To begin with there are four basic Types that needs to be kept in mind while creating and composing parsers,

Types

Scanner, an interface type that encapsulates the input text. A built in scanner called SimpleScanner is supplied along with this package. Developers can also implement their own scanner types. Following example create a new instance of SimpleScanner, using an input text:

var exprText = []byte(`4 + 123 + 23 + 67 +89 + 87 *78`)
s := parsec.NewScanner(exprText)

Nodify, callback function is supplied while combining parser functions. If the underlying parsing logic matches with i/p text, then callback will be dispatched with list of matching ParsecNode. Value returned by callback function will further be used as ParsecNode item in higher-level list of ParsecNodes.

Parser, simple parsers are functions that matches i/p text for specific patterns. Simple parsers can be combined using one of the supplied combinators to construct a higher level parser. A parser function takes a Scanner object and applies the underlying parsing logic, if underlying logic succeeds Nodify callback is dispatched and a ParsecNode and a new Scanner object (with its cursor moved forward) is returned. If parser fails to match, it shall return the input scanner object as it is, along with nil ParsecNode.

ParsecNode, an interface type encapsulates one or more tokens from i/p text, as terminal node or non-terminal node.

Combinators

If input text is going to be a single token like `10` or `true` or `"some string"`, then all we need is a single Parser function that can tokenize the i/p text into a terminal node. But our applications are seldom that simple. Almost all the time we need to parse the i/p text for more than one tokens and most of the time we need to compose them into a tree of terminal and non-terminal nodes.

This is where combinators are useful. Package provides a set of combinators to help combine terminal parsers into higher level parsers. They are,

* And, to combine a sequence of terminals and non-terminal parsers.
* OrdChoice, to choose between specified list of parsers.
* Kleene, to repeat the parser zero or more times.
* Many, to repeat the parser one or more times.
* ManyUntil, to repeat the parser until a specified end matcher.
* Maybe, to apply the parser once or none.

All the above mentioned combinators accept one or more parser function as arguments, either by value or by reference. The reason for allowing parser argument by reference is to be able to define recursive parsing logic, like parsing nested arrays:

var Y Parser
var value Parser // circular rats

var opensqrt = Atom("[", "OPENSQRT")
var closesqrt = Atom("]", "CLOSESQRT")
var values = Kleene(nil, &value, Atom(",", "COMMA"))
var array = And(nil, opensqrt, values, closeSqrt)
func init() {
	value = parsec.OrdChoice(nil, Int(), Bool(), String(), array)
	Y = parsec.OrdChoice(nil, value)
}

Terminal parsers

Parsers for standard set of tokens are supplied along with this package. Most of these parsers return Terminal type as ParseNode.

* Char, match a single character skipping leading whitespace.
* Float, match a float literal skipping leading whitespace.
* Hex, match a hexadecimal literal skipping leading whitespace.
* Int, match a decimal number literal skipping leading whitespace.
* Oct, match a octal number literal skipping leading whitespace.
* String, match a string literal skipping leading whitespace.
* Ident, match a identifier token skipping leading whitespace.
* Atom, match a single atom skipping leading whitespace.
* AtomExact, match a single atom without skipping leading whitespace.
* Token, match a single token skipping leading whitespace.
* TokenExact, match a single token without skipping leading whitespace.
* OrdToken, match a single token with specified list of alternatives.
* End, match end of text.
* NoEnd, match not an end of text.

All of the terminal parsers, except End and NoEnd return Terminal type as ParsecNode. While End and NoEnd return a boolean type as ParsecNode.

AST and Queryable

This is an experimental feature to use CSS like selectors for quering an Abstract Syntax Tree (AST). Types, APIs and methods associated with AST and Queryable are unstable, and are expected to change in future.

While Scanner, Parser, ParsecNode types are re-used in AST and Queryable, combinator functions are re-implemented as AST methods. Similarly type ASTNodify is to be used instead of Nodify type. Otherwise all the parsec techniques mentioned above are equally applicable on AST.

Additionally, following points are worth noting while using AST,

* Combinator methods supplied via AST can be named.
* All combinators from AST object will create and return NonTerminal
  as the Queryable type.
* ASTNodify function can interpret its Queryable argument and return
  a different type implementing Queryable interface.

Index

Examples

Package Files

ast.go doc.go nonterminal.go parsec.go scanner.go selector.go terminal.go tokeniser.go

type AST Uses

type AST struct {
    // contains filtered or unexported fields
}

AST to parse and construct Abstract Syntax Tree whose nodes confirm to `Queryable` interface, facilitating tree processing algorithms.

func NewAST Uses

func NewAST(name string, maxnodes int) *AST

NewAST return a new instance of AST, maxnodes is size of internal buffer pool of nodes, it is directly proportional to number of nodes that you expect in the syntax-tree.

func (*AST) And Uses

func (ast *AST) And(name string, callb ASTNodify, parsers ...interface{}) Parser

And combinator, same as package level And combinator function. `name` identifies the NonTerminal nodes constructed by this combinator.

Code:

// parse a configuration line from ini file.
text := []byte(`loglevel = info`)
ast := NewAST("example", 100)
y := ast.And("configline", nil, Ident(), Atom("=", "EQUAL"), Ident())
root, _ := ast.Parsewith(y, NewScanner(text))
nodes := root.GetChildren()
fmt.Println(nodes[0].GetName(), nodes[0].GetValue())
fmt.Println(nodes[1].GetName(), nodes[1].GetValue())
fmt.Println(nodes[2].GetName(), nodes[2].GetValue())

Output:

IDENT loglevel
EQUAL =
IDENT info

func (*AST) Dotstring Uses

func (ast *AST) Dotstring(name string) string

Dotstring return AST in graphviz dot format. Save this string to a dot file and use graphviz tool generate a nice looking graph.

func (*AST) End Uses

func (ast *AST) End(name string) Parser

End is a parser function to detect end of scanner output.

func (*AST) GetValue Uses

func (ast *AST) GetValue() string

GetValue return the full text, called as value here, that was parsed to contruct this syntax-tree.

func (*AST) Kleene Uses

func (ast *AST) Kleene(nm string, callb ASTNodify, ps ...interface{}) Parser

Kleene combinator, same as package level Kleene combinator function. `nm` identifies the NonTerminal nodes constructed by this combinator.

func (*AST) Many Uses

func (ast *AST) Many(nm string, callb ASTNodify, parsers ...interface{}) Parser

Many combinator, same as package level Many combinator function. `nm` identifies the NonTerminal nodes constructed by this combinator.

Code:

// parse comma separated values
text := []byte(`10,30,50 wont parse this`)
ast := NewAST("example", 100)
y := ast.Many("many", nil, Int(), Atom(",", "COMMA"))
root, _ := ast.Parsewith(y, NewScanner(text))
nodes := root.GetChildren()
fmt.Println(nodes[0].GetName(), nodes[0].GetValue())
fmt.Println(nodes[1].GetName(), nodes[1].GetValue())
fmt.Println(nodes[2].GetName(), nodes[2].GetValue())

Output:

INT 10
INT 30
INT 50

func (*AST) ManyUntil Uses

func (ast *AST) ManyUntil(nm string, callb ASTNodify, ps ...interface{}) Parser

ManyUntil combinator, same as package level Many combinator function. `nm` identifies the NonTerminal nodes constructed by this combinator.

Code:

// make sure to parse the entire text
text := []byte("10,30,50")
ast := NewAST("example", 100)
y := ast.ManyUntil("values", nil, Int(), Atom(",", "COMMA"), ast.End("eof"))
root, _ := ast.Parsewith(y, NewScanner(text))
nodes := root.GetChildren()
fmt.Println(nodes[0].GetName(), nodes[0].GetValue())
fmt.Println(nodes[1].GetName(), nodes[1].GetValue())
fmt.Println(nodes[2].GetName(), nodes[2].GetValue())

Output:

INT 10
INT 30
INT 50

func (*AST) Maybe Uses

func (ast *AST) Maybe(name string, callb ASTNodify, parser interface{}) Parser

Maybe combinator, same as package level Maybe combinator function. `nm` identifies the NonTerminal nodes constructed by this combinator.

Code:

// parse an optional token
ast := NewAST("example", 100)
equal := Atom("=", "EQUAL")
maybeand := ast.Maybe("maybeand", nil, Atom("&", "AND"))
y := ast.And("assignment", nil, Ident(), equal, maybeand, Ident())

text := []byte("a = &b")
root, _ := ast.Parsewith(y, NewScanner(text))
nodes := root.GetChildren()
fmt.Println(nodes[0].GetName(), nodes[0].GetValue())
fmt.Println(nodes[1].GetName(), nodes[1].GetValue())
fmt.Println(nodes[2].GetName(), nodes[2].GetValue())
fmt.Println(nodes[3].GetName(), nodes[3].GetValue())

text = []byte("a = b")
ast = ast.Reset()
root, _ = ast.Parsewith(y, NewScanner(text))
nodes = root.GetChildren()
fmt.Println(nodes[0].GetName(), nodes[0].GetValue())
fmt.Println(nodes[1].GetName(), nodes[1].GetValue())
fmt.Println(nodes[2].GetName())
fmt.Println(nodes[3].GetName(), nodes[3].GetValue())

Output:

IDENT a
EQUAL =
AND &
IDENT b
IDENT a
EQUAL =
missing
IDENT b

func (*AST) OrdChoice Uses

func (ast *AST) OrdChoice(nm string, cb ASTNodify, ps ...interface{}) Parser

OrdChoice combinator, same as package level OrdChoice combinator function. `nm` identifies the NonTerminal nodes constructed by this combinator.

Code:

// parse a boolean value
text := []byte(`true`)
ast := NewAST("example", 100)
y := ast.OrdChoice("bool", nil, Atom("true", "TRUE"), Atom("false", "FALSE"))
root, _ := ast.Parsewith(y, NewScanner(text))
fmt.Println(root.GetName(), root.GetValue())

Output:

TRUE true

func (*AST) Parsewith Uses

func (ast *AST) Parsewith(y Parser, s Scanner) (Queryable, Scanner)

Parsewith execute the root parser, y, with scanner s. AST will remember the root parser, and root node. Return the root-node as Queryable, if success and scanner with remaining input.

func (*AST) Prettyprint Uses

func (ast *AST) Prettyprint()

Prettyprint to standard output the syntax-tree in human readable plain text.

func (*AST) Query Uses

func (ast *AST) Query(selectors string, ch chan Queryable)

Query is an experimental method on AST. Developers can use the selector specification to pick one or more nodes from the AST.

func (*AST) Reset Uses

func (ast *AST) Reset() *AST

Reset the AST, forget the root parser, and root node. Reuse the AST object via Parsewith different set of root-parser and scanner.

func (*AST) SetDebug Uses

func (ast *AST) SetDebug() *AST

SetDebug enables console logging while parsing the input test, this is useful while developing a parser.

type ASTNodify Uses

type ASTNodify func(name string, s Scanner, node Queryable) Queryable

ASTNodify callback function to construct custom Queryable. Even when combinators like And, OrdChoice, Many etc.. match input string, it is possible to fail them via ASTNodify callback function, by returning nil. This is useful in cases like:

* where lookahead matching is required.
* exceptional cases for a regex pattern.

Note that some combinators like Kleene shall not interpret the return value from ASTNodify callback. `node` will always be of NonTerminal type, although callback can process it and return a different type, provided it implements Queryable interface.

Code:

text := []byte("10 * 20")
ast := NewAST("example", 100)
y := ast.And(
    "multiply",
    func(name string, s Scanner, node Queryable) Queryable {
        cs := node.GetChildren()
        x, _ := strconv.Atoi(cs[0].(*Terminal).GetValue())
        y, _ := strconv.Atoi(cs[2].(*Terminal).GetValue())
        return &Terminal{Value: fmt.Sprintf("%v", x*y)}
    },
    Int(), Token(`\*`, "MULT"), Int(),
)
node, _ := ast.Parsewith(y, NewScanner(text))
fmt.Println(node.GetValue())

Output:

200

type MaybeNone Uses

type MaybeNone string

MaybeNone is a placeholder type, similar to Terminal type, used by Maybe combinator if parser does not match the input text.

func (MaybeNone) GetAttribute Uses

func (mn MaybeNone) GetAttribute(attrname string) []string

GetAttribute implement Queryable interface.

func (MaybeNone) GetAttributes Uses

func (mn MaybeNone) GetAttributes() map[string][]string

GetAttributes implement Queryable interface.

func (MaybeNone) GetChildren Uses

func (mn MaybeNone) GetChildren() []Queryable

GetChildren implement Queryable interface.

func (MaybeNone) GetName Uses

func (mn MaybeNone) GetName() string

GetName implement Queryable interface.

func (MaybeNone) GetPosition Uses

func (mn MaybeNone) GetPosition() int

GetPosition implement Queryable interface.

func (MaybeNone) GetValue Uses

func (mn MaybeNone) GetValue() string

GetValue implement Queryable interface.

func (MaybeNone) IsTerminal Uses

func (mn MaybeNone) IsTerminal() bool

IsTerminal implement Queryable interface.

func (MaybeNone) SetAttribute Uses

func (mn MaybeNone) SetAttribute(attrname, value string) Queryable

SetAttribute implement Queryable interface.

type Nodify Uses

type Nodify func([]ParsecNode) ParsecNode

Nodify callback function to construct custom ParsecNode. Even when combinators like And, OrdChoice, Many etc.. can match input string, it is still possible to fail them via nodify callback function, by returning nil. This very useful in cases when,

* lookahead matching is required.
* an exceptional cases for regex pattern.

Note that some combinators like KLEENE shall not interpret the return value from Nodify callback.

Code:

text := []byte("10 * 20")
s := NewScanner(text)
y := And(
    func(nodes []ParsecNode) ParsecNode {
        x, _ := strconv.Atoi(nodes[0].(*Terminal).GetValue())
        y, _ := strconv.Atoi(nodes[2].(*Terminal).GetValue())
        return x * y // this is retuned as node further down.
    },
    Int(), Token(`\*`, "MULT"), Int(),
)
node, _ := y(s)
fmt.Println(node)

Output:

200

type NonTerminal Uses

type NonTerminal struct {
    Name       string      // contains terminal's token type
    Children   []Queryable // list of children to this node.
    Attributes map[string][]string
}

NonTerminal will be used by AST methods to construct intermediate nodes. Note that user supplied ASTNodify callback can construct a different type of intermediate node that confirms to Queryable interface.

func NewNonTerminal Uses

func NewNonTerminal(name string) *NonTerminal

NewNonTerminal create and return a new NonTerminal instance.

func (*NonTerminal) GetAttribute Uses

func (nt *NonTerminal) GetAttribute(attrname string) []string

GetAttribute implement Queryable interface.

func (*NonTerminal) GetAttributes Uses

func (nt *NonTerminal) GetAttributes() map[string][]string

GetAttributes implement Queryable interface.

func (*NonTerminal) GetChildren Uses

func (nt *NonTerminal) GetChildren() []Queryable

GetChildren implement Queryable interface.

func (*NonTerminal) GetName Uses

func (nt *NonTerminal) GetName() string

GetName implement Queryable interface.

func (*NonTerminal) GetPosition Uses

func (nt *NonTerminal) GetPosition() int

GetPosition implement Queryable interface.

func (*NonTerminal) GetValue Uses

func (nt *NonTerminal) GetValue() string

GetValue implement Queryable interface.

func (*NonTerminal) IsTerminal Uses

func (nt *NonTerminal) IsTerminal() bool

IsTerminal implement Queryable interface.

func (*NonTerminal) SetAttribute Uses

func (nt *NonTerminal) SetAttribute(attrname, value string) Queryable

SetAttribute implement Queryable interface.

type ParsecNode Uses

type ParsecNode interface{}

ParsecNode for parsers return input text as parsed nodes.

type Parser Uses

type Parser func(Scanner) (ParsecNode, Scanner)

Parser function parses input text encapsulated by Scanner, higher order parsers are constructed using combinators.

func And Uses

func And(callb Nodify, parsers ...interface{}) Parser

And combinator accepts a list of `Parser`, or reference to a parser, that must match the input string, atleast until the last Parser argument. Return a parser function that can further be used to construct higher-level parsers.

If all parser matches, a list of ParsecNode, where each ParsecNode is constructed by matching parser, will be passed as argument to Nodify callback. Even if one of the input parser function fails, And will fail without consuming the input.

Code:

// parse a configuration line from ini file.
text := []byte(`loglevel = info`)
y := And(nil, Ident(), Atom("=", "EQUAL"), Ident())
root, _ := y(NewScanner(text))
nodes := root.([]ParsecNode)
t := nodes[0].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())
t = nodes[1].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())
t = nodes[2].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())

Output:

IDENT loglevel
EQUAL =
IDENT info

func Atom Uses

func Atom(match string, name string) Parser

Atom is similar to Token, takes a string to match with input byte-by-byte. Internally uses the MatchString() API from Scanner. Skip leading whitespace. For example:

scanner := NewScanner([]byte("cosmos"))
Atom("cos", "ATOM")(scanner) // will match

func AtomExact Uses

func AtomExact(match string, name string) Parser

AtomExact is similar to Atom(), but string will be matched without skipping leading whitespace.

func Char Uses

func Char() Parser

Char return parser function to match a single character in the input stream. Skip leading whitespace.

func End Uses

func End() Parser

End is a parser function to detect end of scanner output, return boolean as ParseNode, hence incompatible with AST{}. Instead, use AST:End method.

func Float Uses

func Float() Parser

Float return parser function to match a float literal in the input stream. Skip leading whitespace.

func Hex Uses

func Hex() Parser

Hex return parser function to match a hexadecimal literal in the input stream. Skip leading whitespace.

func Ident Uses

func Ident() Parser

Ident return parser function to match an identifier token in the input stream, an identifier is matched with the following pattern: `^[A-Za-z][0-9a-zA-Z_]*`. Skip leading whitespace.

func Int Uses

func Int() Parser

Int return parser function to match an integer literal in the input stream. Skip leading whitespace.

func Kleene Uses

func Kleene(callb Nodify, parsers ...interface{}) Parser

Kleene combinator accepts two parsers, or reference to parsers, namely opScan and sepScan, where opScan parser will be used to match input string and contruct ParsecNode, and sepScan parser will be used to match input string and ignore the matched string. If sepScan parser is not supplied, then opScan parser will be applied on the input until it fails.

The process of matching opScan parser and sepScan parser will continue in a loop until either one of them fails on the input stream.

For every successful match of opScan, the returned ParsecNode from matching parser will be accumulated and passed as argument to Nodify callback. If there is not a single match for opScan, then []ParsecNode of ZERO length will be passed as argument to Nodify callback. Kleene combinator will never fail.

func Many Uses

func Many(callb Nodify, parsers ...interface{}) Parser

Many combinator accepts two parsers, or reference to parsers, namely opScan and sepScan, where opScan parser will be used to match input string and contruct ParsecNode, and sepScan parser will be used to match input string and ignore the matched string. If sepScan parser is not supplied, then opScan parser will be applied on the input until it fails.

The process of matching opScan parser and sepScan parser will continue in a loop until either one of them fails on the input stream.

The difference between `Many` combinator and `Kleene` combinator is that there shall atleast be one match of opScan.

For every successful match of opScan, the returned ParsecNode from matching parser will be accumulated and passed as argument to Nodify callback. If there is not a single match for opScan, then Many will fail without consuming the input.

Code:

// parse comma separated values
text := []byte(`10,30,50 wont parse this`)
y := Many(nil, Int(), Atom(",", "COMMA"))
root, _ := y(NewScanner(text))
nodes := root.([]ParsecNode)
t := nodes[0].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())
t = nodes[1].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())
t = nodes[2].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())

Output:

INT 10
INT 30
INT 50

func ManyUntil Uses

func ManyUntil(callb Nodify, parsers ...interface{}) Parser

ManyUntil combinator accepts three parsers, or references to parsers, namely opScan, sepScan and untilScan, where opScan parser will be used to match input string and contruct ParsecNode, and sepScan parser will be used to match input string and ignore the matched string. If sepScan parser is not supplied, then opScan parser will be applied on the input until it fails.

The process of matching opScan parser and sepScan parser will continue in a loop until either one of them fails on the input stream or untilScan matches.

For every successful match of opScan, the returned ParsecNode from matching parser will be accumulated and passed as argument to Nodify callback. If there is not a single match for opScan, then ManyUntil will fail without consuming the input.

Code:

// make sure to parse the entire text
text := []byte("10,20,50")
y := ManyUntil(nil, Int(), Atom(",", "COMMA"), End())
root, _ := y(NewScanner(text))
nodes := root.([]ParsecNode)
t := nodes[0].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())
t = nodes[1].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())
t = nodes[2].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())

Output:

INT 10
INT 20
INT 50

func Maybe Uses

func Maybe(callb Nodify, parser interface{}) Parser

Maybe combinator accepts a single parser, or reference to a parser, and tries to match the input stream with it. If parser fails to match the input, returns MaybeNone.

Code:

// parse an optional token
equal := Atom("=", "EQUAL")
maybeand := Maybe(nil, Atom("&", "AND"))
y := And(nil, Ident(), equal, maybeand, Ident())

text := []byte("a = &b")
root, _ := y(NewScanner(text))
nodes := root.([]ParsecNode)
t := nodes[0].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())
t = nodes[1].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())
t = nodes[2].([]ParsecNode)[0].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())
t = nodes[3].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())

text = []byte("a = b")
root, _ = y(NewScanner(text))
nodes = root.([]ParsecNode)
t = nodes[0].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())
t = nodes[1].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())
fmt.Println(nodes[2])
t = nodes[3].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())

Output:

IDENT a
EQUAL =
AND &
IDENT b
IDENT a
EQUAL =
missing
IDENT b

func NoEnd Uses

func NoEnd() Parser

NoEnd is a parser function to detect not-an-end of scanner output, return boolean as ParsecNode, hence incompatible with AST{}.

func Oct Uses

func Oct() Parser

Oct return parser function to match an octal number literal in the input stream. Skip leading whitespace.

func OrdChoice Uses

func OrdChoice(callb Nodify, parsers ...interface{}) Parser

OrdChoice combinator accepts a list of `Parser`, or reference to a parser, where atleast one of the parser must match the input string. Return a parser function that can further be used to construct higher level parsers.

The first matching parser function's output is passed as argument to Nodify callback, that is []ParsecNode argument will just have one element in it. If none of the parsers match the input, then OrdChoice will fail without consuming any input.

Code:

// parse a boolean value
text := []byte(`true`)
y := OrdChoice(nil, Atom("true", "TRUE"), Atom("false", "FALSE"))
root, _ := y(NewScanner(text))
nodes := root.([]ParsecNode)
t := nodes[0].(*Terminal)
fmt.Println(t.GetName(), t.GetValue())

Output:

TRUE true

func OrdTokens Uses

func OrdTokens(patterns []string, names []string) Parser

OrdTokens to parse a single token based on one of the specified `patterns`. Skip leading whitespaces.

func String Uses

func String() Parser

String parse double quoted string in input text, this parser returns string type as ParsecNode, hence incompatible with AST combinators. Skip leading whitespace.

func Token Uses

func Token(pattern string, name string) Parser

Token takes a regular-expression pattern and return a parser that will match input stream with supplied pattern. Skip leading whitespace. `name` will be used as the Terminal's name.

func TokenExact Uses

func TokenExact(pattern string, name string) Parser

TokenExact same as Token() but pattern will be matched without skipping leading whitespace. `name` will be used as the terminal's name.

type Queryable Uses

type Queryable interface {
    // GetName for the node.
    GetName() string

    // IsTerminal return true if node is a leaf node in syntax-tree.
    IsTerminal() bool

    // GetValue return parsed text, if node is NonTerminal it will
    // concat the entire sub-tree for parsed text and return the same.
    GetValue() string

    // GetChildren relevant only for NonTerminal node.
    GetChildren() []Queryable

    // GetPosition of the first terminal value in input.
    GetPosition() int

    // SetAttribute with a value string, can be called multiple times for the
    // same attrname.
    SetAttribute(attrname, value string) Queryable

    // GetAttribute for attrname, since more than one value can be set on the
    // attribute, return a slice of values.
    GetAttribute(attrname string) []string

    // GetAttributes return a map of all attributes set on this node.
    GetAttributes() map[string][]string
}

Queryable interface to be implemented by all nodes, both terminal and non-terminal nodes constructed using AST object.

type Scanner Uses

type Scanner interface {
    // SetWSPattern to configure white space pattern. Typically used as
    //		scanner := NewScanner(input).SetWSPattern(" ")
    SetWSPattern(pattern string) Scanner

    // TrackLineno as cursor moves forward, this can slow down parsing.
    // Useful when developing with parsec package.
    TrackLineno() Scanner

    // Clone will return new clone of the underlying scanner structure.
    // This will be used by combinators to _backtrack_.
    Clone() Scanner

    // GetCursor gets the current cursor position inside input text.
    GetCursor() int

    // Match the input stream with `pattern` and return matching string
    // after advancing the scanner's cursor.
    Match(pattern string) ([]byte, Scanner)

    // Match the input stream with a simple string, rather that a
    // pattern. It should be more efficient. Return a bool indicating
    // if the match was succesfull after advancing the scanner's cursor.
    MatchString(string) (bool, Scanner)

    // SubmatchAll the input stream with a choice of `patterns`
    // and return matching string and submatches, after advancing the
    // Scanner's cursor.
    SubmatchAll(pattern string) (map[string][]byte, Scanner)

    // SkipWs skips white space characters in the input stream.
    // Return skipped whitespaces as byte-slice after advance the
    // Scanner's cursor.
    SkipWS() ([]byte, Scanner)

    // SkipAny any occurrence of the elements of the slice.
    // Equivalent to Match(`(b[0]|b[1]|...|b[n])*`)
    // Returns Scanner after advancing its cursor.
    SkipAny(pattern string) ([]byte, Scanner)

    // Lineno return the current line-number of the cursor.
    Lineno() int

    // Endof detects whether end-of-file is reached in the input
    // stream and return a boolean indicating the same.
    Endof() bool
}

Scanner interface defines necessary methods to match the input stream.

func NewScanner Uses

func NewScanner(text []byte) Scanner

NewScanner create and return a new instance of SimpleScanner object.

type SimpleScanner Uses

type SimpleScanner struct {
    // contains filtered or unexported fields
}

SimpleScanner implements Scanner interface based on golang's regexp module.

func (*SimpleScanner) Clone Uses

func (s *SimpleScanner) Clone() Scanner

Clone implement Scanner{} interface.

func (*SimpleScanner) Endof Uses

func (s *SimpleScanner) Endof() bool

Endof implement Scanner{} interface.

func (*SimpleScanner) GetCursor Uses

func (s *SimpleScanner) GetCursor() int

GetCursor implement Scanner{} interface.

func (*SimpleScanner) Lineno Uses

func (s *SimpleScanner) Lineno() int

Lineno implement Scanner{} interface.

func (*SimpleScanner) Match Uses

func (s *SimpleScanner) Match(pattern string) ([]byte, Scanner)

Match implement Scanner{} interface.

func (*SimpleScanner) MatchString Uses

func (s *SimpleScanner) MatchString(str string) (bool, Scanner)

MatchString implement Scanner{} interface.

func (*SimpleScanner) SetWSPattern Uses

func (s *SimpleScanner) SetWSPattern(pattern string) Scanner

SetWSPattern implement Scanner{} interface.

func (*SimpleScanner) SkipAny Uses

func (s *SimpleScanner) SkipAny(pattern string) ([]byte, Scanner)

SkipAny implement Scanner{} interface.

func (*SimpleScanner) SkipWS Uses

func (s *SimpleScanner) SkipWS() ([]byte, Scanner)

SkipWS implement Scanner{} interface.

func (*SimpleScanner) SkipWSUnicode Uses

func (s *SimpleScanner) SkipWSUnicode() ([]byte, Scanner)

SkipWSUnicode for looping through runes checking for whitespace.

func (*SimpleScanner) SubmatchAll Uses

func (s *SimpleScanner) SubmatchAll(patt string) (map[string][]byte, Scanner)

SubmatchAll implement Scanner{} interface.

func (*SimpleScanner) TrackLineno Uses

func (s *SimpleScanner) TrackLineno() Scanner

TrackLineno implement Scanner{} interface.

type Terminal Uses

type Terminal struct {
    Name       string // contains terminal's token type
    Value      string // value of the terminal
    Position   int    // Offset into the text stream where token was identified
    Attributes map[string][]string
}

Terminal type can be used to construct a terminal ParsecNode. It implements Queryable interface, hence can be used with AST object.

func NewTerminal Uses

func NewTerminal(name, value string, position int) *Terminal

NewTerminal create a new Terminal instance. Supply the name of the terminal, its matching text from i/p stream as value. And its position within the i/p stream.

func (*Terminal) GetAttribute Uses

func (t *Terminal) GetAttribute(attrname string) []string

GetAttribute implement Queryable interface.

func (*Terminal) GetAttributes Uses

func (t *Terminal) GetAttributes() map[string][]string

GetAttributes implement Queryable interface.

func (*Terminal) GetChildren Uses

func (t *Terminal) GetChildren() []Queryable

GetChildren implement Queryable interface.

func (*Terminal) GetName Uses

func (t *Terminal) GetName() string

GetName implement Queryable interface.

func (*Terminal) GetPosition Uses

func (t *Terminal) GetPosition() int

GetPosition implement Queryable interface.

func (*Terminal) GetValue Uses

func (t *Terminal) GetValue() string

GetValue implement Queryable interface.

func (*Terminal) IsTerminal Uses

func (t *Terminal) IsTerminal() bool

IsTerminal implement Queryable interface.

func (*Terminal) SetAttribute Uses

func (t *Terminal) SetAttribute(attrname, value string) Queryable

SetAttribute implement Queryable interface.

Directories

PathSynopsis
examples
expr
jsonPackage json provide a parser to parse JSON string.

Package parsec imports 14 packages (graph) and is imported by 16 packages. Updated 2018-08-07. Refresh now. Tools for package owners.