syntax

package
v0.0.0-...-0f4c570 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 1, 2018 License: Apache-2.0 Imports: 28 Imported by: 0

Documentation

Overview

Package syntax implements the Reflow language.

It is a simple, type-safe, applicative domain specific language that compiles to Reflow's Flow IR.

A Reflow type is one of the following (where t1, t2, etc. are themselves types):

int                                // the type of arbitrary precision integers
float                              // the type of arbitrary precision floats
string                             // the type of (UTF-8 encoded) strings
bool                               // the type of booleans
file                               // the type of files
dir                                // the type of directories
(t1, t2, .., tn)                   // the type of the tuple consisting of types
                                   // t1, t2, t3...
(id1, id2 t1, .., idn tn)          // the type of the tuple (t1, t1, ..., tn), but
                                   // with optional labels (syntactic affordance)
[t]                                // the type of list of type t
[k:v]                              // the type of map with keys of type k
                                   // and values of type v
{id1 t1, id2 t2}                   // the type of a struct with fields id1, id2
                                   // of types t1, t2 respectively
{id1, id2 t1, id3 t3}              // the type of struct{id1 t1, id2 t1, id3 t3}
                                   // (syntactic affordance)
func(t1, t2, ..., tn) tr           // the type of a function with argument types
                                   // t1, t2, ..., tn, and return type tr
func(a1, a2 t1, ..., an tn) tr     // a function of type func(t1, t1, ..., tn) tr
                                   // with labels (syntactic affordance)

A Reflow expression is one of the following (where e1, e2, .. are themselves expressions, d1, d2, .. are declarations; t1, t2, .. are types):

123                                // a literal (arbitrary precision) integer
"abc"                              // a literal UTF-8 encoded string
ident                              // an identifier
[e1, e2, e3, ..]                   // a list of e1, e2, e3..
[e1, e2, ...e3]                    // list e1, e2, concatenated with list e3
[e1:e2, e3:e4, ..]                 // a map  key e1 to value e2, etc.
[e1:e2, e3:e4, ...e5]              // as above, with map e5 appended
(e1, e2, e3, ..)                   // a tuple of e1, e2, e3, ..
{id1: e1, id2: e2, ..}             // a struct with fields id1 with value e1, id2 with value e2, ..
{id1, id2, ..}                     // a shorthand for {id1: id1, id2: id2}
{d1; d2; ...; dn; e1}              // a block of declarations usable by expression e1
func(id1, id2 t1, id3 t3) t4 => e1 // a function literal with arguments and return type; evaluates e1
func(id1, id2 t1, id3 t3) => e1    // a function literal with arguments, return type omitted
exec(d1, d2, ..) t1 {{ template }} // an exec with declarations d1, d2, .., returning t1 with template
                                   // identifiers are valid declarations in this context; they are
                                   // deparsed as id := id.
e1 <op> e2                         // a binary op (||, &&, <, >, <=, >=, !=, ==, +, /, %, &, <<, >>)
<op> e1                            // unary expression (!)
if e1 { d1; d2; ..; e2 }
else { d3; d4; ..; e3 }            // conditional expression (with declaration blocks)
(e1)                               // parenthesized expression to control precedence
int(e1)                            // builtin float to int type conversion
float(e1)                          // builtin int to float type conversion
len(e1)                            // builtin length operator
zip(e1, e2)                        // builtin zip operator
unzip(e1)                          // builtin unzip operator
map(e1)                            // convert e1 to a map
list(e1)                           // convert e1 to a list
make(strlit, d1, ..., dn)          // builtin make primitive. identifiers are valid declarations in
                                   // this context; they are deparsed as id := id.
panic(e1)                          // terminate the program with error e1
[e1 | c1, c2,..., cn]              // list comprehension: evaluate e1 in the environment provided by
                                   // the given clauses (see below)
e1 ~> e2                           // force evaluation of e1, ignore its result, then evaluate e2.
flatten(e1)                        // flatten a list of lists into a single list
trace(e1)                          // trace expression e1: evaluate it, print it to console,
                                   // and return it. Can be used for debugging.
range(e1, e2)                      // produce a list of integers with the range of the two expressions.

A comprehension clause is one of the following:

pat <- e1                          // bind a pattern to each of the list or map e1
if e1                              // filter entries that do not meet the predicate e1

A Reflow declaration is one of the following:

val id = e1 or id := e1            // assign e1 to id
val id t1 = e1                     // assign e1 to id with type t1
type id t1                         // declare id as a type alias to t1
func id(a1, a2 t1) r1 = e1         // sugar for id := func(a1, a2 t1) r1 => e1
func id(a1, a2 t1) = e1            // sugar for id := func(a1, a2 t1) => e1

Value declarations may be preceded by one of the following annoations, each of which takes a list of declarations.

@requires(...)                     // resource requirement annotation,
                                   // takes declarations mem int,
                                   // cpu int or cpu float, disk int,
                                   // cpufeatures[string, and wide
                                   // bool. They indicate resource
                                   // requirements for computing the
                                   // declaration; if wide is set to
                                   // true, then the resource
                                   // requirements have no
                                   // theoretical upper bound. Wide
                                   // is thus useful for
                                   // declarations whose
                                   // parallelization factor is not
                                   // known statically, for example
                                   // when processing sharded data.

Value declarations can take destructive pattern bindings, mimicing value constructors. Currently tuples and lists are supported. Patterns accept identifiers and "_" (ignore), but not yet literal values. Patterns follow the grammar:

_
[p1, p2, ..., pn]
(p1, p2, ..., pn)
{id1: p1, ..., idn: pn}
{id1, ..., id3}                    // sugar for {id1: id1, ..., idn: idn}

For example, the declaration

val [(x1, y1), (x2, y2), _] = e1

pattern matches e1 to a list of length three, whose elements are 2-tuples. The first two tuples are bound; the third is ignored.

A Reflow module consists of, in order: a optional keyspace, a set of optional parameters, and a set of declarations.

keyspace "com.grail.WGSv1"         // defines the keyspace to "com.grail.WGS1"

param id1 = e1                     // defines parameter id1 with default e1
param id2 string                   // defines parameter without default, of type t1

param (                            // block version of parameters
	id3 int
	id4 stirng
)

declarations                       // declarations as above

The Reflow language infers types, except for in function arguments and non-default params. Everywhere else, types can safely be omitted. When a type is supplied, it is used as an ascription: the expression is ascribed to the given type; the type checker fails if the expression's type is incompatible with its ascription.

Following Go, semicolons are inserted if a newline appears after the following tokens: identifier, string, integer, template, ')', '}', or ']'

All bytes in a line following the characters "//" are commentary.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Force

func Force(v values.T, t *types.T) values.T

Force produces a strict version of v. Force either returns an immediate value v, or else a *reflow.Flow that will produce the immediate value.

func Modules

func Modules() (names []string)

Modules returns the names of the available systems modules.

func RegisterModule

func RegisterModule(name string, m *ModuleImpl)

RegisterModule is a hook to register custom reflow intrinsics.

func Stdlib

func Stdlib() (*types.Env, *values.Env)

Stdlib returns the type and value environments for reflow's standard library.

Types

type ComprClause

type ComprClause struct {
	// Kind is the clause's kind.
	Kind ComprKind
	// Pat is the clause's pattern (ComprEnum).
	Pat *Pat
	// Expr is the clause's expression (ComprEnum, ComprFilter).
	Expr *Expr
}

A ComprClause is a single clause in a comprehension expression.

type ComprKind

type ComprKind int

ComprKind is the type of the kind of a comprehension clause.

const (
	// ComprEnum is the kind of an enumeration clause.
	ComprEnum ComprKind = iota
	// ComprFilter is the kind of a filter clause.
	ComprFilter
)

type Decl

type Decl struct {
	// Position contains the source position of the node.
	// It is set by the parser.
	scanner.Position
	// Comment stores an optional comment attached to the declaration.
	Comment string
	// Kind is the Decl's op; see above.
	Kind DeclKind
	// Ident is the identifier in a DeclDeclare and DeclType.
	Ident string
	// Pat stores the pattern for this declaration.
	Pat *Pat
	// Expr stores the rvalue expression for this declaration.
	Expr *Expr
	// Type stores the type for DeclDeclare and DeclType.
	Type *types.T
}

A Decl is a Reflow declaration.

func (*Decl) Equal

func (d *Decl) Equal(e *Decl) bool

Equal tests whether Decl d is equivalent to Decl e.

func (*Decl) Eval

func (d *Decl) Eval(sess *Session, env *values.Env, ident string) (values.T, error)

Eval evaluates this declaration with the given environment and session.

func (*Decl) ID

func (d *Decl) ID(id string) string

ID returns the best identifier for this declaration, or else id.

func (*Decl) Init

func (d *Decl) Init(sess *Session, env *types.Env) error

Init type checks this declaration. Type errors are returned.

func (*Decl) String

func (d *Decl) String() string

String renders a tree-formatted version of d.

type DeclKind

type DeclKind int

DeclKind is the type of declaration.

const (
	// DeclError is an illegal declaration.
	DeclError DeclKind = iota
	// DeclAssign is an assignment declaration (rvalue expression).
	DeclAssign
	// DeclDeclare is a "pure" declaration (type only).
	DeclDeclare
	// DeclType declares a type alias
	DeclType
)

type Expr

type Expr struct {
	// Position contains the source position of the node.
	// It is set by the parser.
	scanner.Position

	// Comment is the commentary text that precedes this expression,
	// if any.
	Comment string

	// Kind is the expression's op; see above.
	Kind ExprKind

	// Cond is the condition expression in ExprCond.
	Cond *Expr

	// Left is the "left" operand for expressions.
	Left *Expr
	// Right is the "right" operand for expressions.
	Right *Expr

	// Op is the binary operation in ExprBinop, unary operation in
	// ExprUnop, and builtin in ExprBuiltin.
	Op string

	// Args holds function arguments in an ExprFunc.
	Args []*types.Field

	// List holds expressions for list literals.
	List []*Expr

	// Map holds expressions for map literals.
	Map map[*Expr]*Expr

	// Decls holds declarations for ExprBlock and ExprExec.
	Decls []*Decl

	// Fields holds field definitions (identifiers and expressions)
	// in ExprStruct, ExprTuple
	Fields []*FieldExpr

	// Ident stores the identifier for ExprIdent
	Ident string

	// Val stores constant values in ExprConst.
	Val values.T

	// Type holds the Type in ExprAscribe, ExprExec, and ExprConst.
	Type *types.T

	// Template is the exec template in ExprExec.
	Template *Template

	ComprExpr    *Expr
	ComprClauses []*ComprClause

	// Env stores a value environmetn for ExprThunk.
	Env *values.Env

	// Pat stores the bind pattern in a comprehension.
	Pat *Pat

	// Module stores the module as opened during type checking.
	Module Module
}

An Expr is a node in Reflow's expression AST.

func (*Expr) Abbrev

func (e *Expr) Abbrev() string

Abbrev shows an "abbreviated" pretty-printed version of expression e. These are useful when showing expression values in documentary output; but they do not necessarily parse. Abbrev strips unnecessary parentheses from arithmetic expressions.

func (*Expr) Digest

func (e *Expr) Digest(env *values.Env) digest.Digest

Digest computes an identifier that uniquely identifies what Expr e will compute with the given environment. i.e., it can be used as a key to identify the computation represented by Expr e and an environment.

On a semantic level, Digest is one-to-many: that is, there are many candidate digests for a given semantic computation. However, we go through some lengths to normalize, for example by using De Bruijn indices (levels) to remove dependence on concrete names. In the future, we could consider canonicalizing the expression tree as well (e.g., by exploiting commutatvity, etc.)

func (*Expr) Equal

func (e *Expr) Equal(f *Expr) bool

Equal tests whether expression e is equivalent to expression f.

func (*Expr) Init

func (e *Expr) Init(sess *Session, env *types.Env) error

Init performs type checking and synthesis on this expression tree; sets each node's Type field, and then returns any type error.

func (*Expr) String

func (e *Expr) String() string

String renders a tree-formatted version of e.

func (*Expr) Subexpr

func (e *Expr) Subexpr() []*Expr

Subexpr returns a slice of this expression's dependencies.

type ExprKind

type ExprKind int

ExprKind is the kind of an expression.

const (
	// ExprError indicates an erroneous expression (e.g., through a parse error)
	ExprError ExprKind = iota
	// ExprIdent is an identifier reference.
	ExprIdent
	// ExprBinop is a binary operation.
	ExprBinop
	// ExprUnop is a unary operation.
	ExprUnop
	// ExprApply is function application.
	ExprApply
	// ExprConst is a const (literal).
	ExprConst
	// ExprAscribe is an ascription (static type assertion).
	ExprAscribe
	// ExprBlock is a declaration-block.
	ExprBlock
	// ExprFunc is a function definition.
	ExprFunc
	// ExprTuple is a tuple literal.
	ExprTuple
	// ExprStruct is a struct literal.
	ExprStruct
	// ExprList is a list literal.
	ExprList
	// ExprMap is a map literal.
	ExprMap
	// ExprExec is an exec expression.
	ExprExec
	// ExprCond is a conditional expression.
	ExprCond
	// ExprDeref is a struct derefence expression.
	ExprDeref
	// ExprIndex is an indexing (map or list) expression.
	ExprIndex
	// ExprCompr is a comprehension expression.
	ExprCompr
	// ExprMake is a module instantiation expression.
	ExprMake
	// ExprBuiltin is a builtin expression (e.g., len, zip, unzip).
	ExprBuiltin
	// ExprRequires assigns resources to the underlying expression.
	// It also necessarily forces the value.
	ExprRequires
	// ExprThunk is a delayed evaluation (expression + environment).
	// These are never produced from parsing--they are used internally
	// by the evaluator. (But see note there.)
	ExprThunk
)

type ExprValue

type ExprValue struct {
	Left, Right values.T
	Cond        values.T
	List        []values.T
	Map         []struct{ K, V *values.T }
	Fields      []values.T
	Extras      []values.T
}

ExprValue stores the evaluated values associated with the dependencies of an Expr node. It is used while evaluating an expression tree.

type FieldExpr

type FieldExpr struct {
	Name string
	*Expr
}

FieldExpr stores a field name and expression.

func (*FieldExpr) Equal

func (f *FieldExpr) Equal(e *FieldExpr) bool

Equal tests whether f is equivalent to e.

type MatchKind

type MatchKind int

MatchKind is the kind of match performed by a Matcher.

const (
	// MatchError is an erroneous matcher.
	MatchError MatchKind = iota
	// MatchValue matches a value.
	MatchValue
	// MatchTuple indexes a tuple.
	MatchTuple
	// MatchList indexes a list.
	MatchList
	// MatchStruct indexes a struct
	MatchStruct
)

type Matcher

type Matcher struct {
	// Kind is the kind of matcher.
	Kind MatchKind
	// Index is the index of the match (MatchTuple, MatchList).
	Index int
	// Parent is this matcher's parent.
	Parent *Matcher
	// Field holds a struct field (MatchStruct).
	Field string
}

A Matcher binds individual pattern components (identifiers) in a pattern. Matchers form a tree; their interpretation (through method Match) performs value destructuring.

func (*Matcher) Path

func (m *Matcher) Path() Path

Path constructs a path from this matcher. The path may be used to simultaneously deconstruct a value and type.

type Module

type Module interface {
	// Make creates a new module instance in the provided session
	// with the provided parameters.
	Make(sess *Session, params *values.Env) (values.T, error)
	// ParamErr type-checks parameter types, returning an error on failure.
	ParamErr(env *types.Env) error
	// Flags returns the set of flags provided by this module.
	// Note that not all modules may have parameters that are supported
	// by the regular flag types. These return an error.
	Flags(sess *Session, env *values.Env) (*flag.FlagSet, error)
	// FlagEnv adds flags from the FlagSet to value environment env.
	// The FlagSet should be produced by Module.Flags.
	FlagEnv(flags *flag.FlagSet, env *values.Env) error
	// Params returns the parameter descriptors for this module.
	Params() []Param
	// Doc returns the docstring for a toplevel identifier.
	Doc(string) string
	// Type returns the type of the module.
	Type() *types.T
	// Eager tells whether the module requires eager parameters.
	// When it does, all parameters are forced and fully evaluated
	// before instantiating a new module instance.
	Eager() bool
}

Module abstracts a Reflow module, having the ability to type check parameters, inspect its type, and mint new instances.

type ModuleImpl

type ModuleImpl struct {
	// Keyspace is the (optional) key space of this module.
	Keyspace *Expr
	// Reservation is the set of reservation declarations.
	Reservation []*Decl
	// ParamDecls is the set of declared parameters for this module.
	ParamDecls []*Decl
	// Decls is the set of declarations in this module.
	Decls []*Decl

	Docs map[string]string
	// contains filtered or unexported fields
}

ModuleImpl defines a Reflow module comprising: a keyspace, a set of parameters, and a set of declarations.

func (*ModuleImpl) Doc

func (m *ModuleImpl) Doc(ident string) string

Doc returns the documentation for the provided identifier.

func (*ModuleImpl) Eager

func (m *ModuleImpl) Eager() bool

Eager returns false.

func (*ModuleImpl) FlagEnv

func (m *ModuleImpl) FlagEnv(flags *flag.FlagSet, env *values.Env) error

FlagEnv adds flags from the FlagSet to value environment env. The FlagSet should be produced by (*Module).Flags.

func (*ModuleImpl) Flags

func (m *ModuleImpl) Flags(sess *Session, env *values.Env) (*flag.FlagSet, error)

Flags returns a FlagSet that captures the parameters of this module. This can be used to parameterize a module from the command line. The returned FlagSet uses parameter documentation as the help text.

func (*ModuleImpl) Init

func (m *ModuleImpl) Init(sess *Session, env *types.Env) error

Init type checks this module and returns any type checking errors.

func (*ModuleImpl) Make

func (m *ModuleImpl) Make(sess *Session, params *values.Env) (values.T, error)

Make creates a new instance of this module. ParamDecls contains the value environment storing parameter values.

func (*ModuleImpl) Param

func (m *ModuleImpl) Param(id string) (*types.T, bool)

Param returns the type of the module parameter with identifier id, and whether it is mandatory.

func (*ModuleImpl) ParamErr

func (m *ModuleImpl) ParamErr(env *types.Env) error

ParamErr type checks the type environment env against the parameters of this module. It returns any type checking errors (e.g., badly typed parameters, or missing ones).

func (*ModuleImpl) Params

func (m *ModuleImpl) Params() []Param

Params returns the parameter metadata for this module.

func (*ModuleImpl) String

func (m *ModuleImpl) String() string

String renders a tree-formatted version of m.

func (*ModuleImpl) Type

func (m *ModuleImpl) Type() *types.T

Type returns the module type of m.

type Param

type Param struct {
	Ident    string
	Type     *types.T
	Doc      string
	Expr     *Expr
	Required bool
}

Param holds module parameter metadata.

type Parser

type Parser struct {
	// File is prefixed to parser error locations.
	File string
	// Body is the io.Reader that is parsed.
	Body io.Reader

	// Mode governs how the parser is started. See documentation above.
	// The fields Module, Decls, Expr, and Type are set depending on the
	// parser mode.
	Mode ParserMode

	// Module contains the parsed module (LexerModule).
	Module *ModuleImpl
	// Decls contains the parsed declarations (LexerDecls).
	Decls []*Decl
	// Expr contains the parsed expression (LexerExpr).
	Expr *Expr
	// Type contains the pased type (LexerType).
	Type *types.T
	// contains filtered or unexported fields
}

Parser is a Reflow lexer. It composes an (internal) scanner to produce tokens for the YACC grammar. Parser inserts semicolons following the rules outlined in the package docs. Lex implements the (internal) yyParser.

func (*Parser) Error

func (x *Parser) Error(s string)

Error reports an error to the lexer.

func (*Parser) Lex

func (x *Parser) Lex(yy *yySymType) (xx int)

Lex returns the next token.

func (*Parser) Parse

func (x *Parser) Parse() error

Parse parses the parser's body and reports any parsing error. The parse result is deposited in x.Module, x.Decls, or x.Expr, depending on the parser's mode.

type ParserMode

type ParserMode int

ParserMode determines the lexer's entry behavior.

const (
	// ParseModule parses a module.
	ParseModule ParserMode = iota
	// ParseDecls parses a set of declarations.
	ParseDecls
	// ParseExpr parses an expression.
	ParseExpr
	// ParseType parses a type.
	ParseType
)

type Pat

type Pat struct {
	scanner.Position

	Kind PatKind

	Ident string
	List  []*Pat

	Map map[string]*Pat
}

A Pat stores a pattern tree used in destructuring operations. Patterns can bind type and value environments. They can also produce matchers that can be used to selectively match and bind identifiers.

func (*Pat) BindTypes

func (p *Pat) BindTypes(env *types.Env, t *types.T) error

BindTypes binds the pattern's identifier's types in the passed environment, given the type of binding value t.

func (*Pat) BindValues

func (p *Pat) BindValues(env *values.Env, v values.T) bool

BindValues binds this pattern's values in the given value environment.

func (*Pat) Debug

func (p *Pat) Debug() string

Debug prints the pattern's AST for debugging.

func (*Pat) Equal

func (p *Pat) Equal(q *Pat) bool

Equal tells whether pattern p is equal to pattern q.

func (*Pat) Idents

func (p *Pat) Idents(ids []string) []string

Idents appends the pattern's bound identifiers to the passed slice.

func (*Pat) Matchers

func (p *Pat) Matchers() map[string]*Matcher

Matchers returns a map of matchers representing this pattern.

func (*Pat) String

func (p *Pat) String() string

String prints a parseable representation of the pattern.

type PatKind

type PatKind int

PatKind is the kind of pattern.

const (
	// PatError is an erroneous pattern
	// (e.g., uninitialized or parse error).
	PatError PatKind = iota
	// PatIdent is an identifier pattern.
	PatIdent
	// PatTuple is a tuple pattern.
	PatTuple
	// PatList is a list pattern.
	PatList
	// PatStruct is a struct pattern.
	PatStruct
	// PatIgnore is an ignore pattern.
	PatIgnore
)

type Path

type Path []*Matcher

Path represents a path to a value.

func (Path) Digest

func (p Path) Digest() digest.Digest

Digest returns a digest representing this path.

func (Path) Done

func (p Path) Done() bool

Done tells whether this path is complete.

func (Path) Match

func (p Path) Match(v values.T, t *types.T) (values.T, *types.T, bool, Path)

Match performs single step deconstruction of a type and value. It returns the next level; terminating when len(Path) == 0.

type Session

type Session struct {
	// Stdout and stderr is the writer to which standard output and error are written.
	Stdout, Stderr io.Writer

	Types  *types.Env
	Values *values.Env
	// contains filtered or unexported fields
}

A Session is a compiler session. It's responsible for opening, parsing and type checking modules.

func NewSession

func NewSession() *Session

NewSession creates and initializes a session.

func (*Session) Images

func (s *Session) Images() []string

Images returns images encountered so far during expression evaluation.

func (*Session) Open

func (s *Session) Open(path string) (Module, error)

Open parses and type checks, and then returns the module at the given path. It then returns the module and any associated error.

func (*Session) SeeImage

func (s *Session) SeeImage(image string)

SeeImage records an image name. Call during expression evaluation.

type SystemFunc

type SystemFunc struct {
	Module string
	Id     string
	Doc    string
	Type   *types.T
	Force  bool
	Do     func(loc values.Location, args []values.T) (values.T, error)
}

SystemFunc is utility to define a reflow intrinsic.

func (SystemFunc) Apply

func (s SystemFunc) Apply(loc values.Location, args []values.T) (values.T, error)

Apply applied the intrinsic with the given arguments.

func (SystemFunc) Decl

func (s SystemFunc) Decl() *Decl

Decl returns the intrinsic as a reflow declaration.

func (SystemFunc) Digest

func (s SystemFunc) Digest() digest.Digest

Digest computes the digest of the intrinsic.

type Template

type Template struct {
	Text  string
	Frags []string
	Args  []*Expr
}

Template is an exec template and its interpolation arguments. The template is stored as a number of fragments interspersed by argument expressions to be rendered.

The following are guaranteed invariant:

len(Frags) > 0
len(Frags) == len(Args)+1

func (*Template) FormatString

func (t *Template) FormatString() string

FormatString returns a format string that can be used to render the final template. It's provided for backwards compatibility only.

func (*Template) String

func (t *Template) String() string

String returns t.Text.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL