tavor: github.com/zimmski/tavor/token Index | Files | Directories

package token

import "github.com/zimmski/tavor/token"

Index

Package Files

analyze.go parsererrortype_string.go print.go set.go token.go transform.go typed.go walk.go

func ListTyped Uses

func ListTyped() []string

ListTyped returns a list of all registered typed token names.

func LoopExists Uses

func LoopExists(root Token) bool

LoopExists determines if a cycle exists in the internal token graph

func PrettyPrintInternalTree Uses

func PrettyPrintInternalTree(w io.Writer, root Token)

PrettyPrintInternalTree prints the internal token represenation as a text tree

func PrettyPrintTree Uses

func PrettyPrintTree(w io.Writer, root Token)

PrettyPrintTree prints the token represenation as a text tree

func RegisterTyped Uses

func RegisterTyped(name string, ct CreateTypedFunc)

RegisterTyped registers a typed token creation function with the given name.

func ReleaseTokens Uses

func ReleaseTokens(root Token)

ReleaseTokens traverses the token graph and calls Release for every release token

func ResetCombinedScope Uses

func ResetCombinedScope(root Token)

ResetCombinedScope resets all scopes of external and interal tokens in the token graph that fullfill the ScopeToken interface

func ResetInternalScope Uses

func ResetInternalScope(root Token)

ResetInternalScope resets all scopes of interal tokens in the token graph that fullfill the ScopeToken interface

func ResetResetTokens Uses

func ResetResetTokens(root Token) (err error)

ResetResetTokens resets all tokens in the token graph that fullfill the ResetToken interface

func ResetScope Uses

func ResetScope(root Token)

ResetScope resets all scopes of tokens in the token graph that fullfill the ScopeToken interface

func SetCombinedScope Uses

func SetCombinedScope(root Token, scope *VariableScope)

SetCombinedScope sets all scopes of external and internal tokens in the token graph that fullfill the ScopeToken interface

func SetInternalScope Uses

func SetInternalScope(root Token, scope *VariableScope)

SetInternalScope sets all scopes of internal tokens in the token graph that fullfill the ScopeToken interface

func SetScope Uses

func SetScope(root Token, scope *VariableScope)

SetScope sets all scopes of tokens in the token graph that fullfill the ScopeToken interface

func Walk Uses

func Walk(root Token, walkFunc func(tok Token) error) error

Walk traverses a token graph beginning from the given token and calls for every newly visited token the given function. A depth-first algorithm is used to traverse the graph. If the given walk function returns an error, the whole walk process ends by returning the error back to the caller

func WalkInternal Uses

func WalkInternal(root Token, walkFunc func(tok Token) error) error

WalkInternal traverses a internal token graph beginning from the given token and calls for every newly visited token the given function. A depth-first algorithm is used to traverse the graph. If the given walk function returns an error, the whole walk process ends by returning the error back to the caller

func WalkInternalTail Uses

func WalkInternalTail(root Token, walkFunc func(tok Token) error) error

WalkInternalTail traverses a internal token graph beginning from the given token and calls for every newly visited token the given function after it has traversed all children. A depth-first algorithm is used to traverse the graph. If the given walk function returns an error, the whole walk process ends by returning the error back to the caller

type ArgumentsTypedParser Uses

type ArgumentsTypedParser interface {
    // GetInt tries to parse the argument name and returns its integer value or defaultValue if the argument is not found.
    // The return value is valid only if Err returns nil.
    GetInt(name string, defaultValue int) int
    // Err returns the first error encountered by the ArgumentsTypedParser.
    Err() error
}

ArgumentsTypedParser defines a parser for the arguments of a typed token. Parsing stops unrecoverably at the first error. The return value of Err must be checked before using the values returned by precedings calls to GetInt.

type CreateTypedFunc Uses

type CreateTypedFunc func(argParser ArgumentsTypedParser) (Token, error)

CreateTypedFunc defines a function to create a typed token

type Follow Uses

type Follow interface {
    // Follow returns if the children of the token should be traversed
    Follow() bool
}

Follow defines if the children of a token should be traversed

type Forward Uses

type Forward interface {
    // Get returns the current referenced token
    Get() Token

    // InternalGet returns the current referenced internal token
    InternalGet() Token
    // InternalLogicalRemove removes the referenced internal token and returns the replacement for the current token or nil if the current token should be removed.
    InternalLogicalRemove(tok Token) Token
}

Forward defines a forward token which can reference another token

type ForwardToken Uses

type ForwardToken interface {
    Token
    Forward
    InternalReplace
}

ForwardToken combines the Token and Forward interface

type Index Uses

type Index interface {
    // Index returns the index of this token in its parent token
    Index() int
}

Index defines an index token which provides the index in its parent token

type IndexToken Uses

type IndexToken interface {
    Token
    Index
}

IndexToken combines the Token and Index interface

type InternalParser Uses

type InternalParser struct {
    Data    string
    DataLen int
}

InternalParser holds the data information for an internal parser

func (InternalParser) GetPosition Uses

func (p InternalParser) GetPosition(i int) scanner.Position

GetPosition returns a text position in the data given an index of the data

type InternalReplace Uses

type InternalReplace interface {
    // InternalReplace replaces an old with a new internal token if it is referenced by this token. The error return argument is not nil, if the replacement is not suitable.. The error return argument is not nil, if the replacement is not suitable.
    InternalReplace(oldToken, newToken Token) error
}

InternalReplace defines if a token has methods to replace internal tokens

type Len Uses

type Len interface {
    // Len returns the number of the current referenced tokens
    Len() int
}

Len defines a general len token

type LenToken Uses

type LenToken interface {
    Token
    Len
}

LenToken combines the Token and Len interface

type List Uses

type List interface {
    // Get returns the current referenced token at the given index. The error return argument is not nil, if the index is out of bound.
    Get(i int) (Token, error)
    // Len returns the number of the current referenced tokens
    Len() int

    // InternalGet returns the current referenced internal token at the given index. The error return argument is not nil, if the index is out of bound.
    InternalGet(i int) (Token, error)
    // InternalLen returns the number of referenced internal tokens
    InternalLen() int
    // InternalLogicalRemove removes the referenced internal token and returns the replacement for the current token or nil if the current token should be removed.
    InternalLogicalRemove(tok Token) Token
}

List defines a general list token

type ListToken Uses

type ListToken interface {
    Token
    List
    InternalReplace
}

ListToken combines the Token and List interface

type Minimize Uses

type Minimize interface {
    // Minimize tries to minimize itself and returns a token if it was successful, or nil if there was nothing to minimize
    Minimize() Token
}

Minimize defines a minimize token which has methods to reduce itself to easier constructs

type MinimizeToken Uses

type MinimizeToken interface {
    Token
    Minimize
}

MinimizeToken combines the Token and Minimize interface

type Optional Uses

type Optional interface {
    // IsOptional checks dynamically if this token is in the current state optional
    IsOptional() bool
    // Activate activates this token
    Activate()
    // Deactivate deactivates this token
    Deactivate()
}

Optional defines an optional token which can be (de)activated

type OptionalToken Uses

type OptionalToken interface {
    Token
    Optional
}

OptionalToken combines the Token and Optional interface

type ParserError Uses

type ParserError struct {
    Message string
    Type    ParserErrorType

    Position scanner.Position
}

ParserError holds a parser error

func (*ParserError) Error Uses

func (err *ParserError) Error() string

type ParserErrorType Uses

type ParserErrorType int

ParserErrorType the parser error type

const (
    // ParseErrorNoStart no Start token was defined
    ParseErrorNoStart ParserErrorType = iota
    // ParseErrorNewLineNeeded a new line is needed
    ParseErrorNewLineNeeded
    // ParseErrorEarlyNewLine an unexpected new line was encountered
    ParseErrorEarlyNewLine
    // ParseErrorEmptyExpressionIsInvalid empty expressions are not allowed
    ParseErrorEmptyExpressionIsInvalid
    // ParseErrorEmptyString an empty string was detected which is not allowed
    ParseErrorEmptyString
    // ParseErrorEmptyTokenDefinition empty token definitions are not allowed
    ParseErrorEmptyTokenDefinition
    // ParseErrorInvalidArgumentValue invalid argument value
    ParseErrorInvalidArgumentValue
    // ParseErrorInvalidTokenName invalid token name
    ParseErrorInvalidTokenName
    // ParseErrorInvalidTokenType invalid token type
    ParseErrorInvalidTokenType
    // ParseErrorUnusedToken token is unused
    ParseErrorUnusedToken
    // ParseErrorMissingTypedTokenArgument a typed token argument is missing
    ParseErrorMissingTypedTokenArgument
    // ParseErrorNonTerminatedString string is not properly terminated
    ParseErrorNonTerminatedString
    // ParseErrorNoTokenForVariable variable is not assigned to a token
    ParseErrorNoTokenForVariable
    // ParseErrorNotAlwaysUsedAsAVariable token is not always used as a variable but at least sometimes in a variable context
    ParseErrorNotAlwaysUsedAsAVariable
    // ParseErrorRepeatWithOptionalTerm a repeat with an optional term was detected, which is forbidden
    ParseErrorRepeatWithOptionalTerm
    // ParseErrorTokenAlreadyDefined token name is already in use
    ParseErrorTokenAlreadyDefined
    // ParseErrorTokenNotDefined there is no token with this name
    ParseErrorTokenNotDefined
    // ParseErrorTypeNotDefinedForTypedToken type is not defined for this typed token
    ParseErrorTypeNotDefinedForTypedToken
    // ParseErrorExpectRune the given rune would be expected
    ParseErrorExpectRune
    // ParseErrorExpectOperator the given operator would be expected
    ParseErrorExpectOperator
    // ParseErrorUnknownBooleanOperator the boolean operator is unknown
    ParseErrorUnknownBooleanOperator
    // ParseErrorUnknownCondition the condition is unknown
    ParseErrorUnknownCondition
    // ParseErrorUnkownOperator the operator is unknown
    ParseErrorUnkownOperator
    // ParseErrorUnknownTypedTokenArgument the typed token argument is unknown
    ParseErrorUnknownTypedTokenArgument
    // ParseErrorUnknownTypedTokenType the typed token type is unknown
    ParseErrorUnknownTypedTokenType
    // ParseErrorUnknownTokenAttribute the token attribute is unknown
    ParseErrorUnknownTokenAttribute
    // ParseErrorUnexpectedTokenDefinitionTermination token definition was unexpectedly terminated
    ParseErrorUnexpectedTokenDefinitionTermination
    // ParseErrorExpectedExpressionTerm expression term is expected
    ParseErrorExpectedExpressionTerm
    // ParseErrEndlessLoopDetected an invalid loop was detected
    ParseErrEndlessLoopDetected

    // ParseErrorExpectedEOF expected EOF
    ParseErrorExpectedEOF
    // ParseErrorRootIsNil root token is nil
    ParseErrorRootIsNil
    // ParseErrorUnexpectedEOF EOF was not expected
    ParseErrorUnexpectedEOF
    // ParseErrorUnexpectedData additional data was not expected
    ParseErrorUnexpectedData
)

go:generate stringer -type=ParserErrorType

func (ParserErrorType) String Uses

func (i ParserErrorType) String() string

type PermutationError Uses

type PermutationError struct {
    Type PermutationErrorType
}

PermutationError holds a permutation error

func (*PermutationError) Error Uses

func (err *PermutationError) Error() string

type PermutationErrorType Uses

type PermutationErrorType int

PermutationErrorType the permutation error type

const (
    // PermutationErrorIndexOutOfBound an index not in the bound of available permutations was used.
    PermutationErrorIndexOutOfBound PermutationErrorType = iota
)

type Pointer Uses

type Pointer interface {
    Forward

    // Set sets the referenced token which must conform to the pointers token reference type
    Set(o Token) error
}

Pointer defines a pointer token which can reference another token and can be reset to reference another token

type PointerToken Uses

type PointerToken interface {
    Token
    Pointer
}

PointerToken combines the Token and Pointer interface

type Reduce Uses

type Reduce interface {
    // Reduce sets a specific reduction for this token
    Reduce(i uint) error
    // Reduces returns the number of reductions for this token
    Reduces() uint
}

Reduce defines a reduce token which provides methods to reduce itself and its children

type ReduceError Uses

type ReduceError struct {
    Type ReduceErrorType
}

ReduceError holds a reduce error

func (*ReduceError) Error Uses

func (err *ReduceError) Error() string

type ReduceErrorType Uses

type ReduceErrorType int

ReduceErrorType the reduce error type

const (
    // ReduceErrorIndexOutOfBound an index not in the bound of available reductions was used.
    ReduceErrorIndexOutOfBound ReduceErrorType = iota
)

type ReduceToken Uses

type ReduceToken interface {
    Token
    Reduce
}

ReduceToken combines the Token and Reduce interface

type Release Uses

type Release interface {
    // Release gives the token a chance to remove resources
    Release()
}

Release defines a release token which provides methods to release resources on removal

type ReleaseToken Uses

type ReleaseToken interface {
    Token
    Release
}

ReleaseToken combines the Token and Release interface

type Reset Uses

type Reset interface {
    // Reset resets the (internal) state of this token and its dependences, returns an error if the reseted state should not be used for a generation.
    Reset() error
}

Reset defines a reset token which can reset its (internal) state

type ResetToken Uses

type ResetToken interface {
    Token
    Reset
}

ResetToken combines the Token and Index interface

type Resolve Uses

type Resolve interface {
    // Resolve returns the token which is referenced by the token, or a path of tokens
    Resolve() Token
}

Resolve defines if a token has methods to resolve its token path

type Scope Uses

type Scope interface {
    // SetScope sets the scope of the token
    SetScope(variableScope *VariableScope)
}

Scope defines a scope token which holds a scope

type ScopeToken Uses

type ScopeToken interface {
    Token
    Scope
}

ScopeToken combines the Token and Scope interface

type Scoping Uses

type Scoping interface {
    // Scoping returns if the token holds a new scope
    Scoping() bool
}

Scoping defines a scoping token which holds a new scope

type Token Uses

type Token interface {
    fmt.Stringer

    // Clone returns a copy of the token and all its children
    Clone() Token

    // Permutation sets a specific permutation for this token
    Permutation(i uint) error
    // Permutations returns the number of permutations for this token
    Permutations() uint
    // PermutationsAll returns the number of all possible permutations for this token including its children
    PermutationsAll() uint

    // Parse tries to parse the token beginning from the current position in the parser data.
    // If the parsing is successful the error argument is nil and the next current position after the token is returned.
    Parse(pars *InternalParser, cur int) (int, []error)
}

Token defines a general token

func MinimizeTokens Uses

func MinimizeTokens(root Token) (Token, error)

MinimizeTokens traverses the token graph and replaces unnecessary complicated constructs with their simpler form One good example is an All list token with one token which can be replaced by this one token. The minimize checks and operation is done by the token itself which has to implement the MinimizeToken interface, since it is not always predictable if a token with one child is doing something special,

func NewTyped Uses

func NewTyped(name string, argParser ArgumentsTypedParser, pos scanner.Position) (Token, error)

NewTyped creates a typed token by calling the function registered with the given name. The error return argument is not nil if the name does not exist in the registered typed token list or if the token creation failed.

func UnrollPointers Uses

func UnrollPointers(root Token) (Token, error)

UnrollPointers unrolls pointer tokens by copying their referenced graphs. Pointers that lead to themselves are unrolled at maximum tavor.MaxRepeat times.

type Variable Uses

type Variable interface {
    Forward
    Index
    Scope

    // Name returns the name of the variable
    Name() string

    // Len returns the number of the current referenced tokens
    Len() int
}

Variable defines a variable token which holds a variable

type VariableScope Uses

type VariableScope struct {
    // contains filtered or unexported fields
}

VariableScope holds a variable scope and a reference to its parent scope

func NewVariableScope Uses

func NewVariableScope() *VariableScope

NewVariableScope returns a new instance of a variable scope

func NewVariableScopeFrom Uses

func NewVariableScopeFrom(s map[string]Token) *VariableScope

NewVariableScopeFrom returns a new instance of a variable scope initializing the scope with the given map

func (*VariableScope) Combine Uses

func (s *VariableScope) Combine() map[string]Token

Combine returns a map which holds the combination of all variable scopes

func (*VariableScope) Get Uses

func (s *VariableScope) Get(name string) Token

Get searches the variable scope for a variable with the given name and returns the token, or nil if there is no variable with the given name

func (*VariableScope) Pop Uses

func (s *VariableScope) Pop() *VariableScope

Pop returns the parent scope, or panics if there is no parent scope

func (*VariableScope) Push Uses

func (s *VariableScope) Push() *VariableScope

Push creates a new variable scope and returns it

func (*VariableScope) Set Uses

func (s *VariableScope) Set(name string, tok Token)

Set sets a variable with the given name

type VariableToken Uses

type VariableToken interface {
    Token
    Variable
}

VariableToken combines the Token and Variable interface

Directories

PathSynopsis
aggregates
conditions
constraints
expressions
filters
lists
primitives
sequences
variables

Package token imports 8 packages (graph) and is imported by 26 packages. Updated 2018-11-29. Refresh now. Tools for package owners.