README
¶
A framework for constructing powerful lexical preprocessors for go, in go. Quick overview of how it works: tokens (goprep.Token) are sent down a 'pipeline', which is a series of 'pipes'. Each pipe represents a link between two goroutines. Each goroutine may drop, add, or modify tokens it receives before passing them on to the next goroutine. Goroutines may also directly write string output (to a `chan string'). Look at goforpp for an example.
Documentation
¶
Overview ¶
This package provides a framework for creating powerful lexical preprocessors for go code.
Index ¶
- func False(Token) bool
- func Ignore(f func(Token) bool) func(*Pipe)
- func IgnoreToken(str string) func(*Pipe)
- func IgnoreType(tok token.Token) func(*Pipe)
- func Lines(pipeline *Pipe)
- func Link(f func(chan Token, chan Token, chan string, chan interface{})) func(*Pipe)
- func Pass(f func(Token) bool) func(*Pipe)
- func PassToken(str string) func(*Pipe)
- func PassType(tok token.Token) func(*Pipe)
- func PipeEnd(p *Pipe, oWriter io.Writer)
- func True(Token) bool
- type Pipe
- type Token
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Ignore ¶
Ignore produces a modified input stream that does not include any tokens for which f evaluates to true, thus discarding a certain class of tokens.
func IgnoreToken ¶
IgnoreToken is like Ignore, discarding all tokens whose string content is equal to the given string.
func IgnoreType ¶
IgnoreType is like Ignore, discarding all tokens of a certain type.
func Lines ¶
func Lines(pipeline *Pipe)
Lines passes line pragma information as needed to the output channel, thus ensuring that line numbers in the output match line numbers in the input.
func Pass ¶
Pass redirects all tokens for which f evaluates to true to the output channel, returning the altered input channel.
func PassToken ¶
PassToken is like Pass, passing all tokens whose string content is equal to the given string.
Types ¶
type Pipe ¶
Pipe represents a unit of the pipeline from Token inputs to string outputs. This would probably be a monad in haskell.
We're using channels to serve as an abstracted for-loop, not because we want parallelism. All processing is synchronized with Sync. A message to Sync signals that the token has been 'used' (sent to Output or discarded), and the sender is ready for the next. Since adding a read to sync (which is 'always' being read from by the frontend goroutine) would result in undeterministic behaviour, routines originating artificial tokens must create a new synchronization channel.
Source Files