scanner

type MacroMap ¶ added in v0.6.0

type MacroMap map[string]func(string) ([]Token, error)

MacroMap specifies mappings of macro names to their definitions. For example, an entry with the key "example" would be available in scanned code as

@example[some input or another]

In this example, the string "some input or another" would be passed to the definition. The tokens returned by the macro are inserted raw into the token stream that is yielded by the scanner. Returned tokens of type Macro are reprocessed via the same map.

type Scanner ¶

type Scanner struct {
	// contains filtered or unexported fields
}

A Scanner tokenizes runes from an io.Reader.

package main

import (
	"io"

	"github.com/DeedleFake/wdte/scanner"
)

var r io.Reader

func main() {
	s := scanner.New(r, nil)
	for s.Scan() {
		/* Do something with s.Tok(). */
	}
	if err := s.Err(); err != nil {
		panic(err)
	}
}

Output:

func New(r io.Reader, macros MacroMap) *Scanner

func (s *Scanner) Err() error

func (s *Scanner) Pos() (line, col int)

func (s *Scanner) Scan() bool

func (s *Scanner) Tok() Token

type Token ¶

type Token struct {
	Line, Col int
	Type      TokenType
	Val       interface{}
}

A Token is a usable element parsed from a string.

type TokenType ¶

type TokenType uint

TokenType is the type of a token.

const (
	Invalid TokenType = iota
	Number
	String
	ID
	Keyword
	Macro
	EOF
)

func (t TokenType) String() string