fishi

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 5, 2023 License: MIT Imports: 29 Imported by: 0

Documentation

Overview

Package fishi provides parsing and reading of FISHI frontend specifications for ictiobus.

Index

Constants

View Source
const (
	ComponentTokens   = "tokens"
	ComponentLexer    = "lexer"
	ComponentParser   = "parser"
	ComponentSDTS     = "sdts"
	ComponentFrontend = "frontend"
	ComponentMainFile = "main"
)

Names of each component of the generated compiler. Each component represents one file that is generated.

View Source
const (
	// DiagGenerationDir is the name of the directory in which source code for
	// the diagnostic binary is placed when it is generated, prior to
	// compilation.
	DiagGenerationDir = ".gen"

	// SimGenerationDir is the name of the directory in which source code for
	// the simulation binary is placed when it is generated, prior to
	// compilation.
	SimGenerationDir = ".sim"
)
View Source
const (
	// CommandName is the name of the ictiobus binary.
	CommandName = "ictcc"
)

Variables

This section is empty.

Functions

func GenerateDiagnosticsBinary added in v0.6.1

func GenerateDiagnosticsBinary(spec Spec, md SpecMetadata, params DiagBinParams) error

GenerateDiagnosticsBinary generates a binary that can read input written in the language specified by the given Spec and SpecMetadata and print out basic information about the analysis, with the goal of printing out the constructed intermediate representation from analyzed files.

The args formatPkgDir and formatCall are used to specify preformatting for code that that the generated binary will analyze. If set, io.Readers that are opened on code input will be passed to the function specified by formatCall. If formatCall is set, formatPkgDir must also be set, even if it is already specified by another parameter. formatCall must be the name of a function within the package specified by formatPkgDir which takes an io.Reader and returns a new io.Reader that wraps the one passed in and returns preformatted code ready to be analyzed by the generated frontend.

localSource only needed if doing dev in ictiobus; otherwise latest ictiobus published version is used.

func GenerateFrontendGo added in v0.7.0

func GenerateFrontendGo(spec Spec, md SpecMetadata, pkgName, pkgDir string, pkgImport string, opts *CodegenOptions) error

GenerateFrontendGo generates the source code for a compiler frontend that can handle a fishi spec. The source code is placed in the given directory. This does *not* copy the hooks package, it only outputs the frontend code.

If opts is nil, the default options will be used.

func NewSpec added in v0.5.0

func NewSpec(ast AST) (spec Spec, warnings []Warning, err error)

NewSpec reads an AST and creates a LanguageSpec from it. If the AST has any errors, then an error is returned. If the AST has non-error warnings, they will be returned in the warnings slice. Warnings will be present and valid even if err is non-nil; spec will only be valid if err is nil.

Uses the Options to determine what to validate. Only the SDTS options are recognized at this time.

func ValidateSimulatedInput added in v0.6.1

func ValidateSimulatedInput(spec Spec, md SpecMetadata, params SimulatedInputParams) error

ValidateSimulatedInput generates a lightweight compiler with the spec'd frontend in a special directory (".sim" in the local directory or in the path specified by pathPrefix, if set) and then runs SDTS validation on a variety of parse tree inputs designed to cover all the productions of the grammar at least once.

If running validation with the test compiler succeeds, it and the directory it was generated in are deleted. If it fails, the directory is left in place for inspection.

IRType is required to be set in cgOpts.

valOpts is not required to be set, and if nil will be treated as if it were set to an empty struct.

No binary is generated as part of this, but source is which is then executed. If PreserveBinarySource is set in cgOpts, the source will be left in the .sim directory.

localSource is an optional path to a local copy of ictiobus to use instead of the currently published latest version. This is useful for debugging while developing ictiobus itself.

Types

type AST added in v0.3.0

type AST = syntax.AST

AST is an alias to syntax.AST. Callers should use that type instead as this shortcut will likely eventually be removed.

type CodegenOptions added in v0.5.2

type CodegenOptions struct {
	// DumpPreFormat will dump the generated code before it is formatted.
	DumpPreFormat bool

	// TemplateFiles is a map of template names to a path to a custom template
	// file for that template. If entries are detected under the key of one of
	// the ComponentX constants, the path in it is parsed as a template file and
	// used for outputting the generated code for that component instead of the
	// default embedded template.
	TemplateFiles map[string]string

	// IRType is the fully-qualified type of the intermediate representation in
	// the frontend. This is used to make the Frontend function return a
	// specific type instead of requiring an explicit type instantiation when
	// called.
	IRType string

	// PreserveBinarySource is whether to keep the source files for any
	// generated binary after the binary has been successfully
	// compiled/executed. Normally, these files are removed, but preserving them
	// allows for diagnostics on the generated source.
	PreserveBinarySource bool
}

CodegenOptions is options used for code generation.

type DiagBinParams added in v0.7.0

type DiagBinParams struct {
	// Parser is the built parser of the frontend to be validated.
	Parser parse.Parser

	// HooksPkgDir is the path to the directory containing the hooks package.
	HooksPkgDir string

	// HooksExpr is the expression to use to get the hooks map. This can be a
	// function call, constant name, or var name.
	HooksExpr string

	// PathPrefix is a prefix to apply to the paths of generated source files.
	// If empty, the current directory will be used.
	PathPrefix string

	// Opts are options for code generation. This must be set and its IRType
	// field is required to be set, but all other fields within it are optional.
	Opts CodegenOptions

	// LocalIctiobusSource is used to specify a local path to ictiobus to use
	// instead of the currently published latest version. This is useful for
	// debugging while developing ictiobus itself.
	LocalIctiobusSource string

	// FormatPkgDir is the path to the directory containing the format package.
	// It is completely optional; if not set, the generated main will not
	// contain any pre-formatting code and will assume files are directly ready
	// to be fed into the frontend. Must be set if FormatCall is set.
	FormatPkgDir string

	// FormatCall is the name of a function within the package specified by
	// FormatPkgDir that gets an io.Reader that will run any required
	// pre-formatting on an input io.Reader to get code that can be analyzed by
	// the frontend. Is is optional; if not set, the generated main will not
	// contain any pre-formatting code and will assume files are directly ready
	// to be fed into the frontend. Must be set if FormatPkgDir is set.
	FormatCall string

	// FrontendPkgName is the name of the package to place generated frontend
	// code in.
	FrontendPkgName string

	// BinPath is the path to the binary to create.
	BinPath string
}

DiagBinParams are parameters for the generation of diagnostic binaries.

type GeneratedCodeInfo added in v0.5.2

type GeneratedCodeInfo struct {
	// MainFile is the path to the main executable file, relative to Path.
	MainFile string

	// Path is the location of the root of the generated code.
	Path string
}

GeneratedCodeInfo contains information about the generated code.

type Options added in v0.4.0

type Options struct {
	LexerTrace  bool
	ParserTrace bool
	SDTSTrace   bool
}

Options is options to the FISHI frontend that can be supplied by callers.

type Pattern added in v0.5.0

type Pattern struct {
	// Regex is the regular expression that is used to match the token.
	Regex *regexp.Regexp

	// Action is the action that the lexer should take when it matches the
	// token.
	Action lex.Action

	// Priority is the priority of the pattern. 0 is the lowest priority, and
	// higher numbers will take precedence over lower numbers.
	Priority int
}

Pattern is a lexer pattern that is used to match a token, along with the action that the lexer should take when it matches.

type Results added in v0.4.0

type Results struct {
	AST  *AST
	Tree *parse.Tree
}

Results is the results of attempting to parse a FISHI spec.

func Parse added in v0.5.0

func Parse(r io.Reader, opts *Options) (Results, error)

Parse parses the fishi source code read from the given reader into an AST.

func ParseMarkdown added in v0.5.0

func ParseMarkdown(r io.Reader, opts *Options) (Results, error)

ParseMarkdownFile parses a FISHI spec read from the named Reader containing markdown-formatted text with fishi codeblocks. If opts is nil, it is treated as a pointer to a zero-valued Options struct.

func ParseMarkdownFile added in v0.5.0

func ParseMarkdownFile(filename string, opts *Options) (Results, error)

ParseMarkdownFile parses a FISHI spec in the named file containing markdown text with fishi codeblocks. If opts is nil, it is treated as a pointer to a zero-valued Options struct.

type SDD added in v0.5.0

type SDD struct {
	// Attribute is the reference to the attribute that this SDD will set. If
	// this is RelHead, then the attribute will be set on the head of the
	// relation and it is a synthesized attribute; otherwise, this is an
	// inherited attribute.
	Attribute trans.AttrRef

	// Rule is the grammar haed and production that this SDD is for. This will
	// have exactly one production in it, as opposed to Rule structs stored in
	// the grammar, which contain *all* productions for a given head symbol.
	Rule grammar.Rule

	// Hook is the name of the hook that is called to get the value to set on
	// the attribute.
	Hook string

	// Args is the list of arguments to the hook.
	Args []trans.AttrRef
}

SDD is a Syntax-Directed Definition, that is, a single instruction for a single attribute.

type SimulatedInputParams added in v0.7.0

type SimulatedInputParams struct {
	// Parser is the built parser of the frontend to be validated.
	Parser parse.Parser

	// HooksPkgDir is the path to the directory containing the hooks package.
	HooksPkgDir string

	// HooksExpr is the expression to use to get the hooks map. This can be a
	// function call, constant name, or var name.
	HooksExpr string

	// PathPrefix is a prefix to apply to the paths of generated source files.
	// If empty, the current directory will be used.
	PathPrefix string

	// LocalIctiobusSource is used to specify a local path to ictiobus to use
	// instead of the currently published latest version. This is useful for
	// debugging while developing ictiobus itself.
	LocalIctiobusSource string

	// Opts are options for code generation. This must be set and its IRType
	// field is required to be set, but all other fields within it are optional.
	Opts CodegenOptions

	// ValidationOpts are options for executing the validation itself. This can
	// be nil and if so will be treated as an empty struct.
	ValidationOpts *trans.ValidationOptions

	// WarningHandler is the current warning handler and is queried to see which
	// warning fatal/suppression options should be passed to the simulation
	// binary.
	WarningHandler *WarnHandler

	// QuietMode is whether quiet mode should be enabled in the simulation
	// execution.
	QuietMode bool
}

SimulatedInputParams are parameters for simulating input on a generated parser.

type Spec added in v0.5.0

type Spec struct {
	// Tokens is all of the tokens that are used in the language.
	Tokens []lex.TokenClass

	// Patterns is a map of state names to the lexer patterns that are used
	// while in that state.
	Patterns map[string][]Pattern

	// Grammar is the syntactical specification of the language.
	Grammar grammar.CFG

	// TranslationScheme outlines the Syntax-Directed Translation Scheme for the
	// language by giving the instructions for each attribute.
	TranslationScheme []SDD
}

Spec is a series of statements that together give the specification for a compiler frontend of a language. It is created by processing an AST and checking it for errors with NewSpec.

func (Spec) ClassMap added in v0.5.1

func (spec Spec) ClassMap() map[string]lex.TokenClass

ClassMap is a map of string to token class with that ID.

func (Spec) CreateLexer added in v0.5.1

func (spec Spec) CreateLexer(lazy bool) (lex.Lexer, error)

CreateLexer uses the Tokens and Patterns in the spec to create a new Lexer.

func (Spec) CreateMostRestrictiveParser added in v0.5.2

func (spec Spec) CreateMostRestrictiveParser(allowAmbig bool) (parse.Parser, []Warning, error)

CreateMostRestrictiveParser creates the most restrictive parser possible for the language it represents. They will be tried in this order: LL(1), SLR(1), LALR(1), CLR(1).

AllowAmbig only applies for parser types that can auto-resolve ambiguity, e.g. it does not apply to an LL(k) parser.

func (Spec) CreateParser added in v0.5.1

func (spec Spec) CreateParser(t parse.Algorithm, allowAmbig bool) (parse.Parser, []Warning, error)

CreateParser uses the Grammar in the spec to create a new Parser of the given type. Returns an error if the type is not supported.

func (Spec) CreateSDTS added in v0.5.1

func (spec Spec) CreateSDTS() (trans.SDTS, error)

CreateSDTS uses the TranslationScheme in the spec to create a new SDTS.

func (Spec) ValidateSDTS added in v0.5.1

func (spec Spec) ValidateSDTS(opts trans.ValidationOptions, hooks trans.HookMap) ([]Warning, error)

ValidateSDTS builds the Lexer and SDTS and runs validation on several simulated parse trees to ensure that the SDTS is valid and works.

type SpecMetadata added in v0.5.1

type SpecMetadata struct {
	// Language is name of the language.
	Language string

	// Version is the version of the language.
	Version string

	// InvocationArgs are the arguments
	InvocationArgs string
}

SpecMetadata is data that is not strictly part of the spec but tells info about the language it was generated for and how it was generated.

type WarnHandler added in v0.8.0

type WarnHandler struct {

	// Output is the writer that output is sent to. By default this will be
	// os.Stderr in newly-created WarnHandlers, but this can be changed by
	// setting Output to a different io.Writer.
	Output io.Writer

	// ErrorPrefix is the string to prepend error messages written to Output
	// with.
	ErrorPrefix string

	// WarnPrefix is the string to prepend error messages written to Output
	// with.
	WarnPrefix string
	// contains filtered or unexported fields
}

WarnHandler handles warnings and can be configured by reading slices of names of warnings. The zero-value is not ready to be used; call NewWarnHandler() or NewWarnHandlerFromCLI to create one.

func NewWarnHandler added in v0.8.0

func NewWarnHandler() *WarnHandler

NewWarnHandler creates a new WarnHandler with default settings.

func NewWarnHandlerFromCLI added in v0.8.0

func NewWarnHandlerFromCLI(suppressions, fatals []string) (*WarnHandler, error)

NewWarnHandlerFromCLI creates a new WarnHandler by using the provided options for setting short-codes of WarnTypes, presumably as provided from CLI flags.

func (*WarnHandler) Fatal added in v0.8.0

func (wh *WarnHandler) Fatal(wt WarnType)

Fatal sets the handling for the given warn type to be 'fatal'. Fatal warnings are shown with the Error prefix and will cause handling to return a non-nil error containing a message that the warning is treated as a fatal error.

func (*WarnHandler) Handle added in v0.8.0

func (wh *WarnHandler) Handle(w Warning) (fatal error)

Handle performs whatever handling has been specified for the given warning. It returns a non-nil error if and only if the warning is treated as fatal.

func (*WarnHandler) Handlef added in v0.8.0

func (wh *WarnHandler) Handlef(fmtStr string, w Warning) (fatal error)

Handlef is identical to Handle but allows a custom format string to be supplied. It returns a non-nil error if and only if the warning is treated as fatal.

func (*WarnHandler) HandlingType added in v0.8.0

func (wh *WarnHandler) HandlingType(t WarnType) WarnHandling

HandlingType returns the WarnHandling configured for the given warning type.

If the warning type has no handling defined for it, the default of WarnHandlingOutput will be returned.

func (*WarnHandler) Normal added in v0.8.0

func (wh *WarnHandler) Normal(wt WarnType)

Output sets the handling for the given warn type to be 'normal'. Normal warnings are shown with the Warn prefix.

func (*WarnHandler) Suppressed added in v0.8.0

func (wh *WarnHandler) Suppressed(wt WarnType)

Suppressed sets the handling for a type of warning to be 'suppression'. Suppressed warnings are not shown in output and no further action is taken.

type WarnHandling added in v0.8.0

type WarnHandling int

WarnHandling is a type of handling that should be used for a particular type of Warning.

const (
	// WarnHandlingOutput is the default handling of a warning, and indicates
	// it should be output as normal.
	WarnHandlingOutput WarnHandling = iota

	// WarnHandlingSuppress indicates that a warning should be suppressed and
	// that no action should be taken when it is encountered.
	WarnHandlingSuppress

	// WarnHandlingFatal indicates that a warning should be treated as a fatal
	// error, causing immediate termination of the current procedure and
	// possibly the entire program.
	WarnHandlingFatal
)

func (WarnHandling) String added in v0.8.0

func (wh WarnHandling) String() string

String returns the string represtnation of a WarnHandling.

type WarnType added in v0.5.0

type WarnType int

WarnType is a type of warning that can be generated by FISHI functions.

const (
	WarnNone WarnType = iota
	WarnDuplicateHumanDefs
	WarnMissingHumanDef
	WarnPriorityZero
	WarnUnusedTerminal
	WarnAmbiguousGrammar
	WarnValidation
	WarnValidationArgs
	WarnImportInference
	WarnEFInheritedAttributes
)

func ParseShortWarnType added in v0.8.0

func ParseShortWarnType(s string) (WarnType, error)

ParseShortWarnType parses a WarnType from the given short string. The short string may be a string consisting of the same string as the Short() method returns, or be an integer represented as a string that corresponds to the desired warning.

func WarnTypeAll added in v0.8.0

func WarnTypeAll() []WarnType

WarnTypeAll() returns a slice of all the WarnType constants.

func (WarnType) Short added in v0.8.0

func (wt WarnType) Short() string

Short returns the short-code of a WarnType.

func (WarnType) String added in v0.8.0

func (wt WarnType) String() string

String returns the string representation of a WarnType.

type Warning added in v0.5.0

type Warning struct {
	Type    WarnType
	Message string
}

Warning is a warning that is generated when processing an AST. It is not an error per-se, but fulfills the error interface so that it can be treated as one by caller if desired.

func (Warning) Error added in v0.5.0

func (w Warning) Error() string

Error returns the string representation of the warning.

Directories

Path Synopsis
fe
Package fe contains the frontend for analyzing FISHI code.
Package fe contains the frontend for analyzing FISHI code.
fetoken
Package fetoken contains the token classes used by the frontend of FISHI.
Package fetoken contains the token classes used by the frontend of FISHI.
Package format contains functions for producing a [CodeReader] from a stream that contains markdown files with fishi codeblocks.
Package format contains functions for producing a [CodeReader] from a stream that contains markdown files with fishi codeblocks.
Package syntax provides functions for building up an abstract syntax tree from a FISHI markdown file.
Package syntax provides functions for building up an abstract syntax tree from a FISHI markdown file.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL