fmr

package module
v0.1.1-0...-96a8f70 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 17, 2022 License: Apache-2.0 Imports: 21 Imported by: 7

README

FMR: Functional Meaning Representation & Semantic Parsing Framework

GoDocGo Report Card

Projects that uses FMR

mathsolver

What is semantic parsing?

Semantic parsing is the process of mapping a natural language sentence into an intermediate logical form which is a formal representation of its meaning.

The formal representation should be a detailed representation of the complete meaning of the natural language sentence in a fully formal language that:

  • Has a rich ontology of types, properties, and relations.
  • Supports automated reasoning or execution.

Representation languages

Early semantic parsers used highly domain-specific meaning representation languages, with later systems using more extensible languages like Prolog, lambda calculus, lambda dependancy-based compositional semantics (λ-DCS), SQL, Python, Java, and the Alexa Meaning Representation Language. Some work has used more exotic meaning representations, like query graphs or vector representations.

FMR, a formal meaning representation language

Tasks

  • Grammar checkers
  • Dialogue management
  • Question answering
  • Information extraction
  • Machine translation

What can FMR do, a glance overview

// semantic parsing
"五与5.8的和的平方的1.5次方与two的和减去261.712" =>
nf.math.sub(
  nf.math.sum(
    nf.math.pow(
      nf.math.pow(
        nf.math.sum(
          5,
          nf.math.to_number("5.8")
        ),
        2
      ),
      nf.math.to_number("1.5")
    ),
    2
  ),
  nf.math.to_number("261.712")
); // denotation: 1000

// slot filling
"从上海到天津的机票" => nf.flight("上海", "天津");
"到重庆，明天，从北京" => nf.flight("北京", "重庆");
"到上海去" => nf.flight(null, "上海");

References

Documentation

Index

Constants

View Source 
const DOT = "\u2022" // "\u00B7"

DOT indicates the current position inside a TableState

View Source 
const GammaRule = "\u0263" // "\u0194"

GammaRule is the name of the special "gamma" rule added by the algorithm (this is unicode for 'LATIN SMALL LETTER GAMMA')

Variables

View Source 
var Debug = false

Debug flag

Functions

func Call 

func Call(fn string, args ...interface{}) (interface{}, error)

Call funcs by name fn and args

func NLP 

func NLP() *ling.Pipeline

NLP returns handler for the ling nlp toolkit

Types

type Arg 

type Arg struct {
	Type  string      `json:"type"`
	Value interface{} `json:"value"`
}

Arg is the type of argument for functions

type FMR 

type FMR struct {
	Fn   string `json:"fn,omitempty"`
	Args []*Arg `json:"args,omitempty"`
}

FMR stands for Funtional Meaning Representation

func (*FMR) Equal 

func (f *FMR) Equal(fmr *FMR) bool

Equal func for FMR

func (*FMR) String 

func (f *FMR) String() string

type Frame 

type Frame struct {
	Slots    map[uint64][]*Slot
	Complete bool
}

A Frame is a frame consists of Slots

func (*Frame) String 

func (f *Frame) String() string

type Grammar 

type Grammar struct {
	Name    string            `json:"name"`
	Rules   map[string]*Rule  `json:"rules"`
	Frames  map[string]*Rule  `json:"frames"`
	Regexps map[string]string `json:"regexps"`
	Refined bool              `json:"refined"`
	// contains filtered or unexported fields
}

A Grammar stores a Context-Free Grammar

func GrammarFromFile 

func GrammarFromFile(file string) (*Grammar, error)

GrammarFromFile constructs the Context-Free Grammar from file

func GrammarFromString 

func GrammarFromString(d, name string) (*Grammar, error)

GrammarFromString constructs the Contex-Free Grammar from string d with name

func (*Grammar) EarleyParse 

func (g *Grammar) EarleyParse(text string, starts ...string) (*Parse, error)

EarleyParse parses text for rule <start> at beginning

func (*Grammar) EarleyParseAll 

func (g *Grammar) EarleyParseAll(
	text string, starts ...string) ([]*Parse, error)

EarleyParseAll extracts all submatches in text for rule <start>

func (*Grammar) EarleyParseAllWithContext 

func (g *Grammar) EarleyParseAllWithContext(
	context, text string, starts ...string) ([]*Parse, error)

EarleyParseAllWithContext with context information

func (*Grammar) EarleyParseAny 

func (g *Grammar) EarleyParseAny(
	text string, starts ...string) (*Parse, error)

EarleyParseAny parses text for rule <start> at any position

func (*Grammar) EarleyParseAnyWithContext 

func (g *Grammar) EarleyParseAnyWithContext(
	context, text string, starts ...string) (*Parse, error)

EarleyParseAnyWithContext with context information

func (*Grammar) EarleyParseMaxAll 

func (g *Grammar) EarleyParseMaxAll(
	text string, starts ...string) ([]*Parse, error)

EarleyParseMaxAll extracts all submatches in text for rule <start>

func (*Grammar) EarleyParseMaxAllWithContext 

func (g *Grammar) EarleyParseMaxAllWithContext(
	context, text string, starts ...string) ([]*Parse, error)

EarleyParseMaxAllWithContext with context information

func (*Grammar) EarleyParseWithContext 

func (g *Grammar) EarleyParseWithContext(
	context, text string, starts ...string) (*Parse, error)

EarleyParseWithContext with context information

func (*Grammar) ExtractAll 

func (g *Grammar) ExtractAll(text string, starts ...string) ([]*Node, error)

ExtractAll extracts all parse trees in text for rule <start>

func (*Grammar) ExtractAllWithContext 

func (g *Grammar) ExtractAllWithContext(
	context, text string, starts ...string) ([]*Node, error)

ExtractAllWithContext extracts all parse trees in text for rule <start>

func (*Grammar) ExtractMaxAll 

func (g *Grammar) ExtractMaxAll(
	text string, starts ...string) ([]*Node, error)

ExtractMaxAll extracts all parse trees in text for rule <start>

func (*Grammar) ExtractMaxAllWithContext 

func (g *Grammar) ExtractMaxAllWithContext(
	context, text string, starts ...string) ([]*Node, error)

ExtractMaxAllWithContext extracts all parse trees in text for rule <start>

func (*Grammar) FrameFMR 

func (g *Grammar) FrameFMR(text string) ([]string, error)

FrameFMR parses NL text to FMR

func (*Grammar) FrameFMRWithContext 

func (g *Grammar) FrameFMRWithContext(context, text string) ([]string, error)

FrameFMRWithContext parses NL text to FMR

func (*Grammar) MatchFrames 

func (g *Grammar) MatchFrames(text string) (map[RbKey]*Frame, error)

MatchFrames returns the matched frames for NL text

func (*Grammar) MatchFramesWithContext 

func (g *Grammar) MatchFramesWithContext(
	context, text string) (map[RbKey]*Frame, error)

MatchFramesWithContext returns the matched frames for NL text

func (*Grammar) Parse 

func (g *Grammar) Parse(text string, starts ...string) ([]*Node, error)

Parse returns parse trees for rule <start> at beginning

func (*Grammar) ParseAny 

func (g *Grammar) ParseAny(text string, starts ...string) ([]*Node, error)

ParseAny returns parse trees for rule <start> at any position

func (*Grammar) ParseAnyWithContext 

func (g *Grammar) ParseAnyWithContext(
	context, text string, starts ...string) ([]*Node, error)

ParseAnyWithContext returns parse trees for rule <start> at any position

func (*Grammar) ParseWithContext 

func (g *Grammar) ParseWithContext(
	context, text string, starts ...string) ([]*Node, error)

ParseWithContext returns parse trees for rule <start> at beginning

type Index 

type Index struct {
	Frames map[RbKey]struct{}
	Rules  map[RbKey]struct{}
}

An Index contains two sets for frames' names and rules' names

type Node 

type Node struct {
	Value    *TableState `json:"value"`
	Children []*Node     `json:"children,omitempty"`
	// contains filtered or unexported fields
}

Node is the AST of tree structure

func (*Node) Bracketed 

func (n *Node) Bracketed() string

Bracketed returns the labeled bracket notation of Node

func (*Node) Eval 

func (n *Node) Eval() (interface{}, error)

Eval returns the denotation of Node n

func (*Node) F 

func (n *Node) F() *FMR

F returns the FMR signature of node

func (*Node) NL 

func (n *Node) NL() string

NL returns the normalized text of Node n

func (*Node) OriginalText 

func (n *Node) OriginalText() string

OriginalText returns the original text of Node n

func (*Node) Pos 

func (n *Node) Pos() *Pos

Pos returns the corresponding pos of Node n in original text

func (*Node) Print 

func (n *Node) Print(out io.Writer)

Print this tree to out

func (*Node) Semantic 

func (n *Node) Semantic() (string, error)

Semantic returns the stringified FMR of Node n

func (*Node) String 

func (n *Node) String() string

func (*Node) Term 

func (n *Node) Term() *Term

Term returns the root Term of tree node

func (*Node) Tree 

func (n *Node) Tree() map[string]interface{}

Tree returns the parsed tree of Node n

func (*Node) TreePrint 

func (n *Node) TreePrint()

TreePrint to out

type Parse 

type Parse struct {
	// contains filtered or unexported fields
}

Parse stores a parse chart by grammars

func (*Parse) Boundary 

func (p *Parse) Boundary(finalState *TableState) *Pos

Boundary returns the start, end position in NL for finalState

func (*Parse) GetFinalStates 

func (p *Parse) GetFinalStates() []*TableState

GetFinalStates returns the final states of p

func (*Parse) GetTrees 

func (p *Parse) GetTrees(finalState *TableState) []*Node

GetTrees returns all possible parse results

func (*Parse) String 

func (p *Parse) String() string

func (*Parse) Tag 

func (p *Parse) Tag(finalState *TableState) string

Tag returns the Nonterminal name of finalState

type Pos 

type Pos struct {
	StartByte int `json:"start_byte"`
	EndByte   int `json:"end_byte"`
}

A Pos specifies the start and end positions

type RbKey 

type RbKey struct {
	RuleName string `json:"rule_name"`
	BodyID   uint64 `json:"body_id"`
}

A RbKey identifies a specific RuleBody by name and id

type Rule 

type Rule struct {
	Name string               `json:"-"`
	Body map[uint64]*RuleBody `json:"body,omitempty"`
}

A Rule stores a set of production rules of Name

type RuleBody 

type RuleBody struct {
	Terms []*Term `json:"terms"`
	F     *FMR    `json:"f,omitempty"`
}

A RuleBody is one production rule

func (*RuleBody) Equal 

func (r *RuleBody) Equal(rb *RuleBody) bool

Equal func for RuleBody

type Slot 

type Slot struct {
	Pos
	Trees []*Node
}

A Slot contains the Pos and its corresponding parse trees

type TableColumn 

type TableColumn struct {
	// contains filtered or unexported fields
}

TableColumn is the TableState set

func (*TableColumn) String 

func (tc *TableColumn) String() (out string)

type TableState 

type TableState struct {
	Term  *Term     `json:"term"`
	Rb    *RuleBody `json:"rb,omitempty"`
	Start int       `json:"start"`
	End   int       `json:"end"`
	Dot   int       `json:"dot"`
}

TableState uses Earley's dot notation: given a production X → αβ, the notation X → α • β represents a condition in which α has already been parsed and β is expected.

func (*TableState) Equal 

func (s *TableState) Equal(ts *TableState) bool

Equal func for TableState

func (*TableState) String 

func (ts *TableState) String() string

type Term 

type Term struct {
	Value string      `json:"value"`
	Type  TermType    `json:"type"`
	Meta  interface{} `json:"meta"`
}

A Term is the component of RuleBody

func (*Term) Equal 

func (t *Term) Equal(t1 *Term) bool

Equal func for Term

func (*Term) Key 

func (t *Term) Key() uint64

Key returns a unique key for Term t

type TermType 

type TermType byte

TermType of grammar terms 

const (
	EOF TermType = iota
	Nonterminal
	Terminal
	Any
	List
)

definition of TermTypes

func (TermType) MarshalJSON 

func (r TermType) MarshalJSON() ([]byte, error)

MarshalJSON is generated so TermType satisfies json.Marshaler.

func (TermType) String 

func (i TermType) String() string

func (*TermType) UnmarshalJSON 

func (r *TermType) UnmarshalJSON(data []byte) error

UnmarshalJSON is generated so TermType satisfies json.Unmarshaler.

Source Files

View all Source files

Directories

Path Synopsis
examples
arithmetic
builtin
math

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.