tagengine

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 13, 2023 License: MIT Imports: 5 Imported by: 0

README

tagengine

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BasicSanitizer

func BasicSanitizer(s string) string

The basic sanitizer: * lower-case * put spaces around numbers * put slaces around punctuation * collapse multiple spaces

func Tokenize

func Tokenize(
	input string,
	maxNgram int,
) (
	tokens []string,
)

Types

type Match

type Match struct {
	Tag string

	// Confidence is used to sort all matches, and is normalized so the sum of
	// Confidence values for all matches is 1. Confidence is relative to the
	// number of matches and the size of matches in terms of number of tokens.
	Confidence float64 // In the range (0,1].
}

type Rule

type Rule struct {
	// The purpose of a Rule is to attach it's Tag to matching text.
	Tag string

	// Includes is a list of strings that must be found in the input in order to
	// match.
	Includes []string

	// Excludes is a list of strings that can exclude a match for this rule.
	Excludes []string

	// Blocks: If this rule is matched, then it will block matches of any tags
	// listed here.
	Blocks []string

	// The Score encodes the complexity of the Rule. A higher score indicates a
	// more specific match. A Rule more includes, or includes with multiple words
	// should havee a higher Score than a Rule with fewer includes or less
	// complex includes.
	Score int
	// contains filtered or unexported fields
}

func NewRule

func NewRule(tag string) Rule

func (Rule) Block

func (r Rule) Block(l ...string) Rule

func (Rule) Exc

func (r Rule) Exc(l ...string) Rule

func (Rule) Inc

func (r Rule) Inc(l ...string) Rule

type RuleGroup

type RuleGroup struct {
	Tag      string
	Includes [][]string
	Excludes []string
	Blocks   []string
}

A RuleGroup can be converted into a list of rules. Each rule will point to the same tag, and have the same exclude set and blocks.

func NewRuleGroup

func NewRuleGroup(tag string) RuleGroup

func (RuleGroup) Block

func (g RuleGroup) Block(l ...string) RuleGroup

func (RuleGroup) Exc

func (g RuleGroup) Exc(l ...string) RuleGroup

func (RuleGroup) Inc

func (g RuleGroup) Inc(l ...string) RuleGroup

func (RuleGroup) ToList

func (g RuleGroup) ToList() (l []Rule)

type RuleSet

type RuleSet struct {
	// contains filtered or unexported fields
}

func NewRuleSet

func NewRuleSet() *RuleSet

func NewRuleSetFromList

func NewRuleSetFromList(rules []Rule) *RuleSet

func (*RuleSet) Add

func (t *RuleSet) Add(ruleOrGroup ...interface{})

func (*RuleSet) AddRule

func (t *RuleSet) AddRule(rules ...Rule)

func (*RuleSet) AddRuleGroup

func (t *RuleSet) AddRuleGroup(ruleGroups ...RuleGroup)

func (*RuleSet) Match

func (t *RuleSet) Match(input string) []Match

Return a list of matches with confidence. This is useful if you'd like to find the best matching rule out of all the matched rules.

If you just want to find all matching rules, then use MatchRules.

func (*RuleSet) MatchRules

func (t *RuleSet) MatchRules(input string) (rules []*Rule)

MatchRules will return a list of all matching rules. The rules are sorted by the match's Score. The best match will be first.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL