pos

package

v0.80.2 Latest Latest Go to latest Published: Jan 16, 2023 License: Apache-2.0 Imports: 7 Imported by: 6

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/go-ego/gse

Links

Open Source Insights

Documentation ¶

Overview ¶

Package pos model data The data from https://github.com/fxsjy/jieba

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Dict ¶

type Dict struct {
	Seg gse.Segmenter
	// contains filtered or unexported fields
}

Dict type a dictionary for the word segmentation.

func (*Dict) AddToken ¶

func (d *Dict) AddToken(text string, freq float64, pos ...string) error

AddToken add new text to token

func (*Dict) Freq ¶ added in v0.69.7

func (d *Dict) Freq(key string) (float64, string, bool)

Freq find the word return the frequency and existenced

func (*Dict) Pos ¶

func (d *Dict) Pos(key string) (string, bool)

Pos find the key return the POS and existenced

func (*Dict) RemoveToken ¶ added in v0.69.7

func (d *Dict) RemoveToken(text string) error

RemoveToken remove the token in the dict

type SegPos ¶

type SegPos struct {
	Text, Pos string
}

SegPos type POS struct

type Segmenter ¶

type Segmenter struct {
	// contains filtered or unexported fields
}

Segmenter is a segmentation struct

func (*Segmenter) Cut ¶

func (seg *Segmenter) Cut(text string, hmm ...bool) (result []gse.SegPos)

Cut cuts a text into words. Parameter hmm controls whether to use the HMM

func (*Segmenter) LoadDict ¶

func (seg *Segmenter) LoadDict(fileName ...string) error

LoadDict load dictionary from the file.

func (*Segmenter) Trim ¶

func (seg *Segmenter) Trim(se []gse.SegPos) (re []gse.SegPos)

Trim not space and punct

func (*Segmenter) TrimPunct ¶

func (seg *Segmenter) TrimPunct(se []gse.SegPos) (re []gse.SegPos)

TrimPunct not space and punct

func (*Segmenter) TrimWithPos ¶

func (seg *Segmenter) TrimWithPos(se []gse.SegPos, pos ...string) (re []gse.SegPos)

TrimWithPos trim some pos

func (*Segmenter) WithGse ¶

func (seg *Segmenter) WithGse(segs gse.Segmenter)

WithGse register the gse segmenter

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL