jieba

package

v1.101.9 Latest Latest Go to latest Published: Jan 13, 2020 License: MIT Imports: 9 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

View Source

const DictDirEnvName = "JIEBA_DICT_DIR"

View Source

const FilterName = "filter_jieba"

This section is empty.

func JiebaFilterConstructor(config map[string]interface{}, cache *registry.Cache) (analysis.TokenFilter, error)

type JiebaFilter struct {
	// contains filtered or unexported fields
}

JiebaFilter implements word segmentation for Chinese. It's a filter so that is can used with other tokenizer (e.g. unicode).

func NewJiebaFilter(dictDir string, searchMode, useHMM bool) *JiebaFilter

func (f *JiebaFilter) Filter(input analysis.TokenStream) analysis.TokenStream

func (f *JiebaFilter) JiebaInstance() *JiebaInstance

JiebaInstance returns the underly JiebaInstance.

type JiebaInstance struct {
	// contains filtered or unexported fields
}

JiebaInstance is a thread-safe *gojieba.Jieba for a given dict directory.

func FindAllJiebaInstances() []*JiebaInstance

FindAllJiebaInstances returns all exists JiebaInstances.

func FindJiebaInstance(dictDir string) *JiebaInstance

FindJiebaInstance returns an exists JiebaInstance for a given dict directory or nil if not found.

func NewJiebaInstance(dictDir string) *JiebaInstance

NewJiebaInstance creates a new JiebaInstance or returns an exists JiebaInstance for a given dict directory.

func (inst *JiebaInstance) DictDir() string

DictDir returns the dict directory.

func (inst *JiebaInstance) Get() (*gojieba.Jieba, func())

Get returns *gojieba.Jieba and a defer function which MUST be called after using.

func (inst *JiebaInstance) LoadTime() (t time.Time, dur time.Duration)

LoadTime returns the load time of data.

func (inst *JiebaInstance) Reload()

Reload reloads data.