Documentation ¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type PipeTokenizeProcessor ¶
type PipeTokenizeProcessor struct {
// contains filtered or unexported fields
}
PipeTokenizeProcessor 文本分词器
func NewPipeTokenizeProcessor ¶
func NewPipeTokenizeProcessor(storage storage.Persister, language common.LanguageType) *PipeTokenizeProcessor
NewPipeTokenizeProcessor 新建文本分词器.
func (*PipeTokenizeProcessor) InfoTokenize ¶
func (p *PipeTokenizeProcessor) InfoTokenize(pGroup *sync.WaitGroup, input common.PacketChannel, output common.ConcordanceChannel)
InfoTokenize 对中/英文文本进行分词.
func (*PipeTokenizeProcessor) QueryTokenize ¶
func (p *PipeTokenizeProcessor) QueryTokenize(query string, language common.LanguageType, concordance map[string]uint64)
QueryTokenize 对查询语句进行分词.
Click to show internal directories.
Click to hide internal directories.