stemmer

package module
v0.0.0-...-8616c93 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 10, 2022 License: MIT Imports: 22 Imported by: 1

README

Documentation

Package stemmer is a set of Snowball language stemmers, implemented as transform.Transformer (see golang.org/x/text/). Implementing transform.Transformer allows use with the x/text ecosystem.

(Stemming is the process of trimming words down to their roots, usually for searching purposes.)

My narrower motivation is to offer stemming in this Unicode text segmenter. To use a stemmer in that package, you will call the Transform method on a Segmenter or Scanner, along the lines of:

import "github.com/clipperhouse/stemmer"

text := []byte("I've got my skis and I am heading to the mountains.")
segmenter := words.NewSegmenter(text)
segmenter.Transform(stemmer.English)

// do stuff with segmenter

This stemmer package is a thin layer over this Snowball package, thanks to Bleve.

Supported languages

Arabic, Danish, Dutch, English, Finnish, French, German, Hungarian, Irish, Italian, Norwegian, Porter, Portuguese, Romanian, Russian, Spanish, Swedish, Tamil, Turkish

Documentation

Overview

Package stemmer set of Snowball language stemmers, implemented as transform.Transformer (from golang.org/x/text/).

Index

Constants

This section is empty.

Variables

Arabic is a Snowball stemmer for Arabic

Danish is a Snowball stemmer for Danish

Dutch is a Snowball stemmer for Dutch

English is a Snowball stemmer for English

Finnish is a Snowball stemmer for Finnish

French is a Snowball stemmer for French

German is a Snowball stemmer for German

Hungarian is a Snowball stemmer for Hungarian

Irish is a Snowball stemmer for Irish

Italian is a Snowball stemmer for Italian

Norwegian is a Snowball stemmer for Norwegian

Porter is a Porter stemmer

Portuguese is a Snowball stemmer for Portuguese

Romanian is a Snowball stemmer for Romanian

Russian is a Snowball stemmer for Russian

Spanish is a Snowball stemmer for Spanish

Swedish is a Snowball stemmer for Swedish

Tamil is a Snowball stemmer for Tamil

Turkish is a Snowball stemmer for Turkish

Functions

This section is empty.

Types

type StemFunc

type StemFunc func(*snowballstem.Env) bool

type Transformer

type Transformer struct {
	// contains filtered or unexported fields
}

func NewTransformer

func NewTransformer(stem StemFunc) *Transformer

func (*Transformer) Reset

func (t *Transformer) Reset()

func (*Transformer) Transform

func (t *Transformer) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL