textencoding

package
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 8, 2023 License: BSD-2-Clause Imports: 3 Imported by: 9

Documentation

Index

Constants

View Source 
const (
	// DefaultModel is a sentence-transformers model:
	// It maps sentences & paragraphs to dense vector space and can be used for tasks like clustering or semantic search.
	// Model card: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
	DefaultModel = "sentence-transformers/all-MiniLM-L6-v2"

	// DefaultModelMulti it's like the model before but multilingual:
	// it can be used to map 109 languages to a shared vector space.
	// It works well for finding translation pairs in multiple languages.
	// It doesn't perform so well for assessing the similarity of sentence pairs that are not translations of each other.
	// Model card: https://huggingface.co/sentence-transformers/LaBSE
	DefaultModelMulti = "sentence-transformers/LaBSE"
)

Variables

View Source 
var ErrInputSequenceTooLong = errors.New("input sequence too long")

ErrInputSequenceTooLong means that pre-processing the input text produced a sequence that exceeds the maximum allowed length.

Functions

This section is empty.

Types

type Interface 

type Interface interface {
	// Encode returns the encoded representation of the given example.
	Encode(ctx context.Context, text string, poolingStrategy int) (Response, error)
}

Interface defines the main functions for text encoding task.

type Response 

type Response struct {
	// the encoded representation
	Vector mat.Matrix
}

Response contains the response from text classification.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.