Documentation ¶
Overview ¶
Package llm describes a high level interface to large language models suitable for basic prediction tasks.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Env ¶ added in v0.2.0
Env constructs a configuration map from OS environment variables with the provided prefix, like "llm_"
Types ¶
type Interface ¶
type Interface interface { Release() // Closes the model and releases any associated resources. // Predict calls the provided function with the language model's predicted continuation of the provided input // string. Prediction will stop if the function returns an error, and will eventually stop after the provided // context is cancelled. Predict( ctx context.Context, settings map[string]any, content []string, fn func(Prediction) error, ) (string, error) }
Interface describes the common interface that large language models supported by this package provide.
type Option ¶ added in v0.2.1
type Option struct { // Name is the name of this setting. This is used as the key in the settings map passed to the LLM implementation. Name string `json:"name"` // The name of this setting. // Value is the value of this setting. This is either the default value or the current value, depending on the // context. Value any `json:"value"` // Use describes the purpose of this setting. Use string `json:"use"` // Init identifies options that are only applicable when creating a new instance and not when using its methods. Init bool `json:"init,omitempty"` }
A Option describes a setting that can be used to configure an LLM implementation.
type Prediction ¶
type Prediction interface { // String will return the predicted continuation as a string. String() string }
A Prediction provides a partial prediction of the input continuation from a Predictor.
Directories ¶
Path | Synopsis |
---|---|
cmd
|
|
examples
|
|
slog
Package slog uses either the experimental slog package or the standard slog package depending on the Go version.
|
Package slog uses either the experimental slog package or the standard slog package depending on the Go version. |
Package nats registers an llm implementation that uses NATS to communicate with a worker process.
|
Package nats registers an llm implementation that uses NATS to communicate with a worker process. |
protocol
Package msg describes the protocol used between the NATS client and worker.
|
Package msg describes the protocol used between the NATS client and worker. |
worker
Package worker implements a NATS-based worker for large language models.
|
Package worker implements a NATS-based worker for large language models. |
Click to show internal directories.
Click to hide internal directories.