Documentation
¶
Index ¶
- Variables
- type LLM
- func (o *LLM) Call(ctx context.Context, prompt string, options ...llms.CallOption) (string, error)
- func (o *LLM) Generate(ctx context.Context, prompts []string, options ...llms.CallOption) ([]*llms.Generation, error)
- func (o *LLM) GeneratePrompt(ctx context.Context, prompts []schema.PromptValue, options ...llms.CallOption) (llms.LLMResult, error)
- func (o *LLM) GetNumTokens(text string) int
- type Option
Constants ¶
This section is empty.
Variables ¶
View Source
var ( // ErrEmptyResponse is returned when the local LLM binary returns an empty response. ErrEmptyResponse = errors.New("no response") // ErrMissingBin is returned when the LOCAL_LLM_BIN environment variable is not set. ErrMissingBin = errors.New("missing the local LLM binary path, set the LOCAL_LLM_BIN environment variable") )
Functions ¶
This section is empty.
Types ¶
type LLM ¶
LLM is a local LLM implementation.
func (*LLM) Generate ¶
func (o *LLM) Generate(ctx context.Context, prompts []string, options ...llms.CallOption) ([]*llms.Generation, error)
Generate generates completions using the local LLM binary.
func (*LLM) GeneratePrompt ¶
func (o *LLM) GeneratePrompt( ctx context.Context, prompts []schema.PromptValue, options ...llms.CallOption, ) (llms.LLMResult, error)
func (*LLM) GetNumTokens ¶
type Option ¶
type Option func(*options)
func WithArgs ¶
WithArgs passes the CLI arguments to the local LLM binary. If not set, then will be used the LOCAL_LLM_ARGS environment variable.
func WithBin ¶
WithBin passes the path to the local LLM binary to the client. If not set, then will be used the LOCAL_LLM_BIN environment variable.
func WithGlobalAsArgs ¶
func WithGlobalAsArgs() Option
WithGlobalAsArgs passes the CLI arguments to the local LLM binary formed from global llms.Options.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
localclient
Package localclient provides a client for local LLMs.
|
Package localclient provides a client for local LLMs. |
Click to show internal directories.
Click to hide internal directories.