v1

package
v0.28.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 1, 2024 License: Apache-2.0 Imports: 16 Imported by: 1

Documentation

Overview

Package v1 is a reverse proxy.

It translates gRPC into RESTful JSON APIs.

Index

Constants

This section is empty.

Variables

View Source 
var ChatService_ServiceDesc = grpc.ServiceDesc{
	ServiceName: "llmoperator.chat.server.v1.ChatService",
	HandlerType: (*ChatServiceServer)(nil),
	Methods: []grpc.MethodDesc{
		{
			MethodName: "CreateChatCompletion",
			Handler:    _ChatService_CreateChatCompletion_Handler,
		},
	},
	Streams:  []grpc.StreamDesc{},
	Metadata: "api/v1/inference_server.proto",
}

ChatService_ServiceDesc is the grpc.ServiceDesc for ChatService service. It's only intended for direct use with grpc.RegisterService, and not to be introspected or modified (even as a copy)

View Source 
var File_api_v1_inference_engine_proto protoreflect.FileDescriptor
View Source 
var File_api_v1_inference_server_proto protoreflect.FileDescriptor
View Source 
var InferenceEngineInternalService_ServiceDesc = grpc.ServiceDesc{
	ServiceName: "llmoperator.inference_engine.v1.InferenceEngineInternalService",
	HandlerType: (*InferenceEngineInternalServiceServer)(nil),
	Methods:     []grpc.MethodDesc{},
	Streams:     []grpc.StreamDesc{},
	Metadata:    "api/v1/inference_engine.proto",
}

InferenceEngineInternalService_ServiceDesc is the grpc.ServiceDesc for InferenceEngineInternalService service. It's only intended for direct use with grpc.RegisterService, and not to be introspected or modified (even as a copy)

Functions

func RegisterChatServiceHandler 

func RegisterChatServiceHandler(ctx context.Context, mux *runtime.ServeMux, conn *grpc.ClientConn) error

RegisterChatServiceHandler registers the http handlers for service ChatService to "mux". The handlers forward requests to the grpc endpoint over "conn".

func RegisterChatServiceHandlerClient 

func RegisterChatServiceHandlerClient(ctx context.Context, mux *runtime.ServeMux, client ChatServiceClient) error

RegisterChatServiceHandlerClient registers the http handlers for service ChatService to "mux". The handlers forward requests to the grpc endpoint over the given implementation of "ChatServiceClient". Note: the gRPC framework executes interceptors within the gRPC handler. If the passed in "ChatServiceClient" doesn't go through the normal gRPC flow (creating a gRPC client etc.) then it will be up to the passed in "ChatServiceClient" to call the correct interceptors.

func RegisterChatServiceHandlerFromEndpoint 

func RegisterChatServiceHandlerFromEndpoint(ctx context.Context, mux *runtime.ServeMux, endpoint string, opts []grpc.DialOption) (err error)

RegisterChatServiceHandlerFromEndpoint is same as RegisterChatServiceHandler but automatically dials to "endpoint" and closes the connection when "ctx" gets done.

func RegisterChatServiceHandlerServer 

func RegisterChatServiceHandlerServer(ctx context.Context, mux *runtime.ServeMux, server ChatServiceServer) error

RegisterChatServiceHandlerServer registers the http handlers for service ChatService to "mux". UnaryRPC :call ChatServiceServer directly. StreamingRPC :currently unsupported pending https://github.com/grpc/grpc-go/issues/906. Note that using this registration option will cause many gRPC library features to stop working. Consider using RegisterChatServiceHandlerFromEndpoint instead.

func RegisterChatServiceServer 

func RegisterChatServiceServer(s grpc.ServiceRegistrar, srv ChatServiceServer)

func RegisterInferenceEngineInternalServiceServer 

func RegisterInferenceEngineInternalServiceServer(s grpc.ServiceRegistrar, srv InferenceEngineInternalServiceServer)

Types

type ChatCompletion 

type ChatCompletion struct {
	Id                string                   `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
	Choices           []*ChatCompletion_Choice `protobuf:"bytes,2,rep,name=choices,proto3" json:"choices,omitempty"`
	Created           int32                    `protobuf:"varint,3,opt,name=created,proto3" json:"created,omitempty"`
	Model             string                   `protobuf:"bytes,4,opt,name=model,proto3" json:"model,omitempty"`
	SystemFingerprint string                   `protobuf:"bytes,5,opt,name=system_fingerprint,json=systemFingerprint,proto3" json:"system_fingerprint,omitempty"`
	Object            string                   `protobuf:"bytes,6,opt,name=object,proto3" json:"object,omitempty"`
	Usage             *ChatCompletion_Usage    `protobuf:"bytes,7,opt,name=usage,proto3" json:"usage,omitempty"`
	// contains filtered or unexported fields
}

func (*ChatCompletion) Descriptor deprecated 

func (*ChatCompletion) Descriptor() ([]byte, []int)

Deprecated: Use ChatCompletion.ProtoReflect.Descriptor instead.

func (*ChatCompletion) GetChoices 

func (x *ChatCompletion) GetChoices() []*ChatCompletion_Choice

func (*ChatCompletion) GetCreated 

func (x *ChatCompletion) GetCreated() int32

func (*ChatCompletion) GetId 

func (x *ChatCompletion) GetId() string

func (*ChatCompletion) GetModel 

func (x *ChatCompletion) GetModel() string

func (*ChatCompletion) GetObject 

func (x *ChatCompletion) GetObject() string

func (*ChatCompletion) GetSystemFingerprint 

func (x *ChatCompletion) GetSystemFingerprint() string

func (*ChatCompletion) GetUsage 

func (x *ChatCompletion) GetUsage() *ChatCompletion_Usage

func (*ChatCompletion) ProtoMessage 

func (*ChatCompletion) ProtoMessage()

func (*ChatCompletion) ProtoReflect 

func (x *ChatCompletion) ProtoReflect() protoreflect.Message

func (*ChatCompletion) Reset 

func (x *ChatCompletion) Reset()

func (*ChatCompletion) String 

func (x *ChatCompletion) String() string

type ChatCompletion_Choice 

type ChatCompletion_Choice struct {
	FinishReason string                          `protobuf:"bytes,1,opt,name=finish_reason,json=finishReason,proto3" json:"finish_reason,omitempty"`
	Index        int32                           `protobuf:"varint,2,opt,name=index,proto3" json:"index,omitempty"`
	Message      *ChatCompletion_Choice_Message  `protobuf:"bytes,3,opt,name=message,proto3" json:"message,omitempty"`
	Logprobs     *ChatCompletion_Choice_Logprobs `protobuf:"bytes,4,opt,name=logprobs,proto3" json:"logprobs,omitempty"`
	// contains filtered or unexported fields
}

func (*ChatCompletion_Choice) Descriptor deprecated 

func (*ChatCompletion_Choice) Descriptor() ([]byte, []int)

Deprecated: Use ChatCompletion_Choice.ProtoReflect.Descriptor instead.

func (*ChatCompletion_Choice) GetFinishReason 

func (x *ChatCompletion_Choice) GetFinishReason() string

func (*ChatCompletion_Choice) GetIndex 

func (x *ChatCompletion_Choice) GetIndex() int32

func (*ChatCompletion_Choice) GetLogprobs 

func (x *ChatCompletion_Choice) GetLogprobs() *ChatCompletion_Choice_Logprobs

func (*ChatCompletion_Choice) GetMessage 

func (x *ChatCompletion_Choice) GetMessage() *ChatCompletion_Choice_Message

func (*ChatCompletion_Choice) ProtoMessage 

func (*ChatCompletion_Choice) ProtoMessage()

func (*ChatCompletion_Choice) ProtoReflect 

func (x *ChatCompletion_Choice) ProtoReflect() protoreflect.Message

func (*ChatCompletion_Choice) Reset 

func (x *ChatCompletion_Choice) Reset()

func (*ChatCompletion_Choice) String 

func (x *ChatCompletion_Choice) String() string

type ChatCompletion_Choice_Logprobs 

type ChatCompletion_Choice_Logprobs struct {
	Content []*ChatCompletion_Choice_Logprobs_Content `protobuf:"bytes,1,rep,name=content,proto3" json:"content,omitempty"`
	// contains filtered or unexported fields
}

func (*ChatCompletion_Choice_Logprobs) Descriptor deprecated 

func (*ChatCompletion_Choice_Logprobs) Descriptor() ([]byte, []int)

Deprecated: Use ChatCompletion_Choice_Logprobs.ProtoReflect.Descriptor instead.

func (*ChatCompletion_Choice_Logprobs) GetContent 

func (x *ChatCompletion_Choice_Logprobs) GetContent() []*ChatCompletion_Choice_Logprobs_Content

func (*ChatCompletion_Choice_Logprobs) ProtoMessage 

func (*ChatCompletion_Choice_Logprobs) ProtoMessage()

func (*ChatCompletion_Choice_Logprobs) ProtoReflect 

func (x *ChatCompletion_Choice_Logprobs) ProtoReflect() protoreflect.Message

func (*ChatCompletion_Choice_Logprobs) Reset 

func (x *ChatCompletion_Choice_Logprobs) Reset()

func (*ChatCompletion_Choice_Logprobs) String 

func (x *ChatCompletion_Choice_Logprobs) String() string

type ChatCompletion_Choice_Logprobs_Content 

type ChatCompletion_Choice_Logprobs_Content struct {
	Token   string  `protobuf:"bytes,1,opt,name=token,proto3" json:"token,omitempty"`
	Logprob float64 `protobuf:"fixed64,2,opt,name=logprob,proto3" json:"logprob,omitempty"`
	// A list of integers representing the UTF-8 bytes representation of the token.
	Bytes       []byte                                              `protobuf:"bytes,3,opt,name=bytes,proto3" json:"bytes,omitempty"`
	TopLogprobs *ChatCompletion_Choice_Logprobs_Content_TopLogprobs `protobuf:"bytes,4,opt,name=top_logprobs,json=topLogprobs,proto3" json:"top_logprobs,omitempty"`
	// contains filtered or unexported fields
}

func (*ChatCompletion_Choice_Logprobs_Content) Descriptor deprecated 

func (*ChatCompletion_Choice_Logprobs_Content) Descriptor() ([]byte, []int)

Deprecated: Use ChatCompletion_Choice_Logprobs_Content.ProtoReflect.Descriptor instead.

func (*ChatCompletion_Choice_Logprobs_Content) GetBytes 

func (x *ChatCompletion_Choice_Logprobs_Content) GetBytes() []byte

func (*ChatCompletion_Choice_Logprobs_Content) GetLogprob 

func (x *ChatCompletion_Choice_Logprobs_Content) GetLogprob() float64

func (*ChatCompletion_Choice_Logprobs_Content) GetToken 

func (x *ChatCompletion_Choice_Logprobs_Content) GetToken() string

func (*ChatCompletion_Choice_Logprobs_Content) GetTopLogprobs 

func (x *ChatCompletion_Choice_Logprobs_Content) GetTopLogprobs() *ChatCompletion_Choice_Logprobs_Content_TopLogprobs

func (*ChatCompletion_Choice_Logprobs_Content) ProtoMessage 

func (*ChatCompletion_Choice_Logprobs_Content) ProtoMessage()

func (*ChatCompletion_Choice_Logprobs_Content) ProtoReflect 

func (x *ChatCompletion_Choice_Logprobs_Content) ProtoReflect() protoreflect.Message

func (*ChatCompletion_Choice_Logprobs_Content) Reset 

func (x *ChatCompletion_Choice_Logprobs_Content) Reset()

func (*ChatCompletion_Choice_Logprobs_Content) String 

func (x *ChatCompletion_Choice_Logprobs_Content) String() string

type ChatCompletion_Choice_Logprobs_Content_TopLogprobs 

type ChatCompletion_Choice_Logprobs_Content_TopLogprobs struct {
	Token   string  `protobuf:"bytes,1,opt,name=token,proto3" json:"token,omitempty"`
	Logprob float64 `protobuf:"fixed64,2,opt,name=logprob,proto3" json:"logprob,omitempty"`
	Bytes   []byte  `protobuf:"bytes,3,opt,name=bytes,proto3" json:"bytes,omitempty"`
	// contains filtered or unexported fields
}

func (*ChatCompletion_Choice_Logprobs_Content_TopLogprobs) Descriptor deprecated 

func (*ChatCompletion_Choice_Logprobs_Content_TopLogprobs) Descriptor() ([]byte, []int)

Deprecated: Use ChatCompletion_Choice_Logprobs_Content_TopLogprobs.ProtoReflect.Descriptor instead.

func (*ChatCompletion_Choice_Logprobs_Content_TopLogprobs) GetBytes 

func (x *ChatCompletion_Choice_Logprobs_Content_TopLogprobs) GetBytes() []byte

func (*ChatCompletion_Choice_Logprobs_Content_TopLogprobs) GetLogprob 

func (x *ChatCompletion_Choice_Logprobs_Content_TopLogprobs) GetLogprob() float64

func (*ChatCompletion_Choice_Logprobs_Content_TopLogprobs) GetToken 

func (x *ChatCompletion_Choice_Logprobs_Content_TopLogprobs) GetToken() string

func (*ChatCompletion_Choice_Logprobs_Content_TopLogprobs) ProtoMessage 

func (*ChatCompletion_Choice_Logprobs_Content_TopLogprobs) ProtoMessage()

func (*ChatCompletion_Choice_Logprobs_Content_TopLogprobs) ProtoReflect 

func (x *ChatCompletion_Choice_Logprobs_Content_TopLogprobs) ProtoReflect() protoreflect.Message

func (*ChatCompletion_Choice_Logprobs_Content_TopLogprobs) Reset 

func (x *ChatCompletion_Choice_Logprobs_Content_TopLogprobs) Reset()

func (*ChatCompletion_Choice_Logprobs_Content_TopLogprobs) String 

func (x *ChatCompletion_Choice_Logprobs_Content_TopLogprobs) String() string

type ChatCompletion_Choice_Message 

type ChatCompletion_Choice_Message struct {
	Content   string                                    `protobuf:"bytes,1,opt,name=content,proto3" json:"content,omitempty"`
	ToolCalls []*ChatCompletion_Choice_Message_ToolCall `protobuf:"bytes,2,rep,name=tool_calls,json=toolCalls,proto3" json:"tool_calls,omitempty"`
	Role      string                                    `protobuf:"bytes,3,opt,name=role,proto3" json:"role,omitempty"`
	// contains filtered or unexported fields
}

func (*ChatCompletion_Choice_Message) Descriptor deprecated 

func (*ChatCompletion_Choice_Message) Descriptor() ([]byte, []int)

Deprecated: Use ChatCompletion_Choice_Message.ProtoReflect.Descriptor instead.

func (*ChatCompletion_Choice_Message) GetContent 

func (x *ChatCompletion_Choice_Message) GetContent() string

func (*ChatCompletion_Choice_Message) GetRole 

func (x *ChatCompletion_Choice_Message) GetRole() string

func (*ChatCompletion_Choice_Message) GetToolCalls 

func (x *ChatCompletion_Choice_Message) GetToolCalls() []*ChatCompletion_Choice_Message_ToolCall

func (*ChatCompletion_Choice_Message) ProtoMessage 

func (*ChatCompletion_Choice_Message) ProtoMessage()

func (*ChatCompletion_Choice_Message) ProtoReflect 

func (x *ChatCompletion_Choice_Message) ProtoReflect() protoreflect.Message

func (*ChatCompletion_Choice_Message) Reset 

func (x *ChatCompletion_Choice_Message) Reset()

func (*ChatCompletion_Choice_Message) String 

func (x *ChatCompletion_Choice_Message) String() string

type ChatCompletion_Choice_Message_ToolCall 

type ChatCompletion_Choice_Message_ToolCall struct {
	Id       string                                           `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
	Type     string                                           `protobuf:"bytes,2,opt,name=type,proto3" json:"type,omitempty"`
	Function *ChatCompletion_Choice_Message_ToolCall_Function `protobuf:"bytes,3,opt,name=function,proto3" json:"function,omitempty"`
	// contains filtered or unexported fields
}

func (*ChatCompletion_Choice_Message_ToolCall) Descriptor deprecated 

func (*ChatCompletion_Choice_Message_ToolCall) Descriptor() ([]byte, []int)

Deprecated: Use ChatCompletion_Choice_Message_ToolCall.ProtoReflect.Descriptor instead.

func (*ChatCompletion_Choice_Message_ToolCall) GetFunction 

func (x *ChatCompletion_Choice_Message_ToolCall) GetFunction() *ChatCompletion_Choice_Message_ToolCall_Function

func (*ChatCompletion_Choice_Message_ToolCall) GetId 

func (x *ChatCompletion_Choice_Message_ToolCall) GetId() string

func (*ChatCompletion_Choice_Message_ToolCall) GetType 

func (x *ChatCompletion_Choice_Message_ToolCall) GetType() string

func (*ChatCompletion_Choice_Message_ToolCall) ProtoMessage 

func (*ChatCompletion_Choice_Message_ToolCall) ProtoMessage()

func (*ChatCompletion_Choice_Message_ToolCall) ProtoReflect 

func (x *ChatCompletion_Choice_Message_ToolCall) ProtoReflect() protoreflect.Message

func (*ChatCompletion_Choice_Message_ToolCall) Reset 

func (x *ChatCompletion_Choice_Message_ToolCall) Reset()

func (*ChatCompletion_Choice_Message_ToolCall) String 

func (x *ChatCompletion_Choice_Message_ToolCall) String() string

type ChatCompletion_Choice_Message_ToolCall_Function 

type ChatCompletion_Choice_Message_ToolCall_Function struct {
	Name      string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
	Arguments string `protobuf:"bytes,2,opt,name=arguments,proto3" json:"arguments,omitempty"`
	// contains filtered or unexported fields
}

func (*ChatCompletion_Choice_Message_ToolCall_Function) Descriptor deprecated 

func (*ChatCompletion_Choice_Message_ToolCall_Function) Descriptor() ([]byte, []int)

Deprecated: Use ChatCompletion_Choice_Message_ToolCall_Function.ProtoReflect.Descriptor instead.

func (*ChatCompletion_Choice_Message_ToolCall_Function) GetArguments 

func (x *ChatCompletion_Choice_Message_ToolCall_Function) GetArguments() string

func (*ChatCompletion_Choice_Message_ToolCall_Function) GetName 

func (x *ChatCompletion_Choice_Message_ToolCall_Function) GetName() string

func (*ChatCompletion_Choice_Message_ToolCall_Function) ProtoMessage 

func (*ChatCompletion_Choice_Message_ToolCall_Function) ProtoMessage()

func (*ChatCompletion_Choice_Message_ToolCall_Function) ProtoReflect 

func (x *ChatCompletion_Choice_Message_ToolCall_Function) ProtoReflect() protoreflect.Message

func (*ChatCompletion_Choice_Message_ToolCall_Function) Reset 

func (x *ChatCompletion_Choice_Message_ToolCall_Function) Reset()

func (*ChatCompletion_Choice_Message_ToolCall_Function) String 

func (x *ChatCompletion_Choice_Message_ToolCall_Function) String() string

type ChatCompletion_Usage 

type ChatCompletion_Usage struct {
	CompletionTokens int32 `protobuf:"varint,1,opt,name=completion_tokens,json=completionTokens,proto3" json:"completion_tokens,omitempty"`
	PromptTokens     int32 `protobuf:"varint,2,opt,name=prompt_tokens,json=promptTokens,proto3" json:"prompt_tokens,omitempty"`
	TotalTokens      int32 `protobuf:"varint,3,opt,name=total_tokens,json=totalTokens,proto3" json:"total_tokens,omitempty"`
	// contains filtered or unexported fields
}

func (*ChatCompletion_Usage) Descriptor deprecated 

func (*ChatCompletion_Usage) Descriptor() ([]byte, []int)

Deprecated: Use ChatCompletion_Usage.ProtoReflect.Descriptor instead.

func (*ChatCompletion_Usage) GetCompletionTokens 

func (x *ChatCompletion_Usage) GetCompletionTokens() int32

func (*ChatCompletion_Usage) GetPromptTokens 

func (x *ChatCompletion_Usage) GetPromptTokens() int32

func (*ChatCompletion_Usage) GetTotalTokens 

func (x *ChatCompletion_Usage) GetTotalTokens() int32

func (*ChatCompletion_Usage) ProtoMessage 

func (*ChatCompletion_Usage) ProtoMessage()

func (*ChatCompletion_Usage) ProtoReflect 

func (x *ChatCompletion_Usage) ProtoReflect() protoreflect.Message

func (*ChatCompletion_Usage) Reset 

func (x *ChatCompletion_Usage) Reset()

func (*ChatCompletion_Usage) String 

func (x *ChatCompletion_Usage) String() string

type ChatServiceClient 

type ChatServiceClient interface {
	CreateChatCompletion(ctx context.Context, in *CreateChatCompletionRequest, opts ...grpc.CallOption) (*ChatCompletion, error)
}

ChatServiceClient is the client API for ChatService service.

For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.

func NewChatServiceClient 

func NewChatServiceClient(cc grpc.ClientConnInterface) ChatServiceClient

type ChatServiceServer 

type ChatServiceServer interface {
	CreateChatCompletion(context.Context, *CreateChatCompletionRequest) (*ChatCompletion, error)
	// contains filtered or unexported methods
}

ChatServiceServer is the server API for ChatService service. All implementations must embed UnimplementedChatServiceServer for forward compatibility

type CreateChatCompletionRequest 

type CreateChatCompletionRequest struct {

	// TODO(kenji): Revisit this.
	Messages         []*CreateChatCompletionRequest_Message      `protobuf:"bytes,1,rep,name=messages,proto3" json:"messages,omitempty"`
	Model            string                                      `protobuf:"bytes,2,opt,name=model,proto3" json:"model,omitempty"`
	FrequencyPenalty float64                                     `protobuf:"fixed64,3,opt,name=frequency_penalty,json=frequencyPenalty,proto3" json:"frequency_penalty,omitempty"`
	LogitBias        map[string]float64                          `` /* 178-byte string literal not displayed */
	Logprobs         bool                                        `protobuf:"varint,5,opt,name=logprobs,proto3" json:"logprobs,omitempty"`
	TopLogprobs      int32                                       `protobuf:"varint,6,opt,name=top_logprobs,json=topLogprobs,proto3" json:"top_logprobs,omitempty"`
	MaxTokens        int32                                       `protobuf:"varint,7,opt,name=max_tokens,json=maxTokens,proto3" json:"max_tokens,omitempty"`
	N                int32                                       `protobuf:"varint,8,opt,name=n,proto3" json:"n,omitempty"`
	PresencePenalty  float64                                     `protobuf:"fixed64,9,opt,name=presence_penalty,json=presencePenalty,proto3" json:"presence_penalty,omitempty"`
	ResponseFormat   *CreateChatCompletionRequest_ResponseFormat `protobuf:"bytes,10,opt,name=response_format,json=responseFormat,proto3" json:"response_format,omitempty"`
	Seed             int32                                       `protobuf:"varint,11,opt,name=seed,proto3" json:"seed,omitempty"`
	// string / arrary/ null
	Stop        string                              `protobuf:"bytes,12,opt,name=stop,proto3" json:"stop,omitempty"`
	Stream      bool                                `protobuf:"varint,13,opt,name=stream,proto3" json:"stream,omitempty"`
	Temperature float64                             `protobuf:"fixed64,14,opt,name=temperature,proto3" json:"temperature,omitempty"`
	TopP        float64                             `protobuf:"fixed64,15,opt,name=top_p,json=topP,proto3" json:"top_p,omitempty"`
	Tools       []*CreateChatCompletionRequest_Tool `protobuf:"bytes,16,rep,name=tools,proto3" json:"tools,omitempty"`
	// string or object
	ToolChocise bool   `protobuf:"varint,17,opt,name=tool_chocise,json=toolChocise,proto3" json:"tool_chocise,omitempty"`
	User        string `protobuf:"bytes,18,opt,name=user,proto3" json:"user,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateChatCompletionRequest) Descriptor deprecated 

func (*CreateChatCompletionRequest) Descriptor() ([]byte, []int)

Deprecated: Use CreateChatCompletionRequest.ProtoReflect.Descriptor instead.

func (*CreateChatCompletionRequest) GetFrequencyPenalty 

func (x *CreateChatCompletionRequest) GetFrequencyPenalty() float64

func (*CreateChatCompletionRequest) GetLogitBias 

func (x *CreateChatCompletionRequest) GetLogitBias() map[string]float64

func (*CreateChatCompletionRequest) GetLogprobs 

func (x *CreateChatCompletionRequest) GetLogprobs() bool

func (*CreateChatCompletionRequest) GetMaxTokens 

func (x *CreateChatCompletionRequest) GetMaxTokens() int32

func (*CreateChatCompletionRequest) GetMessages 

func (x *CreateChatCompletionRequest) GetMessages() []*CreateChatCompletionRequest_Message

func (*CreateChatCompletionRequest) GetModel 

func (x *CreateChatCompletionRequest) GetModel() string

func (*CreateChatCompletionRequest) GetN 

func (x *CreateChatCompletionRequest) GetN() int32

func (*CreateChatCompletionRequest) GetPresencePenalty 

func (x *CreateChatCompletionRequest) GetPresencePenalty() float64

func (*CreateChatCompletionRequest) GetResponseFormat 

func (x *CreateChatCompletionRequest) GetResponseFormat() *CreateChatCompletionRequest_ResponseFormat

func (*CreateChatCompletionRequest) GetSeed 

func (x *CreateChatCompletionRequest) GetSeed() int32

func (*CreateChatCompletionRequest) GetStop 

func (x *CreateChatCompletionRequest) GetStop() string

func (*CreateChatCompletionRequest) GetStream 

func (x *CreateChatCompletionRequest) GetStream() bool

func (*CreateChatCompletionRequest) GetTemperature 

func (x *CreateChatCompletionRequest) GetTemperature() float64

func (*CreateChatCompletionRequest) GetToolChocise 

func (x *CreateChatCompletionRequest) GetToolChocise() bool

func (*CreateChatCompletionRequest) GetTools 

func (x *CreateChatCompletionRequest) GetTools() []*CreateChatCompletionRequest_Tool

func (*CreateChatCompletionRequest) GetTopLogprobs 

func (x *CreateChatCompletionRequest) GetTopLogprobs() int32

func (*CreateChatCompletionRequest) GetTopP 

func (x *CreateChatCompletionRequest) GetTopP() float64

func (*CreateChatCompletionRequest) GetUser 

func (x *CreateChatCompletionRequest) GetUser() string

func (*CreateChatCompletionRequest) ProtoMessage 

func (*CreateChatCompletionRequest) ProtoMessage()

func (*CreateChatCompletionRequest) ProtoReflect 

func (x *CreateChatCompletionRequest) ProtoReflect() protoreflect.Message

func (*CreateChatCompletionRequest) Reset 

func (x *CreateChatCompletionRequest) Reset()

func (*CreateChatCompletionRequest) String 

func (x *CreateChatCompletionRequest) String() string

type CreateChatCompletionRequest_Message 

type CreateChatCompletionRequest_Message struct {

	// The type of content varies. For eample, it is string or array for UserMesasge.
	Content   string                                          `protobuf:"bytes,1,opt,name=content,proto3" json:"content,omitempty"`
	Role      string                                          `protobuf:"bytes,2,opt,name=role,proto3" json:"role,omitempty"`
	Name      string                                          `protobuf:"bytes,3,opt,name=name,proto3" json:"name,omitempty"`
	ToolCalls []*CreateChatCompletionRequest_Message_ToolCall `protobuf:"bytes,4,rep,name=tool_calls,json=toolCalls,proto3" json:"tool_calls,omitempty"`
	// contains filtered or unexported fields
}

Message has fields for system message, user message, assistant message, and tool message.

func (*CreateChatCompletionRequest_Message) Descriptor deprecated 

func (*CreateChatCompletionRequest_Message) Descriptor() ([]byte, []int)

Deprecated: Use CreateChatCompletionRequest_Message.ProtoReflect.Descriptor instead.

func (*CreateChatCompletionRequest_Message) GetContent 

func (x *CreateChatCompletionRequest_Message) GetContent() string

func (*CreateChatCompletionRequest_Message) GetName 

func (x *CreateChatCompletionRequest_Message) GetName() string

func (*CreateChatCompletionRequest_Message) GetRole 

func (x *CreateChatCompletionRequest_Message) GetRole() string

func (*CreateChatCompletionRequest_Message) GetToolCalls 

func (x *CreateChatCompletionRequest_Message) GetToolCalls() []*CreateChatCompletionRequest_Message_ToolCall

func (*CreateChatCompletionRequest_Message) ProtoMessage 

func (*CreateChatCompletionRequest_Message) ProtoMessage()

func (*CreateChatCompletionRequest_Message) ProtoReflect 

func (x *CreateChatCompletionRequest_Message) ProtoReflect() protoreflect.Message

func (*CreateChatCompletionRequest_Message) Reset 

func (x *CreateChatCompletionRequest_Message) Reset()

func (*CreateChatCompletionRequest_Message) String 

func (x *CreateChatCompletionRequest_Message) String() string

type CreateChatCompletionRequest_Message_ToolCall 

type CreateChatCompletionRequest_Message_ToolCall struct {
	Id       string                                                 `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
	Type     string                                                 `protobuf:"bytes,2,opt,name=type,proto3" json:"type,omitempty"`
	Function *CreateChatCompletionRequest_Message_ToolCall_Function `protobuf:"bytes,3,opt,name=function,proto3" json:"function,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateChatCompletionRequest_Message_ToolCall) Descriptor deprecated 

func (*CreateChatCompletionRequest_Message_ToolCall) Descriptor() ([]byte, []int)

Deprecated: Use CreateChatCompletionRequest_Message_ToolCall.ProtoReflect.Descriptor instead.

func (*CreateChatCompletionRequest_Message_ToolCall) GetFunction 

func (x *CreateChatCompletionRequest_Message_ToolCall) GetFunction() *CreateChatCompletionRequest_Message_ToolCall_Function

func (*CreateChatCompletionRequest_Message_ToolCall) GetId 

func (x *CreateChatCompletionRequest_Message_ToolCall) GetId() string

func (*CreateChatCompletionRequest_Message_ToolCall) GetType 

func (x *CreateChatCompletionRequest_Message_ToolCall) GetType() string

func (*CreateChatCompletionRequest_Message_ToolCall) ProtoMessage 

func (*CreateChatCompletionRequest_Message_ToolCall) ProtoMessage()

func (*CreateChatCompletionRequest_Message_ToolCall) ProtoReflect 

func (x *CreateChatCompletionRequest_Message_ToolCall) ProtoReflect() protoreflect.Message

func (*CreateChatCompletionRequest_Message_ToolCall) Reset 

func (x *CreateChatCompletionRequest_Message_ToolCall) Reset()

func (*CreateChatCompletionRequest_Message_ToolCall) String 

func (x *CreateChatCompletionRequest_Message_ToolCall) String() string

type CreateChatCompletionRequest_Message_ToolCall_Function 

type CreateChatCompletionRequest_Message_ToolCall_Function struct {
	Name      string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
	Arguments string `protobuf:"bytes,2,opt,name=arguments,proto3" json:"arguments,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateChatCompletionRequest_Message_ToolCall_Function) Descriptor deprecated 

func (*CreateChatCompletionRequest_Message_ToolCall_Function) Descriptor() ([]byte, []int)

Deprecated: Use CreateChatCompletionRequest_Message_ToolCall_Function.ProtoReflect.Descriptor instead.

func (*CreateChatCompletionRequest_Message_ToolCall_Function) GetArguments 

func (x *CreateChatCompletionRequest_Message_ToolCall_Function) GetArguments() string

func (*CreateChatCompletionRequest_Message_ToolCall_Function) GetName 

func (x *CreateChatCompletionRequest_Message_ToolCall_Function) GetName() string

func (*CreateChatCompletionRequest_Message_ToolCall_Function) ProtoMessage 

func (*CreateChatCompletionRequest_Message_ToolCall_Function) ProtoMessage()

func (*CreateChatCompletionRequest_Message_ToolCall_Function) ProtoReflect 

func (x *CreateChatCompletionRequest_Message_ToolCall_Function) ProtoReflect() protoreflect.Message

func (*CreateChatCompletionRequest_Message_ToolCall_Function) Reset 

func (x *CreateChatCompletionRequest_Message_ToolCall_Function) Reset()

func (*CreateChatCompletionRequest_Message_ToolCall_Function) String 

func (x *CreateChatCompletionRequest_Message_ToolCall_Function) String() string

type CreateChatCompletionRequest_ResponseFormat 

type CreateChatCompletionRequest_ResponseFormat struct {
	Type string `protobuf:"bytes,1,opt,name=type,proto3" json:"type,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateChatCompletionRequest_ResponseFormat) Descriptor deprecated 

func (*CreateChatCompletionRequest_ResponseFormat) Descriptor() ([]byte, []int)

Deprecated: Use CreateChatCompletionRequest_ResponseFormat.ProtoReflect.Descriptor instead.

func (*CreateChatCompletionRequest_ResponseFormat) GetType 

func (x *CreateChatCompletionRequest_ResponseFormat) GetType() string

func (*CreateChatCompletionRequest_ResponseFormat) ProtoMessage 

func (*CreateChatCompletionRequest_ResponseFormat) ProtoMessage()

func (*CreateChatCompletionRequest_ResponseFormat) ProtoReflect 

func (x *CreateChatCompletionRequest_ResponseFormat) ProtoReflect() protoreflect.Message

func (*CreateChatCompletionRequest_ResponseFormat) Reset 

func (x *CreateChatCompletionRequest_ResponseFormat) Reset()

func (*CreateChatCompletionRequest_ResponseFormat) String 

func (x *CreateChatCompletionRequest_ResponseFormat) String() string

type CreateChatCompletionRequest_Tool 

type CreateChatCompletionRequest_Tool struct {
	Type     string                                     `protobuf:"bytes,1,opt,name=type,proto3" json:"type,omitempty"`
	Function *CreateChatCompletionRequest_Tool_Function `protobuf:"bytes,2,opt,name=function,proto3" json:"function,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateChatCompletionRequest_Tool) Descriptor deprecated 

func (*CreateChatCompletionRequest_Tool) Descriptor() ([]byte, []int)

Deprecated: Use CreateChatCompletionRequest_Tool.ProtoReflect.Descriptor instead.

func (*CreateChatCompletionRequest_Tool) GetFunction 

func (x *CreateChatCompletionRequest_Tool) GetFunction() *CreateChatCompletionRequest_Tool_Function

func (*CreateChatCompletionRequest_Tool) GetType 

func (x *CreateChatCompletionRequest_Tool) GetType() string

func (*CreateChatCompletionRequest_Tool) ProtoMessage 

func (*CreateChatCompletionRequest_Tool) ProtoMessage()

func (*CreateChatCompletionRequest_Tool) ProtoReflect 

func (x *CreateChatCompletionRequest_Tool) ProtoReflect() protoreflect.Message

func (*CreateChatCompletionRequest_Tool) Reset 

func (x *CreateChatCompletionRequest_Tool) Reset()

func (*CreateChatCompletionRequest_Tool) String 

func (x *CreateChatCompletionRequest_Tool) String() string

type CreateChatCompletionRequest_Tool_Function 

type CreateChatCompletionRequest_Tool_Function struct {
	Description string `protobuf:"bytes,1,opt,name=description,proto3" json:"description,omitempty"`
	Name        string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
	// TODO(kenji): Revisit this. This specifies parameters the functions accepts,
	// described as a JSON Schema object.
	Parameters string `protobuf:"bytes,3,opt,name=parameters,proto3" json:"parameters,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateChatCompletionRequest_Tool_Function) Descriptor deprecated 

func (*CreateChatCompletionRequest_Tool_Function) Descriptor() ([]byte, []int)

Deprecated: Use CreateChatCompletionRequest_Tool_Function.ProtoReflect.Descriptor instead.

func (*CreateChatCompletionRequest_Tool_Function) GetDescription 

func (x *CreateChatCompletionRequest_Tool_Function) GetDescription() string

func (*CreateChatCompletionRequest_Tool_Function) GetName 

func (x *CreateChatCompletionRequest_Tool_Function) GetName() string

func (*CreateChatCompletionRequest_Tool_Function) GetParameters 

func (x *CreateChatCompletionRequest_Tool_Function) GetParameters() string

func (*CreateChatCompletionRequest_Tool_Function) ProtoMessage 

func (*CreateChatCompletionRequest_Tool_Function) ProtoMessage()

func (*CreateChatCompletionRequest_Tool_Function) ProtoReflect 

func (x *CreateChatCompletionRequest_Tool_Function) ProtoReflect() protoreflect.Message

func (*CreateChatCompletionRequest_Tool_Function) Reset 

func (x *CreateChatCompletionRequest_Tool_Function) Reset()

func (*CreateChatCompletionRequest_Tool_Function) String 

func (x *CreateChatCompletionRequest_Tool_Function) String() string

type InferenceEngineInternalServiceClient 

type InferenceEngineInternalServiceClient interface {
}

InferenceEngineInternalServiceClient is the client API for InferenceEngineInternalService service.

For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.

func NewInferenceEngineInternalServiceClient 

func NewInferenceEngineInternalServiceClient(cc grpc.ClientConnInterface) InferenceEngineInternalServiceClient

type InferenceEngineInternalServiceServer 

type InferenceEngineInternalServiceServer interface {
	// contains filtered or unexported methods
}

InferenceEngineInternalServiceServer is the server API for InferenceEngineInternalService service. All implementations must embed UnimplementedInferenceEngineInternalServiceServer for forward compatibility

type UnimplementedChatServiceServer 

type UnimplementedChatServiceServer struct {
}

UnimplementedChatServiceServer must be embedded to have forward compatible implementations.

func (UnimplementedChatServiceServer) CreateChatCompletion 

func (UnimplementedChatServiceServer) CreateChatCompletion(context.Context, *CreateChatCompletionRequest) (*ChatCompletion, error)

type UnimplementedInferenceEngineInternalServiceServer 

type UnimplementedInferenceEngineInternalServiceServer struct {
}

UnimplementedInferenceEngineInternalServiceServer must be embedded to have forward compatible implementations.

type UnsafeChatServiceServer 

type UnsafeChatServiceServer interface {
	// contains filtered or unexported methods
}

UnsafeChatServiceServer may be embedded to opt out of forward compatibility for this service. Use of this interface is not recommended, as added methods to ChatServiceServer will result in compilation errors.

type UnsafeInferenceEngineInternalServiceServer 

type UnsafeInferenceEngineInternalServiceServer interface {
	// contains filtered or unexported methods
}

UnsafeInferenceEngineInternalServiceServer may be embedded to opt out of forward compatibility for this service. Use of this interface is not recommended, as added methods to InferenceEngineInternalServiceServer will result in compilation errors.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.