ops

package
v0.0.0-...-8d852e1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 20, 2022 License: MIT Imports: 14 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Distancer2Vec 

func Distancer2Vec(d mathx.Distancer) []float64

Distancer2Vec simply converts a mathx.Distancer (collection of float64) into a []float64. The reason this is needed is that Go rpc doesn't seem to work with interfaces.

Types

type AddDataArgs 

type AddDataArgs struct {
	Namespace string
	Vec       []float64
	Data      []byte
	Expires   time.Time
}

AddDataArgs is intended as args for Client.AddData.

type CInfo 

type CInfo Client

CInfo is a namespace on top of Client, which contains methods related to metadata, similar to requestman.Handle.Info().

func (*CInfo) KNNLatency 

func (ci *CInfo) KNNLatency(args KNNLatencyArgs) *ClientResult[KNNLatencyResp]

KNNLatency tries to get some metadata related to KNN latency on the remote server, specifically queue and query durations.

The remote server forwards the call to these methods: - requestman.Handle.Info().KNNQueueLatency(...) - requestman.Handle.Info().KNNQueryLatency(...) See docs for those methods for more details about args, returns, etc.

func (*CInfo) KNNMonitor 

func (ci *CInfo) KNNMonitor(args KNNMonArgs) *ClientResult[rman.KNNMonItemAvg]

KNNMonitor tries to get monitoring data related to KNN queries from the remote server.

The remote server forwards the call to the method with the same name on top of its internal requestmanager.Handle.Info(). See the docs for that path for more details about args, returns, etc.

func (*CInfo) SSpaceCap 

func (ci *CInfo) SSpaceCap(key string) *ClientResult[SSpaceCapResp]

SSpaceCap tries to get the search space (not total vector) capacity for a given key/namespace from the remote server.

The remote server forwards the call to the method with the same name on top of its internal requestmanager.Handle.Info(). See the docs for that path for more details about args, returns, etc.

func (*CInfo) SSpaceDim 

func (ci *CInfo) SSpaceDim(key string) *ClientResult[SSpaceDimResp]

SSpaceDim tries to get the uniform dimension for vectors on the search space with the given key/namespace from the remote server.

The remote server forwards the call to the method with the same name on top of its internal requestmanager.Handle.Info(). See the docs for that path for more details about args, returns, etc.

func (*CInfo) SSpaceLen 

func (ci *CInfo) SSpaceLen(key string) *ClientResult[SSpaceLenResp]

SSpaceLen tries to get the amount of search spaces (and the sum of all their vectors) for a given key/namespace from the remote server.

The remote server forwards the call to the method with the same name on top of its internal requestmanager.Handle.Info(). See the docs for that path for more details about args, returns, etc.

func (*CInfo) SSpaceNamespace 

func (ci *CInfo) SSpaceNamespace(key string) *ClientResult[bool]

SSpaceNamespace checks if the given namespace/key exists on the remote server.

The remote server forwards the call to the method with the same name on top of its internal requestmanager.Handle.Info(). See the docs for that path for more details about args, returns, etc.

func (*CInfo) SSpaceNamespaces 

func (ci *CInfo) SSpaceNamespaces() *ClientResult[[]string]

SSpaceNamespaces tries to get namespaces from the remote server.

The remote server forwards the call to the method with the same name on top of its internal requestmanager.Handle.Info(). See the docs for that path for more details about args, returns, etc.

type CSInfo 

type CSInfo Clients

CSInfo is for namespacing info methods of type Clients.

func (*CSInfo) KNNLatency 

func (csi *CSInfo) KNNLatency(args KNNLatencyArgs) ClientResults[KNNLatencyResp]

KNNLatency does a composite call to Client.Info().KNNLatency(), using all internal addrs. See docs for that method for more details.

func (*CSInfo) KNNMonitor 

func (csi *CSInfo) KNNMonitor(args KNNMonArgs) ClientResults[rman.KNNMonItemAvg]

KNNMonitor does a composite call to Client.Info().KNNMonitor(), using all internal addrs. See docs for that method for more details.

func (*CSInfo) SSpaceCap 

func (csi *CSInfo) SSpaceCap(key string) ClientResults[SSpaceCapResp]

SSpaceCap does a composite call to Client.Info().SSpaceCap(), using all internal addrs. See docs for that method for more details.

func (*CSInfo) SSpaceDim 

func (csi *CSInfo) SSpaceDim(key string) ClientResults[SSpaceDimResp]

SSpaceDim does a composite call to Client.Info().SSpaceDim(), using all internal addrs. See docs for that method for more details.

func (*CSInfo) SSpaceLen 

func (csi *CSInfo) SSpaceLen(key string) ClientResults[SSpaceLenResp]

SSpaceLen does a composite call to Client.Info().SSpaceLen(), using all internal addrs. See docs for that method for more details.

func (*CSInfo) SSpaceNamespace 

func (csi *CSInfo) SSpaceNamespace(key string) ClientResults[bool]

SSpaceNamespace does a composite call to Client.Info().SSpaceNamespace(), using all internal addrs. See docs for that method for more details.

func (*CSInfo) SSpaceNamespaces 

func (csi *CSInfo) SSpaceNamespaces() ClientResults[[]string]

SSpaceNamespaces does a composite call to Client.Info().SSpaceNamespaces(), using all internal addrs. See docs for that method for more details.

type Client 

type Client struct {
	RemoteAddr string
	// Timeout specifies connection timeout.
	Timeout time.Duration
}

Client is a client for the rpc network server defined in this pkg. It is used to interface remote requestman.Handle.

func NewClient 

func NewClient(remoteAddr string, timeout ...time.Duration) *Client

NewClient sets up a new client. If a timeout isn't specified, or has a time.Duration <= 0, then the timeout will be set to 3 seconds as default.

func (*Client) AddData 

func (c *Client) AddData(args []AddDataArgs) *ClientResult[[]bool]

AddData tries to add data to the remote server. The remote server uses requestmanager.Handle.AddData(...), see the docs for more details about args, returns, etc.

func (*Client) Info 

func (c *Client) Info() *CInfo

Info returns a method namespace. Similar to requestman.Handle.Info()

func (*Client) KNNEager 

func (c *Client) KNNEager(args rman.KNNArgs) *ClientResult[KNNResp]

KNNEager tries to (eagerly) do a KNN lookup on a remote server. The remote server uses requestmanager.Handle.KNN(...), see the docs for more details about args, returns, etc.

Note; network latency is factored in with args.TTL.

Note; eagers means that it calls the server, which waits for the entire knn request before returning any results.

func (*Client) Ping 

func (c *Client) Ping() *ClientResult[bool]

Ping pings the remote server. The returned ClientResult.Payload will be true if the server is alive.

type ClientResult 

type ClientResult[T any] struct {
	RemoteAddr string
	NetErr     error
	Payload    T

	NetworkLatency time.Duration
}

ClientResult is a wrapper around any result returned from a client -> server call. It contains additional meta info, specifically the address used, err, as well as network latency.

type ClientResults 

type ClientResults[T any] <-chan *ClientResult[T]

ClientResults is the general return from T Clients methods.

type Clients 

type Clients struct {
	RemoteAddrs []string
	Timeout     time.Duration // This is passed to each individual Client.
}

Clients T is used for composite Client calls.

func NewClients 

func NewClients(remoteAddrs []string, timeout ...time.Duration) *Clients

NewClients sets up a new composite client. If a timeout isn't specified, or has time.Duration <= 0, then the timeout will be set to 3 seconds as default.

func (*Clients) AddData 

func (cs *Clients) AddData(args []AddDataArgs) ClientResults[[]bool]

AddData does a composite call to Client.AddData(), using all internal addrs. Do note that the data to add (i.e "args") is added to a single remote node, picked at random, as a way of avoiding data duplication. See docs for that method for more details.

func (*Clients) Info 

func (cs *Clients) Info() *CSInfo

Info returns a method namespace. Similar to Client.Info()

func (*Clients) KNNEager 

func (cs *Clients) KNNEager(args rman.KNNArgs) ClientResults[KNNResp]

KNNEager does a composite call to Client.KNNEager(), using all internal addrs. See docs for that method for more details. Also see Clients.KNNEagerx for merging and ordering the results.

func (*Clients) KNNEagerx 

func (cs *Clients) KNNEagerx(args rman.KNNArgs) []*ClientResult[KNNRespItem]

KNNEagerx is a convenience on top of Clients.KNNEager. It calls the latter method, then orders KNN results into max args.K. The return is a flat slice of ClientResult containing a single KNNRespItem, where lower indexes are better KNN. It can look something like the following (simplified, using cosine similarity where higher scores are better): [ 

addr: ":3000", score: 0.99, Vec: ...,
addr: ":3000", score: 0.98, Vec: ...,
addr: ":3001", score: 0.97, Vec: ...,

] This is to include network information in addition to actual KNN results.

func (*Clients) Ping 

func (cs *Clients) Ping() ClientResults[bool]

Ping does a composite call to Client.Ping(), using all internal addrs. See docs for that method for more details.

type KNNLatencyArgs 

type KNNLatencyArgs struct {
	Key    string        // Key specifies the namespace to use.
	Period time.Duration // Period specifies what period (since now) to check.
}

KNNLatencyArgs is intended for CInfo.KNNLatency.

type KNNLatencyResp 

type KNNLatencyResp struct {
	LookupOk bool          // LookupOk indicates if the namespace/key was valid.
	Queue    time.Duration // Queue is for knn queue latency.
	Query    time.Duration // Query is for knn query latency.
	BoundsOk bool          // BoundsOk is false if the period used was too large.
}

type KNNMonArgs 

type KNNMonArgs struct {
	Period time.Duration
}

KNNMonArgs is intended for CInfo.KNNMonitor

type KNNResp 

type KNNResp struct {
	KNN []KNNRespItem
	// Ok is generally matched with the bool returned from
	// requestman.Handle.KNN. But it is also false if the
	// requestman.KNNArgs.TTL is less than network latency.
	Ok bool
}

KNNResp is intended as the response of Client.KNNEager.

type KNNRespItem 

type KNNRespItem struct {
	Vec   []float64
	Score float64
}

KNNRespItem is intended as a single item in KNNResp.

func KNNRespItemFromScoreItem 

func KNNRespItemFromScoreItem(scoreItem knnc.ScoreItem) KNNRespItem

KNNRespItemFromScoreItem converts KNN results (pkg knnc and requestman) into a KNNRespItem. See docs for Distancer2Vec for why this is needed.

func KNNRespItemsFromScoreItems 

func KNNRespItemsFromScoreItems(scoreItems knnc.ScoreItems) []KNNRespItem

KNNRespItemsFromScoreItems converts KNN results (pkg knnc and requestman) into a KNNResp. See docs for Distancer2Vec for why this is needed.

type SArgs 

type SArgs[T any] struct {
	SendTime time.Time
	Payload  T
}

SArgs is used as a Server argument wrapper with metadata. Go rpc methods are required to have the following signature format: 

x.Method(args any, resp *any) error

This is used as the 'args'.

func NewSArgs 

func NewSArgs[T any](payload T) SArgs[T]

NewSArgs is a convenience func for setting up a new SArgs[T] with instance with the SendTime field set to time.Now().

type SInfo 

type SInfo Server

Info namespace of server (same as CInfo for Client). Registered with rpc in the Server.StartListen method.

func (*SInfo) KNNLatency 

func (i *SInfo) KNNLatency(args SArgs[KNNLatencyArgs], resp *SResp[KNNLatencyResp]) error

KNNLatency forwards the call to the following methods of the internal requestman.Handle: - requestman.Handle.Info().KNNQueueLatency(...) - requestman.Handle.Info().KNNQueryLatency(...) See docs for those methods for more details about args, returns, etc.

func (*SInfo) KNNMonitor 

func (i *SInfo) KNNMonitor(args SArgs[KNNMonArgs], resp *SResp[rman.KNNMonItemAvg]) error

KNNMonitor forwards the call to the method with the same name on top of the internal requestman.Handle.Info(). See docs for that for more details.

func (*SInfo) SSpaceCap 

func (i *SInfo) SSpaceCap(args SArgs[string], resp *SResp[SSpaceCapResp]) error

SSpaceCap forwards the call to the method with the same name on top of the internal requestman.Handle.Info(). See docs for that for more details.

func (*SInfo) SSpaceDim 

func (i *SInfo) SSpaceDim(args SArgs[string], resp *SResp[SSpaceDimResp]) error

SSpaceDim forwards the call to the method with the same name on top of the internal requestman.Handle.Info(). See docs for that for more details.

func (*SInfo) SSpaceLen 

func (i *SInfo) SSpaceLen(args SArgs[string], resp *SResp[SSpaceLenResp]) error

SSpaceLen forwards the call to the method with the same name on top of the internal requestman.Handle.Info(). See docs for that for more details.

func (*SInfo) SSpaceNamespace 

func (i *SInfo) SSpaceNamespace(args SArgs[string], resp *SResp[bool]) error

SSpaceNamespace forwards the call to the method with the same name on top of the internal requestman.Handle.Info(). See docs for that for more details.

func (*SInfo) SSpaceNamespaces 

func (i *SInfo) SSpaceNamespaces(args SArgs[bool], resp *SResp[[]string]) error

SSpaceNamespaces forwards the call to the method with the same name on top of the internal requestman.Handle.Info(). See docs for that for more details.

type SResp 

type SResp[T any] struct {
	RecvTime time.Time
	Payload  T
}

SResp is used as a Server argument wrapper with metadata. Go rpc methods are required to have the following signature format: 

x.Method(args any, resp *any) error

This is used as the 'resp'.

type SSpaceCapResp 

type SSpaceCapResp struct {
	LookupOk bool // LookupOk indicates if the namespace/key was valid.
	Cap      int  // Cap specifies how many search spaces can exist.
}

SSpaceCapresp is intended as a response from CInfo.SSpaceCap.

type SSpaceDimResp 

type SSpaceDimResp struct {
	LookupOk bool // LookupOk indicates if the namespace/key was valid.
	Dim      int  // Uniform vector dimension.
}

SSpaceDimResp is intended as a response from CInfo.SSpaceDim.

type SSpaceLenResp 

type SSpaceLenResp struct {
	LookupOk bool // LookupOk indicates if the namespace/key was valid.
	NSSpaces int  // NSSpaces specifies a number of search spaces.
	NVecs    int  // NVecs specifies the total number of vectors.
}

SSPaceLenResp is intended as a response from CInfo.SSpaceLen.

type Server 

type Server struct {
	LocalAddr string
	// contains filtered or unexported fields
}

Server is an rpc server on top of requestman.Handle.

func NewServer 

func NewServer(localAddr string, rManHandleArgs rman.NewHandleArgs) (*Server, bool)

NewServer is a factory function. Will return (nil, false) is a new requestman.Handle could not be created with the given rManHandleArgs.

func (*Server) AddData 

func (s *Server) AddData(args SArgs[[]AddDataArgs], resp *SResp[[]bool]) error

AddData attempts to add the given data to the internal requestman.Handle with the AddData() method. The returns of those AddData() calls are stored index for index in the response.

func (*Server) KNNEager 

func (s *Server) KNNEager(args SArgs[rman.KNNArgs], resp *SResp[KNNResp]) error

KNNEager attempts to do a KNN request using the KNN method of the internal requestmanager.Handle. It does so eagerly, so will wait until the KNN request is complete.

Note that network latency is factored in with args.Payload.TTL

func (*Server) Ping 

func (s *Server) Ping(args SArgs[bool], resp *SResp[bool]) error

Ping simply sets resp.RecvTime to now and resp.Payload to true (on success).

func (*Server) StartListen 

func (s *Server) StartListen() (stop func(), err error)

StartListen spins up the server and makes it active. The returned func is used for stopping (this also stops the internal requestman.Handle), while the error indicates the following (These are for the setup): - rpc.NewServer().Register(this) returns an err. - net.Listen("tcp", this.LocalAddr) returns an err.

Note, while accepting requests with net.Listener.Accept(), if an err is returned, then the listening event-loop simply fails.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.