shardgrpc

package module
v0.5.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 21, 2023 License: MIT Imports: 17 Imported by: 0

README

SHARD_GRPC

Go Reference

Structure

Required

  • Recommend server need using statefulset for easy routing.
  • GRPC server need install in big machine(pod/service) for high perfomance.

Install

    go get github.com/teng231/shardgrpc
  • Using for client: example
    // for example
    conn, err := grpc.Dial("lhost-0:21240",
        grpc.WithTransportCredentials(insecure.NewCredentials()),
        grpc.WithUnaryInterceptor(UnaryClientInterceptor(
            &DialConfig{ThrottlingDuration: 10 * time.Millisecond, MaxRetryConnect: 3},
            grpc.WithTransportCredentials(insecure.NewCredentials()),
        )))
    if err != nil {
        panic(err)
    }
    client := pb.NewVistorServiceClient(conn)

send request in process

    // ...
    var header metadata.MD // variable to store header and trailer
    c, cancel := MakeContext(20, nil)
    defer cancel()
    c = metadata.AppendToOutgoingContext(c, "s_key", "teng")
    resp, err := client.ListVisitors(c, &pb.VisitorRequest{AccountId: "teng"}, grpc.Header(&header))
    // log.Print("header", header)
    log.Print(resp, err, header)
    if err != nil {
        log.Print(err)
    }
  • Using for server: example
    func (me TestShardApiServer) ServeV1(hostname, port string, shardcount int) {
        lis, err := net.Listen("tcp", hostname+":"+port)
        if err != nil {
            panic(err)
        }
        opts := []grpc.ServerOption{
            grpc.KeepaliveParams(keepalive.ServerParameters{
                MaxConnectionAge: 30 * time.Second,
            }),
            grpc.UnaryInterceptor(
                UnaryServerInterceptorStatefullset(hostname, port, "", shardcount),
            ),
        }
        grpcServer := grpc.NewServer(opts...)
        pb.RegisterVistorServiceServer(grpcServer, &me)
        grpcServer.Serve(lis)
    }

REFERENCES:

Intercepter Structure https://github.com/ori-edge/grpc-interceptor-demo/blob/fd821c3556ff6eebd6eab1a617df0fb68f7c01a2/pkg/interceptor/interceptor.go

Structure of shard service https://www.notion.so/Shard-service-c002bcb0b00c47669bce547be646cd9f

structure example https://medium0.com/m/global-identity?redirectUrl=https%3A%2F%2Fedgehog.blog%2Fa-guide-to-grpc-and-interceptors-265c306d3773

Documentation

Index

Constants

View Source
const (
	MAX_BACKOFF = 3
)
View Source
const (
	TransportError = "transport: error while dialing"
)

Variables

This section is empty.

Functions

func GetClientShardKey added in v0.1.2

func GetClientShardKey(ctx context.Context, message interface{}) string

func GetShardIndex added in v0.4.1

func GetShardIndex(hostname string) int

func UnaryClientInterceptor

func UnaryClientInterceptor(dialConfig *DialConfig, dialOpts ...grpc.DialOption) grpc.UnaryClientInterceptor

UnaryClientInterceptor is called on every request from a client to a unary server operation, here, we grab the operating system of the client and add it to the metadata within the context of the request so that it can be received by the server

func UnaryServerInterceptor

func UnaryServerInterceptor(addrs []string, id int) grpc.UnaryServerInterceptor

UnaryServerInterceptor is called on every request received from a client to a unary server operation, here, we pull out the client operating system from the metadata, and inspect the context to receive the IP address that the request was received from. We then modify the EdgeLocation type to include this information for every request

func UnaryServerInterceptorPreset added in v0.5.1

func UnaryServerInterceptorPreset(servers []*ServerInfo, currentAddr string) grpc.UnaryServerInterceptor

UnaryServerInterceptorPreset is called on every request received from a client to a unary server operation, here, we pull out the client operating system from the metadata, and inspect the context to receive the IP address that the request was received from. We then modify the EdgeLocation type to include this information for every request normal case servers like [{host: localhost, port: 8001},{host: localhost, port: 8002}]

func UnaryServerInterceptorStatefullset

func UnaryServerInterceptorStatefullset(hostname, port, serviceDomain string, totalShard int) grpc.UnaryServerInterceptor

UnaryServerInterceptorStatefullset is called on every request received from a client to a unary server operation, here, we pull out the client operating system from the metadata, and inspect the context to receive the IP address that the request was received from. We then modify the EdgeLocation type to include this information for every request in statefull pod-name hostname like: web-0:3456, web-1:3456, ... web-i:3456 serviceDomain like a12.staging.svc.cluster.local

Types

type Client added in v0.4.1

type Client struct {
	// contains filtered or unexported fields
}

func CreateShardClient added in v0.4.1

func CreateShardClient() *Client

func (*Client) CalcShardIndexByKey added in v0.4.3

func (s *Client) CalcShardIndexByKey(key string) int

func (*Client) GetAddrs added in v0.4.3

func (s *Client) GetAddrs() []string

func (*Client) GetShardCount added in v0.4.1

func (s *Client) GetShardCount() int

func (*Client) UnaryClientInterceptor added in v0.4.1

func (s *Client) UnaryClientInterceptor(dialConfig *DialConfig, dialOpts ...grpc.DialOption) grpc.UnaryClientInterceptor

UnaryClientInterceptor is called on every request from a client to a unary server operation, here, we grab the operating system of the client and add it to the metadata within the context of the request so that it can be received by the server

type DialConfig added in v0.3.2

type DialConfig struct {
	MaxRetryConnect    int
	ThrottlingDuration time.Duration
	DefaultDNS         string
}

type IClient added in v0.4.2

type IClient interface {
	UnaryClientInterceptor(dialConfig *DialConfig, dialOpts ...grpc.DialOption) grpc.UnaryClientInterceptor
	GetShardCount() int
	GetAddrs() []string
	CalcShardIndexByKey(key string) int
}

type ServerInfo added in v0.5.1

type ServerInfo struct {
	Host string
	Port int
}

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL