pachyderm: github.com/pachyderm/pachyderm/src/client/pkg/grpcutil Index | Files

package grpcutil

import "github.com/pachyderm/pachyderm/src/client/pkg/grpcutil"

Index

Package Files

buffer.go dialer.go error.go server.go stream.go

Variables

var (
    // ErrMustSpecifyRegisterFunc is used when a register func is nil.
    ErrMustSpecifyRegisterFunc = errors.New("must specify registerFunc")

    // ErrMustSpecifyPort is used when a port is 0
    ErrMustSpecifyPort = errors.New("must specify port on which to serve")
)
var (
    // MaxMsgSize is used to define the GRPC frame size
    MaxMsgSize = 20 * 1024 * 1024
)

func Chunk Uses

func Chunk(data []byte, chunkSize int) [][]byte

Chunk splits a piece of data up, this is useful for splitting up data that's bigger than MaxMsgSize

func ChunkReader Uses

func ChunkReader(r io.Reader, f func([]byte) error) (int, error)

ChunkReader splits a reader into reasonably sized chunks for the purpose of transmitting the chunks over gRPC. For each chunk, it calls the given function.

func GetBuffer Uses

func GetBuffer() []byte

GetBuffer returns a buffer. The buffer may or may not be freshly allocated, and it may or may not be zero-ed.

func NewStreamingBytesReader Uses

func NewStreamingBytesReader(streamingBytesClient StreamingBytesClient, cancel context.CancelFunc) io.ReadCloser

NewStreamingBytesReader returns an io.Reader for a StreamingBytesClient.

func NewStreamingBytesWriter Uses

func NewStreamingBytesWriter(streamingBytesServer StreamingBytesServer) io.Writer

NewStreamingBytesWriter returns an io.Writer for a StreamingBytesServer.

func PutBuffer Uses

func PutBuffer(buf []byte)

PutBuffer returns the buffer to the pool.

func ScrubGRPC Uses

func ScrubGRPC(err error) error

ScrubGRPC removes GRPC error code information from 'err' if it came from GRPC (and returns it unchanged otherwise)

func Serve Uses

func Serve(
    servers ...ServerOptions,
) (retErr error)

Serve serves stuff.

func WriteFromStreamingBytesClient Uses

func WriteFromStreamingBytesClient(streamingBytesClient StreamingBytesClient, writer io.Writer) error

WriteFromStreamingBytesClient writes from the StreamingBytesClient to the io.Writer.

func WriteToStreamingBytesServer Uses

func WriteToStreamingBytesServer(reader io.Reader, streamingBytesServer StreamingBytesServer) error

WriteToStreamingBytesServer writes the data from the io.Reader to the StreamingBytesServer.

type BufPool Uses

type BufPool struct {
    sync.Pool
}

BufPool is a wrapper around sync.Pool that makes it a little nicer to use for []byte by doing the casting for you and defining the `New` function.

func NewBufPool Uses

func NewBufPool(size int) *BufPool

NewBufPool creates a new BufPool that returns buffers of the given size.

func (*BufPool) GetBuffer Uses

func (b *BufPool) GetBuffer() []byte

GetBuffer returns a buffer. The buffer may or may not be freshly allocated, and it may or may not be zero-ed.

func (*BufPool) PutBuffer Uses

func (b *BufPool) PutBuffer(buf []byte)

PutBuffer returns the buffer to the pool.

type ChunkWriteCloser Uses

type ChunkWriteCloser struct {
    // contains filtered or unexported fields
}

ChunkWriteCloser is a utility for buffering writes into buffers obtained from a buffer pool. The ChunkWriteCloser will buffer up to the capacity of a buffer obtained from a buffer pool, then execute a callback that will receive the buffered data. The ChunkWriteCloser will get a new buffer from the pool for subsequent writes, so it is expected that the callback will return the buffer to the pool.

func NewChunkWriteCloser Uses

func NewChunkWriteCloser(bufPool *BufPool, f func(chunk []byte) error) *ChunkWriteCloser

NewChunkWriteCloser creates a new ChunkWriteCloser.

func (*ChunkWriteCloser) Close Uses

func (w *ChunkWriteCloser) Close() error

Close closes the writer.

func (*ChunkWriteCloser) Write Uses

func (w *ChunkWriteCloser) Write(data []byte) (int, error)

Write performs a write.

type Dialer Uses

type Dialer interface {
    Dial(address string) (*grpc.ClientConn, error)
    CloseConns() error
}

Dialer defines a grpc.ClientConn connection dialer.

func NewDialer Uses

func NewDialer(opts ...grpc.DialOption) Dialer

NewDialer creates a Dialer.

type ReaderWrapper Uses

type ReaderWrapper struct {
    Reader io.Reader
}

ReaderWrapper wraps a reader for the following reason: Go's io.CopyBuffer has an annoying optimization wherein if the reader has the WriteTo function defined, it doesn't actually use the given buffer. As a result, we might write a large chunk to the gRPC streaming server even though we intend to use a small buffer. Therefore we wrap readers in this wrapper so that only Read is defined.

func (ReaderWrapper) Read Uses

func (r ReaderWrapper) Read(p []byte) (int, error)

type ServerOptions Uses

type ServerOptions struct {
    Port         uint16
    MaxMsgSize   int
    Cancel       chan struct{}
    RegisterFunc func(*grpc.Server) error

    // If set, grpcutil may enable TLS.  This should be set for public ports that
    // serve GRPC services to 3rd party clients.
    //
    // If set, the criterion for actually serving over TLS is:
    // if a signed TLS cert and corresponding private key in 'TLSVolumePath',
    // this will serve GRPC traffic over TLS. If either are missing this will
    // serve GRPC traffic over unencrypted HTTP,
    //
    // TODO make the TLS cert and key path a parameter, as pachd will need
    // multiple certificates for multiple ports
    PublicPortTLSAllowed bool
}

ServerOptions represent optional fields for serving.

type StreamingBytesClient Uses

type StreamingBytesClient interface {
    Recv() (*types.BytesValue, error)
}

StreamingBytesClient represents a client for an rpc method of the form:

rpc Foo(Bar) returns (stream google.protobuf.BytesValue) {}

type StreamingBytesServer Uses

type StreamingBytesServer interface {
    Send(bytesValue *types.BytesValue) error
}

StreamingBytesServer represents a server for an rpc method of the form:

rpc Foo(Bar) returns (stream google.protobuf.BytesValue) {}

Package grpcutil imports 17 packages (graph) and is imported by 31 packages. Updated 2019-09-05. Refresh now. Tools for package owners.