README
¶
This is a gRPC shim for go-datastore.
The server wraps a datastore in a gRPC server, so that datastores can run out-of-process. If you have a special datastore implementation, simply wrap your datastore in this server and run it, and your datastore can then be called via gRPC from another process via TCP or Unix domain sockets.
The client implements the datastore interface, and calls out to a remote gRPC server.
Why not HTTP?
This could also be implemented with HTTP, but streaming results with chunked encoding and good error messaging is complicated with HTTP, and each client language would need to implement that non-trivial logic. This is much easier with gRPC, which has streaming support and client codegen built-in. The downside is that gRPC is not available for as many programming languages as HTTP. Another downside is that you can't reuse ubiquitous HTTP caching infrastructure and load balancing. The primary use case for this is to run as a sidecar, and gRPC fits that bill well. If there's a significant need for remote datastores, an HTTP version of this might be more appropriate.
Queries
Queries support a number of challenging features for RPC:
- Results are streamed
- Filters and orders are code, not data, so we have to do extra work to use them as data
- Structured errors are embedded in results
Filters and Orders
Filters and orders require special treatment, since they do not have codecs by default.
To support this, each filter and order must be named, and the name mapped to a codec implementation. Both client and server must do this for the desired filters and orders.
This library includes support for the default datastore filters and orders, except OrderByFunction.
Errors
Each result can also contain a structured Go error. Some of these are meaningful (for example, when a result is not found in must return datastore.ErrNotFound). When sending an error in a result, we encode the error as a standard gRPC status protobuf in the same way we do API calls, and apply the same translation logic on the client side, so that datastore-specific errors remain consistent.
Datastore Features
When the client-side datastore is constructed, it negotiates the features supported by the server-side datastore, and then returns an underlying implementation that only implements those features.
"Features" are the extra interfaces that Datastores may implement, such as datastore.Batching, datastore.CheckedDatastore, etc.
This means that client-side interface assertions will behave exactly the same as if you were using the remote datastore directly.
Transactions
Documentation
¶
Index ¶
- func DSToGRPCError(err error) *status.Status
- func GRPCToDSError(err error) error
- func New(ctx context.Context, client pb.DatastoreClient, optFns ...func(o *Options)) (ds.Datastore, error)
- func WithQueryFilterCodec(codec QueryFilterCodec) func(o *Options)
- func WithQueryOrderCodec(codec QueryOrderCodec) func(o *Options)
- type Datastore
- func (d *Datastore) Batch(ctx context.Context) (ds.Batch, error)
- func (d *Datastore) Check(ctx context.Context) error
- func (d *Datastore) Close() error
- func (d *Datastore) CollectGarbage(ctx context.Context) error
- func (d *Datastore) Delete(ctx context.Context, key ds.Key) error
- func (d *Datastore) DiskUsage(ctx context.Context) (uint64, error)
- func (d *Datastore) Get(ctx context.Context, key ds.Key) (value []byte, err error)
- func (d *Datastore) GetExpiration(ctx context.Context, key ds.Key) (time.Time, error)
- func (d *Datastore) GetSize(ctx context.Context, key ds.Key) (size int, err error)
- func (d *Datastore) Has(ctx context.Context, key ds.Key) (exists bool, err error)
- func (d *Datastore) NewTransaction(ctx context.Context, readOnly bool) (ds.Txn, error)
- func (d *Datastore) Put(ctx context.Context, key ds.Key, value []byte) error
- func (d *Datastore) PutWithTTL(ctx context.Context, key ds.Key, value []byte, ttl time.Duration) error
- func (d *Datastore) Query(ctx context.Context, q query.Query) (query.Results, error)
- func (d *Datastore) Scrub(ctx context.Context) error
- func (d *Datastore) SetTTL(ctx context.Context, key ds.Key, ttl time.Duration) error
- func (d *Datastore) Sync(ctx context.Context, prefix ds.Key) error
- type FilterKeyCompareCodec
- type FilterKeyPrefixCodec
- type FilterValueCompareCodec
- type Options
- type OrderByKeyCodec
- type OrderByKeyDescCodec
- type OrderByValueCodec
- type OrderByValueDescCodec
- type QueryFilterCodec
- type QueryOrderCodec
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DSToGRPCError ¶
DSToGRPCError converts a datastore error into a well-formed gRPC error.
func GRPCToDSError ¶
GRPCToDSError converts a well-formed gRPC error into a Datastore error.
func New ¶
func New(ctx context.Context, client pb.DatastoreClient, optFns ...func(o *Options)) (ds.Datastore, error)
New constructs a new shim client that implements the Datastore interface. This negotiates the datastore features that the remote datastore supports, and constructs an underlying representation that only implements those features, so that type assertions on feature interfaces work as expected.
func WithQueryFilterCodec ¶
func WithQueryFilterCodec(codec QueryFilterCodec) func(o *Options)
WithQueryFilterEncoders sets the given query filter encoders. These map from a concrete type that implements
func WithQueryOrderCodec ¶
func WithQueryOrderCodec(codec QueryOrderCodec) func(o *Options)
Types ¶
type Datastore ¶
type Datastore struct {
QueryFilterCodecs map[reflect.Type]QueryFilterCodec
QueryOrderCodecs map[reflect.Type]QueryOrderCodec
Client pb.DatastoreClient
}
Datastore forwards all requests to a gRPC server.
func (*Datastore) GetExpiration ¶
func (*Datastore) NewTransaction ¶
func (*Datastore) PutWithTTL ¶
type FilterKeyCompareCodec ¶
type FilterKeyCompareCodec struct{}
func (*FilterKeyCompareCodec) Decode ¶
func (f *FilterKeyCompareCodec) Decode(b []byte) (query.Filter, error)
func (*FilterKeyCompareCodec) Encode ¶
func (f *FilterKeyCompareCodec) Encode(filter query.Filter) ([]byte, error)
func (*FilterKeyCompareCodec) Name ¶
func (f *FilterKeyCompareCodec) Name() string
func (*FilterKeyCompareCodec) Type ¶
func (f *FilterKeyCompareCodec) Type() reflect.Type
type FilterKeyPrefixCodec ¶
type FilterKeyPrefixCodec struct{}
func (*FilterKeyPrefixCodec) Decode ¶
func (f *FilterKeyPrefixCodec) Decode(b []byte) (query.Filter, error)
func (*FilterKeyPrefixCodec) Encode ¶
func (f *FilterKeyPrefixCodec) Encode(filter query.Filter) ([]byte, error)
func (*FilterKeyPrefixCodec) Name ¶
func (f *FilterKeyPrefixCodec) Name() string
func (*FilterKeyPrefixCodec) Type ¶
func (f *FilterKeyPrefixCodec) Type() reflect.Type
type FilterValueCompareCodec ¶
type FilterValueCompareCodec struct{}
func (*FilterValueCompareCodec) Decode ¶
func (f *FilterValueCompareCodec) Decode(b []byte) (query.Filter, error)
func (*FilterValueCompareCodec) Encode ¶
func (f *FilterValueCompareCodec) Encode(filter query.Filter) ([]byte, error)
func (*FilterValueCompareCodec) Name ¶
func (f *FilterValueCompareCodec) Name() string
func (*FilterValueCompareCodec) Type ¶
func (f *FilterValueCompareCodec) Type() reflect.Type
type Options ¶
type Options struct {
QueryFilterCodecs map[reflect.Type]QueryFilterCodec
QueryOrderCodecs map[reflect.Type]QueryOrderCodec
}
type OrderByKeyCodec ¶
type OrderByKeyCodec struct{}
func (*OrderByKeyCodec) Name ¶
func (o *OrderByKeyCodec) Name() string
func (*OrderByKeyCodec) Type ¶
func (o *OrderByKeyCodec) Type() reflect.Type
type OrderByKeyDescCodec ¶
type OrderByKeyDescCodec struct{}
func (*OrderByKeyDescCodec) Decode ¶
func (o *OrderByKeyDescCodec) Decode(b []byte) (query.Order, error)
func (*OrderByKeyDescCodec) Encode ¶
func (o *OrderByKeyDescCodec) Encode(query.Order) ([]byte, error)
func (*OrderByKeyDescCodec) Name ¶
func (o *OrderByKeyDescCodec) Name() string
func (*OrderByKeyDescCodec) Type ¶
func (o *OrderByKeyDescCodec) Type() reflect.Type
type OrderByValueCodec ¶
type OrderByValueCodec struct{}
func (*OrderByValueCodec) Decode ¶
func (o *OrderByValueCodec) Decode(b []byte) (query.Order, error)
func (*OrderByValueCodec) Name ¶
func (o *OrderByValueCodec) Name() string
func (*OrderByValueCodec) Type ¶
func (o *OrderByValueCodec) Type() reflect.Type
type OrderByValueDescCodec ¶
type OrderByValueDescCodec struct{}
func (*OrderByValueDescCodec) Decode ¶
func (o *OrderByValueDescCodec) Decode(b []byte) (query.Order, error)
func (*OrderByValueDescCodec) Encode ¶
func (o *OrderByValueDescCodec) Encode(query.Order) ([]byte, error)
func (*OrderByValueDescCodec) Name ¶
func (o *OrderByValueDescCodec) Name() string
func (*OrderByValueDescCodec) Type ¶
func (o *OrderByValueDescCodec) Type() reflect.Type
Source Files
Directories