Documentation
¶
Index ¶
- Constants
- Variables
- type Location
- type Recovery
- type Store
- func (s *Store) Close() error
- func (s *Store) Metrics() []prometheus.Collector
- func (s *Store) Read(ctx context.Context, loc Location, buf []byte) (err error)
- func (s *Store) Release(ctx context.Context, loc Location) error
- func (s *Store) Write(ctx context.Context, data []byte) (loc Location, err error)
Constants ¶
const LocationSize int = 7
LocationSize is the size of the byte representation of Location
Variables ¶
var ( // ErrTooLong returned by Write if the blob length exceeds the max blobsize. ErrTooLong = errors.New("data too long") // ErrQuitting returned by Write when the store is Closed before the write completes. ErrQuitting = errors.New("quitting") )
var ErrShardNotFound = errors.New("shard not found")
Functions ¶
This section is empty.
Types ¶
type Location ¶
Location models the location <shard, slot, length> of a chunk
func LocationFromBinary ¶
LocationFromBinary is a helper to construct a Location object from byte representation
func (*Location) MarshalBinary ¶
MarshalBinary returns byte representation of location
func (*Location) UnmarshalBinary ¶
UnmarshalBinary constructs the location from byte representation
type Recovery ¶
type Recovery struct {
// contains filtered or unexported fields
}
Recovery allows disaster recovery.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store models the sharded fix-length blobstore Design provides lockless sharding: - shard choice responding to backpressure by running operation - read prioritisation over writing - free slots allow write
func New ¶
New constructs a sharded blobstore arguments: - base directory string - shard count - positive integer < 256 - cannot be zero or expect panic - shard size - positive integer multiple of 8 - for others expect undefined behaviour - maxDataSize - positive integer representing the maximum blob size to be stored
func (*Store) Metrics ¶
func (s *Store) Metrics() []prometheus.Collector
Metrics returns set of prometheus collectors.
func (*Store) Read ¶
Read reads the content of the blob found at location into the byte buffer given The location is assumed to be obtained by an earlier Write call storing the blob
func (*Store) Release ¶
Release gives back the slot to the shard From here on the slot can be reused and overwritten Release is meant to be called when an entry in the upstream db is removed Note that releasing is not safe for obfuscating earlier content, since even after reuse, the slot may be used by a very short blob and leaves the rest of the old blob bytes untouched
Source Files