Documentation
¶
Overview ¶
pakcage cas provides a content-addressable storage interface on top of an gocloud compatible interface.
The implementation is written in such way that any service that provides an object storage (s3-like apis) can be used.
MinIO is recommended for instalations that don't rely on a cloud provider
Index ¶
Constants ¶
const (
ErrNotFound = Err("cas reference could not be found")
)
Variables ¶
This section is empty.
Functions ¶
func RefCalculator ¶
func RefCalculator(out *Ref, content io.Reader) io.ReadCloser
RefCaclulator returns a reader that computes the hash from the given content as consumers read data.
The Ref *pointer is updated when content.Read returns 0 bytes or an error is found, including io.EOF
Types ¶
type C ¶
type C struct {
// contains filtered or unexported fields
}
C implements the CAS abstraction on top of a S3 compatible storage
func (*C) GetContent ¶
Get writes the object at ref to the given output it returns the underlying KV error without any modification
func (*C) PutContent ¶
PutContent writes content to a temporary object and later copies that object to the final path under the sha256 hash.
This avoids reading the object twice but might incur in costs on S3-like services, even though the temporary object is alive for a short period of time.
This function does not perform deduplication, so if the content already exists, it will be re-uploaded. The Copy/Move operation wont be executed in this scenario.
If the provided KV object implementes the Mover interface, then instead of Copy/Delete cas will use the Move operation.
type KV ¶
type KV interface {
io.Closer
Write(context.Context, string, io.Reader) (int64, error)
Read(context.Context, io.Writer, string) (int64, error)
Copy(context.Context, string, string) error
Delete(context.Context, string) error
Exists(context.Context, string) (bool, error)
}
KV represents the bare minimum abstraction required by cas
type Mover ¶
MovableObjects contains the methods required by cas which help reduce the load on the remote server
type NewTable ¶
NewTable should return a KV object which is used to store the items
cas package is responsible for closing the KV
type Ref ¶
type Ref [32]byte
Ref contains the binary value of the sha256 hash which identifies any object
func PrecomputeHashBytes ¶
PrecomputeHashBytes returns the expected Ref value for the given set of bytes
type RollingRef ¶
type RollingRef interface {
io.WriteCloser
io.ByteWriter
Ref() Ref
Reset()
}
RollingRef computes a ref value as bytes are written to it it is just a wraper over a hash.Hash
func NewRollingRef ¶
func NewRollingRef() RollingRef
NewRollingRef configures a new rolling hash object
Source Files