README
¶
blubberstore
An HTTP-/RPC-accessible store for binary large objects. Nothing more.
Documentation
¶
Overview ¶
Client library for blubberstore end users.
By design, you will most likely want to use the BlubberStoreClient class, which is part of this package and makes use of all the other packages to seamlessly provide access to blobs stored on remote hosts which are looked up for you in the blob directory.
Simple client library to connect to a single blubber store server.
Index ¶
- Variables
- func NewRPCClient(uri, cert, key, cacert string, insecure bool) (*rpc.Client, *tls.Config, error)
- type BlubberDirectoryClient
- func (b *BlubberDirectoryClient) ExpireHost(hosts BlockHolderList) error
- func (b *BlubberDirectoryClient) GetFreeHosts(freeHostReq FreeHostsRequest) (servers *BlockHolderList, err error)
- func (b *BlubberDirectoryClient) ListHosts() (hosts *BlockHolderList, err error)
- func (b *BlubberDirectoryClient) LookupBlob(id BlockId) (list *BlockHolderList, err error)
- func (b *BlubberDirectoryClient) RemoveBlobHolder(rep BlockRemovalReport) error
- func (b *BlubberDirectoryClient) ReportBlob(report BlockReport) error
- type BlubberRPCClient
- func (self *BlubberRPCClient) CopyBlob(source BlockSource) error
- func (self *BlubberRPCClient) DeleteBlob(id BlockId) error
- func (self *BlubberRPCClient) GetRPCClient() *rpc.Client
- func (self *BlubberRPCClient) RetrieveBlob(id BlockId) (ret BlockWithData, err error)
- func (self *BlubberRPCClient) StatBlob(id BlockId) (stat BlubberStat, err error)
- func (self *BlubberRPCClient) StoreBlob(data BlockWithData) (err error)
- type BlubberStoreClient
Constants ¶
This section is empty.
Variables ¶
var Err_IncompleteWrite = errors.New("Write did not complete to a sufficiently large quorum")
The write failed to reach a fully replicated set, but a quorum has been achieved. This may be safe to ignore, but the file should be distributed better.
var Err_NoHostsReached = errors.New("Unable to copy the block to even one server - please try again")
A number of hosts have been contacted but storing the block failed on all of them (e.g. because they could not be reached). This means that no data has been written whatsoever.
var Err_NoQuorumReached = errors.New("No quorum could be reached")
Less than a quorum of the requested nodes have been written. This means that the data may disappear again in case of outages.
Functions ¶
func NewRPCClient ¶
Create a new generic RPC client connecting to the given server.
uri should be an URI pointing to the desired server and port. cert, key and cacert should be the path of the X.509 client certificate, private key and CA certificate, respectively. The insecure flag can be used to disable encryption (only for testing and if the server is also running in insecure mode).
Types ¶
type BlubberDirectoryClient ¶
type BlubberDirectoryClient struct {
// contains filtered or unexported fields
}
A client to a blubber directory.
func NewBlubberDirectoryClient ¶
func NewBlubberDirectoryClient(uri, cert, key, cacert string, insecure bool) ( *BlubberDirectoryClient, error)
Create a new blubber directory client connecting to the given server.
uri should be an URI pointing to the desired server and port. cert, key and cacert should be the path of the X.509 client certificate, private key and CA certificate, respectively. The insecure flag can be used to disable encryption (only for testing and if the server is also running in insecure mode).
func (*BlubberDirectoryClient) ExpireHost ¶
func (b *BlubberDirectoryClient) ExpireHost(hosts BlockHolderList) error
Delete all block ownerships associated with the given host. This is very useful e.g. if a host goes down or data is lost on it.
func (*BlubberDirectoryClient) GetFreeHosts ¶
func (b *BlubberDirectoryClient) GetFreeHosts(freeHostReq FreeHostsRequest) ( servers *BlockHolderList, err error)
Pick a number of hosts from the available list. This will try to pick hosts which hold less keys than the others.
func (*BlubberDirectoryClient) ListHosts ¶
func (b *BlubberDirectoryClient) ListHosts() ( hosts *BlockHolderList, err error)
Get a list of all hosts known to own blocks. This is mostly used by the cleanup jobs.
func (*BlubberDirectoryClient) LookupBlob ¶
func (b *BlubberDirectoryClient) LookupBlob(id BlockId) (list *BlockHolderList, err error)
Look up all the hosts currently known to hold the requested block.
func (*BlubberDirectoryClient) RemoveBlobHolder ¶
func (b *BlubberDirectoryClient) RemoveBlobHolder(rep BlockRemovalReport) error
Remove the given host from the holders of the blob.
func (*BlubberDirectoryClient) ReportBlob ¶
func (b *BlubberDirectoryClient) ReportBlob(report BlockReport) error
Report to the directory that the block with the given properties is now stored on the specified server.
type BlubberRPCClient ¶
type BlubberRPCClient struct {
// contains filtered or unexported fields
}
A client to connect to blubber servers.
func NewBlubberRPCClient ¶
func NewBlubberRPCClient(uri, cert, key, cacert string, insecure bool) ( *BlubberRPCClient, error)
Create a new blubber client connecting to the given server.
uri should be an URI pointing to the desired server and port. cert, key and cacert should be the path of the X.509 client certificate, private key and CA certificate, respectively. The insecure flag can be used to disable encryption (only for testing and if the server is also running in insecure mode).
func (*BlubberRPCClient) CopyBlob ¶
func (self *BlubberRPCClient) CopyBlob(source BlockSource) error
Instruct the server to copy the blob with the given ID from the given server. It will overwrite any local variants of the blob.
func (*BlubberRPCClient) DeleteBlob ¶
func (self *BlubberRPCClient) DeleteBlob(id BlockId) error
Delete the blob with the given ID.
func (*BlubberRPCClient) GetRPCClient ¶
func (self *BlubberRPCClient) GetRPCClient() *rpc.Client
Get the underlying RPC client from the connection. It can sometimes be helpful, e.g. if the remote interface is double-bound.
func (*BlubberRPCClient) RetrieveBlob ¶
func (self *BlubberRPCClient) RetrieveBlob(id BlockId) ( ret BlockWithData, err error)
Retrieve all data contained in the specified blob. If you want to retrieve more data than will fit into memory, use the HTTP client library instead.
func (*BlubberRPCClient) StatBlob ¶
func (self *BlubberRPCClient) StatBlob(id BlockId) ( stat BlubberStat, err error)
Retrieve status information about the blob with the given ID.
func (*BlubberRPCClient) StoreBlob ¶
func (self *BlubberRPCClient) StoreBlob(data BlockWithData) ( err error)
Send a blob to the server and store it under the given blob ID.
If you want to submit data that is too large to fit into memory, use the HTTP client library instead.
type BlubberStoreClient ¶
type BlubberStoreClient struct {
// contains filtered or unexported fields
}
BlubberStore client which handles replication, quorums and server lookups transparently.
func NewBlubberStoreClient ¶
func NewBlubberStoreClient(uri, cert, key, cacert string, insecure bool, errorlog chan error) (*BlubberStoreClient, error)
Create a new Blubber Store client which will talk independently to the directory service, discover and talk to blubber backends, and handle timeouts etc.
uri is an URI pointing to the blubber directory. cert and key are the path to a PEM encoded X.509 certificate and private key. cacert is the path to a PEM encoded X.509 CA certificate. Setting insecure to true disables the use of certificates. errlog is a channel for reporting non-fatal errors which occur during operation back to the clients.
func (*BlubberStoreClient) RetrieveBlob ¶
func (b *BlubberStoreClient) RetrieveBlob(id []byte) (io.Reader, error)
Read the blob with the given ID. The blob will be looked up for you in the block directory and a read request will be made to an appropriate server.
func (*BlubberStoreClient) StatBlob ¶
func (b *BlubberStoreClient) StatBlob(id []byte) (*BlockStatus, error)
Gather information about the status of the blob with the given ID. Returns an object representing the status of the requested blob or an error which may have occurred when trying to gather the requested information.
func (*BlubberStoreClient) StoreBlock ¶
func (b *BlubberStoreClient) StoreBlock(id []byte, data io.Reader, overwrite bool, replication int) error
Write the given data to a blob with the given ID, overwriting it if specified. This will ensure that there are more than replication/2 copies of the block. overwrite=false will disable lookups for current block holders.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
blubber-blockcat
Simple example program for uploading arbitrary files to blubberstore.
|
Simple example program for uploading arbitrary files to blubberstore. |
|
blubber-blockstat
Simple example program for uploading arbitrary files to blubberstore.
|
Simple example program for uploading arbitrary files to blubberstore. |
|
blubber-copy
Command line interface to initiate copies of blocks between servers.
|
Command line interface to initiate copies of blocks between servers. |
|
blubber-getfreehosts
Command line interface to query the list of free hosts.
|
Command line interface to query the list of free hosts. |
|
blubber-upload
Simple example program for uploading arbitrary files to blubberstore.
|
Simple example program for uploading arbitrary files to blubberstore. |