Documentation
¶
Index ¶
- Variables
- type BlbWatcher
- type Config
- type SqliteDB
- func (s *SqliteDB) Close() (err error)
- func (s *SqliteDB) ConfirmDeletion(blobs []blb.BlobID) (err error)
- func (s *SqliteDB) DeleteIfExpires(lifetime time.Duration) ([]blb.BlobID, error)
- func (s *SqliteDB) Get(id blb.BlobID) (creation time.Time, err error)
- func (s *SqliteDB) GetDeleted() ([]blb.BlobID, error)
- func (s *SqliteDB) Put(id blb.BlobID, creation time.Time) (err error)
- func (s *SqliteDB) Rand() (id blb.BlobID, err error)
Constants ¶
This section is empty.
Variables ¶
var DefaultConfig = Config{ TableFile: "watchblb.db", WriteSize: 1 * 1024 * 1024, ReplFactor: 3, WriteInterval: time.Minute, ReadInterval: 20 * time.Second, CleanInterval: 5 * time.Minute, BlobLifetime: 1440 * time.Hour, }
DefaultConfig includes default configuration parameters.
Functions ¶
This section is empty.
Types ¶
type BlbWatcher ¶
type BlbWatcher struct {
// contains filtered or unexported fields
}
BlbWatcher watches the status of a blb cluster, proactively verifies its healthiness via background read/write traffic, and triggers alerts when something goes wrong.
func NewBlbWatcher ¶
func NewBlbWatcher(cfg Config) *BlbWatcher
NewBlbWatcher creates a new BlbWatcher with the given client.
func (*BlbWatcher) Start ¶
func (bw *BlbWatcher) Start()
Start starts the BlbWatcher. This is blocking.
type Config ¶
type Config struct {
// Master addresses to be used by blb client.
Cluster string
// The persistent file for the sqlite table.
TableFile string
// The byte size of each write.
WriteSize int64
// Replication factor.
ReplFactor int
// How often to perform writes.
WriteInterval time.Duration
// How often to perform reads.
ReadInterval time.Duration
// How often to check blob lifetime.
CleanInterval time.Duration
// For how long a blob lives before getting removed.
BlobLifetime time.Duration
}
Config specifies various parameters.
type SqliteDB ¶
type SqliteDB struct {
// contains filtered or unexported fields
}
SqliteDB is a persistent DB backed by sqlite for storing blob ids and creation times.
func NewSqliteDB ¶
NewSqliteDB creates a SqliteDB backed by the file located at 'path'.
func (*SqliteDB) Close ¶
Close closes the db. All errors will be logged and the last error is returned.
func (*SqliteDB) ConfirmDeletion ¶
ConfirmDeletion removes expired blobs in 'blobs'. All errors are logged and the last error is returned.
func (*SqliteDB) DeleteIfExpires ¶
DeleteIfExpires flag expired blobs for deletion. A blob expires if creation + lifetime < time.Now().
The workflow is:
(1) Flag expired blobs for deletion. (2) Return such blobs to watchblb. (3) After watchblb removes the blobs from the blb cluster, it acknowledges the success by calling 'ConfirmDeletion', which deletes the flagged blobs from the db. (4) If watchblb fails before step (4), it needs to replay the deletion operations upon restart.
By doing this, we guarantee that the information stored in the db is consistent with that in the blb cluster, and thus there is no blob leakage during the process of removing expired blobs.
Note however, there is no guarantee on no blob leakage during blob creation -- if watchblb crashes after creating the blob in the blb cluster but before logging it in the db, the blob is leaked.
func (*SqliteDB) GetDeleted ¶
GetDeleted retrieves blobs that are flagged for deletion.
Source Files