Documentation
¶
Index ¶
- Constants
- Variables
- func Clone(ctx context.Context, srcCS, sinkCS chunks.ChunkStore, ...) error
- func Pull(ctx context.Context, srcCS, sinkCS chunks.ChunkStore, walkAddrs WalkAddrs, ...) error
- func PullWithoutBatching(ctx context.Context, srcCS, sinkCS chunks.ChunkStore, walkAddrs WalkAddrs, ...) error
- type CloneTableFileEvent
- type CmpChnkAndRefs
- type FilledWriters
- type PullProgress
- type Puller
- type Stats
- type TableFileEvent
- type WalkAddrs
Constants ¶
const ( Listed = iota DownloadStart DownloadStats DownloadSuccess DownloadFailed )
Variables ¶
var ErrDBUpToDate = errors.New("the database does not need to be pulled as it's already up to date")
ErrDBUpToDate is the error code returned from NewPuller in the event that there is no work to do.
var ErrIncompatibleSourceChunkStore = errors.New("the chunk store of the source database does not implement NBSCompressedChunkStore.")
ErrIncompatibleSourceChunkStore is the error code returned from NewPuller in the event that the source ChunkStore does not implement `NBSCompressedChunkStore`.
var ErrNoData = errors.New("no data")
Functions ¶
func Clone ¶
func Clone(ctx context.Context, srcCS, sinkCS chunks.ChunkStore, eventCh chan<- TableFileEvent) error
func Pull ¶
func Pull(ctx context.Context, srcCS, sinkCS chunks.ChunkStore, walkAddrs WalkAddrs, sourceHash hash.Hash, progressCh chan PullProgress) error
Pull objects that descend from sourceHash from srcDB to sinkDB.
func PullWithoutBatching ¶
func PullWithoutBatching(ctx context.Context, srcCS, sinkCS chunks.ChunkStore, walkAddrs WalkAddrs, sourceHash hash.Hash, progressCh chan PullProgress) error
PullWithoutBatching effectively removes the batching of chunk retrieval done on each level of the tree. This means all chunks from one level of the tree will be retrieved from the underlying chunk store in one call, which pushes the optimization problem down to the chunk store which can make smarter decisions.
Types ¶
type CloneTableFileEvent ¶
type CloneTableFileEvent int
type CmpChnkAndRefs ¶
type CmpChnkAndRefs struct {
// contains filtered or unexported fields
}
CmpChnkAndRefs holds a CompressedChunk and all of it's references
type FilledWriters ¶
type FilledWriters struct {
// contains filtered or unexported fields
}
FilledWriters store CmpChunkTableWriter that have been filled and are ready to be flushed. In the future will likely add the md5 of the data to this structure to be used to verify table upload calls.
type PullProgress ¶
type PullProgress struct {
DoneCount, KnownCount, ApproxWrittenBytes uint64
}
type Puller ¶
type Puller struct {
// contains filtered or unexported fields
}
Puller is used to sync data between to Databases
func NewPuller ¶
func NewPuller(ctx context.Context, tempDir string, chunksPerTF int, srcCS, sinkCS chunks.ChunkStore, walkAddrs WalkAddrs, rootChunkHash hash.Hash, statsCh chan Stats) (*Puller, error)
NewPuller creates a new Puller instance to do the syncing. If a nil puller is returned without error that means that there is nothing to pull and the sinkDB is already up to date.
type TableFileEvent ¶
type TableFileEvent struct {
EventType CloneTableFileEvent
TableFiles []nbs.TableFile
Stats []iohelp.ReadStats
}
Source Files