shipper

package
v0.26.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 9, 2022 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Overview

Package shipper detects directories on the local file system and uploads them to a block storage.

Index

Constants

View Source
const (
	// MetaFilename is the known JSON filename for meta information.
	MetaFilename = "thanos.shipper.json"

	// MetaVersion1 represents 1 version of meta.
	MetaVersion1 = 1
)

Variables

This section is empty.

Functions

func WriteMetaFile

func WriteMetaFile(logger log.Logger, dir string, meta *Meta) error

WriteMetaFile writes the given meta into <dir>/thanos.shipper.json.

Types

type Meta

type Meta struct {
	Version  int         `json:"version"`
	Uploaded []ulid.ULID `json:"uploaded"`
}

Meta defines the format thanos.shipper.json file that the shipper places in the data directory.

func ReadMetaFile

func ReadMetaFile(dir string) (*Meta, error)

ReadMetaFile reads the given meta from <dir>/thanos.shipper.json.

type Shipper

type Shipper struct {
	// contains filtered or unexported fields
}

Shipper watches a directory for matching files and directories and uploads them to a remote data store.

func New

func New(
	logger log.Logger,
	r prometheus.Registerer,
	dir string,
	bucket objstore.Bucket,
	lbls func() labels.Labels,
	source metadata.SourceType,
	uploadCompacted bool,
	allowOutOfOrderUploads bool,
	hashFunc metadata.HashFunc,
) *Shipper

New creates a new shipper that detects new TSDB blocks in dir and uploads them to remote if necessary. It attaches the Thanos metadata section in each meta JSON file. If uploadCompacted is enabled, it also uploads compacted blocks which are already in filesystem.

func (*Shipper) Sync

func (s *Shipper) Sync(ctx context.Context) (uploaded int, err error)

Sync performs a single synchronization, which ensures all non-compacted local blocks have been uploaded to the object bucket once.

If uploaded.

It is not concurrency-safe, however it is compactor-safe (running concurrently with compactor is ok).

func (*Shipper) Timestamps

func (s *Shipper) Timestamps() (minTime, maxSyncTime int64, err error)

Timestamps returns the minimum timestamp for which data is available and the highest timestamp of blocks that were successfully uploaded.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL