maintenance

package
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 16, 2024 License: Apache-2.0 Imports: 28 Imported by: 2

Documentation

Overview

Package maintenance manages automatic repository maintenance.

Index

Constants

View Source
const (
	TaskSnapshotGarbageCollection    = "snapshot-gc"
	TaskDeleteOrphanedBlobsQuick     = "quick-delete-blobs"
	TaskDeleteOrphanedBlobsFull      = "full-delete-blobs"
	TaskRewriteContentsQuick         = "quick-rewrite-contents"
	TaskRewriteContentsFull          = "full-rewrite-contents"
	TaskDropDeletedContentsFull      = "full-drop-deleted-content"
	TaskIndexCompaction              = "index-compaction"
	TaskExtendBlobRetentionTimeFull  = "extend-blob-retention-time"
	TaskCleanupLogs                  = "cleanup-logs"
	TaskEpochAdvance                 = "advance-epoch"
	TaskEpochDeleteSupersededIndexes = "delete-superseded-epoch-indexes"
	TaskEpochCleanupMarkers          = "cleanup-epoch-markers"
	TaskEpochGenerateRange           = "generate-epoch-range-index"
	TaskEpochCompactSingle           = "compact-single-epoch"
)

Task IDs.

Variables

View Source
var (
	// SafetyNone has safety parameters which allow full garbage collection without unnecessary
	// delays, but it is safe only if no other kopia clients are running and storage backend is
	// strongly consistent.
	SafetyNone = SafetyParameters{
		BlobDeleteMinAge:                 0,
		DropContentFromIndexExtraMargin:  0,
		MarginBetweenSnapshotGC:          0,
		MinContentAgeSubjectToGC:         0,
		RewriteMinAge:                    0,
		SessionExpirationAge:             0,
		RequireTwoGCCycles:               false,
		DisableEventualConsistencySafety: true,
	}

	// SafetyFull has default safety parameters which allow safe GC concurrent with snapshotting
	// by other Kopia clients.
	SafetyFull = SafetyParameters{
		BlobDeleteMinAge:                24 * time.Hour,
		DropContentFromIndexExtraMargin: time.Hour,
		MarginBetweenSnapshotGC:         4 * time.Hour,
		MinContentAgeSubjectToGC:        24 * time.Hour,
		RewriteMinAge:                   2 * time.Hour,
		SessionExpirationAge:            96 * time.Hour,
		RequireTwoGCCycles:              true,
		MinRewriteToOrphanDeletionDelay: time.Hour,
	}
)

Supported safety levels.

Functions

func CheckExtendRetention added in v0.14.0

func CheckExtendRetention(ctx context.Context, blobCfg format.BlobStorageConfiguration, p *Params) error

CheckExtendRetention verifies if extension can be enabled due to maintenance and blob parameters.

func CleanupLogs added in v0.9.0

CleanupLogs deletes old logs blobs beyond certain age, total size or count.

func DeleteUnreferencedBlobs

func DeleteUnreferencedBlobs(ctx context.Context, rep repo.DirectRepositoryWriter, opt DeleteUnreferencedBlobsOptions, safety SafetyParameters) (int, error)

DeleteUnreferencedBlobs deletes o was created after maintenance startederenced by index entries.

func DropDeletedContents

func DropDeletedContents(ctx context.Context, rep repo.DirectRepositoryWriter, dropDeletedBefore time.Time, safety SafetyParameters) error

DropDeletedContents rewrites indexes while dropping deleted contents above certain age.

func ExtendBlobRetentionTime added in v0.14.0

func ExtendBlobRetentionTime(ctx context.Context, rep repo.DirectRepositoryWriter, opt ExtendBlobRetentionTimeOptions) (int, error)

ExtendBlobRetentionTime extends the retention time of all relevant blobs managed by storage engine with Object Locking enabled.

func HasParams

func HasParams(ctx context.Context, rep repo.Repository) (bool, error)

HasParams determines whether repository-wide maintenance parameters have been set.

func IsOwnedByThisUser added in v0.8.4

func IsOwnedByThisUser(ctx context.Context, rep repo.Repository) (bool, error)

IsOwnedByThisUser determines whether current user is the maintenance owner.

func ReportRun

func ReportRun(ctx context.Context, rep repo.DirectRepositoryWriter, taskType TaskType, s *Schedule, run func() error) error

ReportRun reports timing of a maintenance run and persists it in repository.

func RewriteContents

RewriteContents rewrites contents according to provided criteria and creates new blobs and index entries to point at them.

func Run

func Run(ctx context.Context, runParams RunParameters, safety SafetyParameters) error

Run performs maintenance activities for a repository.

func RunExclusive

func RunExclusive(ctx context.Context, rep repo.DirectRepositoryWriter, mode Mode, force bool, cb func(ctx context.Context, runParams RunParameters) error) error

RunExclusive runs the provided callback if the maintenance is owned by local user and lock can be acquired. Lock is passed to the function, which ensures that every call to Run() is within the exclusive context.

func SetParams

func SetParams(ctx context.Context, rep repo.RepositoryWriter, par *Params) error

SetParams sets the maintenance parameters.

func SetSchedule

func SetSchedule(ctx context.Context, rep repo.DirectRepositoryWriter, s *Schedule) error

SetSchedule updates scheduled maintenance times.

func TimeToAttemptNextMaintenance added in v0.9.5

func TimeToAttemptNextMaintenance(ctx context.Context, rep repo.DirectRepository) (time.Time, error)

TimeToAttemptNextMaintenance returns the time when we should attempt next maintenance. if the maintenance is not owned by this user, returns time.Time{}.

Types

type CycleParams

type CycleParams struct {
	Enabled  bool          `json:"enabled"`
	Interval time.Duration `json:"interval"`
}

CycleParams specifies parameters for a maintenance cycle (quick or full).

type DeleteUnreferencedBlobsOptions

type DeleteUnreferencedBlobsOptions struct {
	Parallel     int
	Prefix       blob.ID
	DryRun       bool
	NotAfterTime time.Time
}

DeleteUnreferencedBlobsOptions provides option for blob garbage collection algorithm.

type ExtendBlobRetentionTimeOptions added in v0.14.0

type ExtendBlobRetentionTimeOptions struct {
	Parallel int
	DryRun   bool
}

ExtendBlobRetentionTimeOptions provides options for extending blob retention algorithm.

type LogRetentionOptions added in v0.9.0

type LogRetentionOptions struct {
	MaxTotalSize int64            `json:"maxTotalSize"`
	MaxCount     int              `json:"maxCount"`
	MaxAge       time.Duration    `json:"maxAge"`
	DryRun       bool             `json:"-"`
	TimeFunc     func() time.Time `json:"-"`
}

LogRetentionOptions provides options for logs retention.

func (LogRetentionOptions) OrDefault added in v0.9.0

OrDefault returns default LogRetentionOptions.

type Mode

type Mode string

Mode describes the mode of maintenance to perform.

const (
	ModeNone  Mode = "none"
	ModeQuick Mode = "quick"
	ModeFull  Mode = "full"
	ModeAuto  Mode = "auto" // run either quick of full if required by schedule
)

Supported maintenance modes.

type NotOwnedError

type NotOwnedError struct {
	Owner string
}

NotOwnedError is returned when maintenance cannot run because it is owned by another user.

func (NotOwnedError) Error

func (e NotOwnedError) Error() string

type Params

type Params struct {
	Owner string `json:"owner"`

	QuickCycle CycleParams `json:"quick"`
	FullCycle  CycleParams `json:"full"`

	LogRetention LogRetentionOptions `json:"logRetention"`

	ExtendObjectLocks bool `json:"extendObjectLocks"`
}

Params is a JSON-serialized maintenance configuration stored in a repository.

func DefaultParams

func DefaultParams() Params

DefaultParams represents default values of maintenance parameters.

func GetParams

func GetParams(ctx context.Context, rep repo.Repository) (*Params, error)

GetParams returns repository-wide maintenance parameters.

type RewriteContentsOptions

type RewriteContentsOptions struct {
	Parallel       int
	ContentIDs     []content.ID
	ContentIDRange content.IDRange
	PackPrefix     blob.ID
	ShortPacks     bool
	FormatVersion  int
	DryRun         bool
}

RewriteContentsOptions provides options for RewriteContents.

type RunInfo

type RunInfo struct {
	Start   time.Time `json:"start"`
	End     time.Time `json:"end"`
	Success bool      `json:"success,omitempty"`
	Error   string    `json:"error,omitempty"`
}

RunInfo represents information about a single run of a maintenance task.

type RunParameters

type RunParameters struct {
	Mode Mode

	Params *Params

	// timestamp of the last update of maintenance schedule blob
	MaintenanceStartTime time.Time
	// contains filtered or unexported fields
}

RunParameters passes essential parameters for maintenance. It is generated by RunExclusive and can't be create outside of its package and is required to ensure all maintenance tasks run under an exclusive lock.

type SafetyParameters added in v0.8.0

type SafetyParameters struct {
	// Do not rewrite contents younger than this age.
	RewriteMinAge time.Duration

	// Snapshot GC: MinContentAgeSubjectToGC is the minimum age of content to be subject to garbage collection.
	MinContentAgeSubjectToGC time.Duration

	// MarginBetweenSnapshotGC is the minimal amount of time that must pass between snapshot
	// GC cycles to allow all in-flight snapshots during earlier GC to be flushed and
	// visible to a following GC. The uploader will automatically create a checkpoint every 45 minutes,
	// so ~1 hour should be enough but we're setting this to a higher conservative value for extra safety.
	MarginBetweenSnapshotGC time.Duration

	// RequireTwoGCCycles indicates that two GC cycles are required.
	RequireTwoGCCycles bool

	// DisableEventualConsistencySafety disables wait time to allow settling of eventually-consistent writes in blob stores.
	DisableEventualConsistencySafety bool

	// DropContentFromIndexExtraMargin is the amount of margin time before dropping deleted contents from indices.
	DropContentFromIndexExtraMargin time.Duration

	// Blob GC: Delete unused blobs above this age.
	BlobDeleteMinAge time.Duration

	// Blob GC: Drop incomplete session blobs above this age.
	SessionExpirationAge time.Duration

	// Minimum time that must pass after content rewrite before we delete orphaned blobs.
	MinRewriteToOrphanDeletionDelay time.Duration
}

SafetyParameters specifies timing parameters that affect safety of maintenance.

type Schedule

type Schedule struct {
	NextFullMaintenanceTime  time.Time `json:"nextFullMaintenance"`
	NextQuickMaintenanceTime time.Time `json:"nextQuickMaintenance"`

	Runs map[TaskType][]RunInfo `json:"runs"`
}

Schedule keeps track of scheduled maintenance times.

func GetSchedule

func GetSchedule(ctx context.Context, rep repo.DirectRepository) (*Schedule, error)

GetSchedule gets the scheduled maintenance times.

func (*Schedule) ReportRun

func (s *Schedule) ReportRun(taskType TaskType, info RunInfo)

ReportRun adds the provided run information to the history and discards oldest entried.

type TaskType added in v0.8.0

type TaskType string

TaskType identifies the type of a maintenance task.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL