repository

package
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 30, 2023 License: BSD-2-Clause Imports: 34 Imported by: 2

Documentation

Overview

Package repository implements a restic repository on top of a backend. In the following the abstractions used for this package are listed. More information can be found in the restic design document.

File

A file is a named handle for some data saved in the backend. For the local backend, this corresponds to actual files saved to disk. Usually, the SHA256 hash of the content is used for a file's name (hexadecimal, in lower-case ASCII characters). An exception is the file `config`. Most files are encrypted before being saved in a backend. This means that the name is the hash of the ciphertext.

Blob

A blob is a number of bytes that has a type (data or tree). Blobs are identified by an ID, which is the SHA256 hash of the blobs' contents. One or more blobs are bundled together in a Pack and then saved to the backend. Blobs are always encrypted before being bundled in a Pack.

Pack

A Pack is a File in the backend that contains one or more (encrypted) blobs, followed by a header at the end of the Pack. The header is encrypted and contains the ID, type, length and offset for each blob contained in the Pack.

Index

Constants

View Source
const DefaultPackSize = 16 * 1024 * 1024
View Source
const MaxPackSize = 128 * 1024 * 1024
View Source
const MaxStreamBufferSize = 4 * 1024 * 1024
View Source
const MinPackSize = 4 * 1024 * 1024
View Source
const TestChunkerPol = chunker.Pol(0x3DA3358B4DC173)

Variables

View Source
var (
	// ErrNoKeyFound is returned when no key for the repository could be decrypted.
	ErrNoKeyFound = errors.New("wrong password or no key found")

	// ErrMaxKeysReached is returned when the maximum number of keys was checked and no key could be found.
	ErrMaxKeysReached = errors.New("maximum number of keys reached")
)
View Source
var (
	// KDFTimeout specifies the maximum runtime for the KDF.
	KDFTimeout = 500 * time.Millisecond

	// KDFMemory limits the memory the KDF is allowed to use.
	KDFMemory = 60
)
View Source
var Params *crypto.Params

Params tracks the parameters used for the KDF. If not set, it will be calibrated on the first run of AddKey().

Functions

func BenchmarkAllVersions

func BenchmarkAllVersions(b *testing.B, bench VersionedBenchmark)

func Repack

func Repack(ctx context.Context, repo restic.Repository, dstRepo restic.Repository, packs restic.IDSet, keepBlobs repackBlobSet, p *progress.Counter) (obsoletePacks restic.IDSet, err error)

Repack takes a list of packs together with a list of blobs contained in these packs. Each pack is loaded and the blobs listed in keepBlobs is saved into a new pack. Returned is the list of obsolete packs which can then be removed.

The map keepBlobs is modified by Repack, it is used to keep track of which blobs have been processed.

func StreamPack

func StreamPack(ctx context.Context, beLoad BackendLoadFn, key *crypto.Key, packID restic.ID, blobs []restic.Blob, handleBlobFn func(blob restic.BlobHandle, buf []byte, err error) error) error

StreamPack loads the listed blobs from the specified pack file. The plaintext blob is passed to the handleBlobFn callback or an error if decryption failed or the blob hash does not match. In case of download errors handleBlobFn might be called multiple times for the same blob. If the callback returns an error, then StreamPack will abort and not retry it.

func TestAllVersions

func TestAllVersions(t *testing.T, test VersionedTest)

func TestBackend

func TestBackend(_ testing.TB) backend.Backend

TestBackend returns a fully configured in-memory backend.

func TestOpenLocal

func TestOpenLocal(t testing.TB, dir string) (r restic.Repository)

TestOpenLocal opens a local repository.

func TestRepository

func TestRepository(t testing.TB) restic.Repository

TestRepository returns a repository initialized with a test password on an in-memory backend. When the environment variable RESTIC_TEST_REPO is set to a non-existing directory, a local backend is created there and this is used instead. The directory is not removed, but left there for inspection.

func TestRepositoryWithBackend

func TestRepositoryWithBackend(t testing.TB, be backend.Backend, version uint) restic.Repository

TestRepositoryWithBackend returns a repository initialized with a test password. If be is nil, an in-memory backend is used. A constant polynomial is used for the chunker and low-security test parameters.

func TestRepositoryWithVersion

func TestRepositoryWithVersion(t testing.TB, version uint) restic.Repository

func TestUseLowSecurityKDFParameters

func TestUseLowSecurityKDFParameters(t logger)

TestUseLowSecurityKDFParameters configures low-security KDF parameters for testing.

func ZeroChunk

func ZeroChunk() restic.ID

ZeroChunk computes and returns (cached) the ID of an all-zero chunk with size chunker.MinSize

Types

type BackendLoadFn

type BackendLoadFn func(ctx context.Context, h backend.Handle, length int, offset int64, fn func(rd io.Reader) error) error

type CompressionMode

type CompressionMode uint

CompressionMode configures if data should be compressed.

const (
	CompressionAuto    CompressionMode = 0
	CompressionOff     CompressionMode = 1
	CompressionMax     CompressionMode = 2
	CompressionInvalid CompressionMode = 3
)

Constants for the different compression levels.

func (*CompressionMode) Set

func (c *CompressionMode) Set(s string) error

Set implements the method needed for pflag command flag parsing.

func (*CompressionMode) String

func (c *CompressionMode) String() string

func (*CompressionMode) Type

func (c *CompressionMode) Type() string

type Key

type Key struct {
	Created  time.Time `json:"created"`
	Username string    `json:"username"`
	Hostname string    `json:"hostname"`

	KDF  string `json:"kdf"`
	N    int    `json:"N"`
	R    int    `json:"r"`
	P    int    `json:"p"`
	Salt []byte `json:"salt"`
	Data []byte `json:"data"`
	// contains filtered or unexported fields
}

Key represents an encrypted master key for a repository.

func AddKey

func AddKey(ctx context.Context, s *Repository, password, username, hostname string, template *crypto.Key) (*Key, error)

AddKey adds a new key to an already existing repository.

func LoadKey

func LoadKey(ctx context.Context, s *Repository, id restic.ID) (k *Key, err error)

LoadKey loads a key from the backend.

func OpenKey

func OpenKey(ctx context.Context, s *Repository, id restic.ID, password string) (*Key, error)

OpenKey tries do decrypt the key specified by name with the given password.

func SearchKey

func SearchKey(ctx context.Context, s *Repository, password string, maxKeys int, keyHint string) (k *Key, err error)

SearchKey tries to decrypt at most maxKeys keys in the backend with the given password. If none could be found, ErrNoKeyFound is returned. When maxKeys is reached, ErrMaxKeysReached is returned. When setting maxKeys to zero, all keys in the repo are checked.

func (Key) ID

func (k Key) ID() restic.ID

ID returns an identifier for the key.

func (*Key) String

func (k *Key) String() string

func (*Key) Valid

func (k *Key) Valid() bool

Valid tests whether the mac and encryption keys are valid (i.e. not zero)

type Options

type Options struct {
	Compression CompressionMode
	PackSize    uint
}

type Packer

type Packer struct {
	*pack.Packer
	// contains filtered or unexported fields
}

Packer holds a pack.Packer together with a hash writer.

type Repository

type Repository struct {
	Cache *cache.Cache
	// contains filtered or unexported fields
}

Repository is used to access a repository in a backend.

func New

func New(be backend.Backend, opts Options) (*Repository, error)

New returns a new repository with backend be.

func (*Repository) Backend

func (r *Repository) Backend() backend.Backend

Backend returns the backend for the repository.

func (*Repository) Close

func (r *Repository) Close() error

Close closes the repository by closing the backend.

func (*Repository) Config

func (r *Repository) Config() restic.Config

Config returns the repository configuration.

func (*Repository) Connections

func (r *Repository) Connections() uint

func (*Repository) CreateIndexFromPacks

func (r *Repository) CreateIndexFromPacks(ctx context.Context, packsize map[restic.ID]int64, p *progress.Counter) (invalid restic.IDs, err error)

CreateIndexFromPacks creates a new index by reading all given pack files (with sizes). The index is added to the MasterIndex but not marked as finalized. Returned is the list of pack files which could not be read.

func (*Repository) Delete

func (r *Repository) Delete(ctx context.Context) error

Delete calls backend.Delete() if implemented, and returns an error otherwise.

func (*Repository) DisableAutoIndexUpdate

func (r *Repository) DisableAutoIndexUpdate()

DisableAutoIndexUpdate deactives the automatic finalization and upload of new indexes once these are full

func (*Repository) Flush

func (r *Repository) Flush(ctx context.Context) error

Flush saves all remaining packs and the index

func (*Repository) Index

func (r *Repository) Index() restic.MasterIndex

Index returns the currently used MasterIndex.

func (*Repository) Init

func (r *Repository) Init(ctx context.Context, version uint, password string, chunkerPolynomial *chunker.Pol) error

Init creates a new master key with the supplied password, initializes and saves the repository config.

func (*Repository) Key

func (r *Repository) Key() *crypto.Key

Key returns the current master key.

func (*Repository) KeyID

func (r *Repository) KeyID() restic.ID

KeyID returns the id of the current key in the backend.

func (*Repository) List

func (r *Repository) List(ctx context.Context, t restic.FileType, fn func(restic.ID, int64) error) error

List runs fn for all files of type t in the repo.

func (*Repository) ListPack

func (r *Repository) ListPack(ctx context.Context, id restic.ID, size int64) ([]restic.Blob, uint32, error)

ListPack returns the list of blobs saved in the pack id and the length of the pack header.

func (*Repository) LoadBlob

func (r *Repository) LoadBlob(ctx context.Context, t restic.BlobType, id restic.ID, buf []byte) ([]byte, error)

LoadBlob loads a blob of type t from the repository. It may use all of buf[:cap(buf)] as scratch space.

func (*Repository) LoadIndex

func (r *Repository) LoadIndex(ctx context.Context, p *progress.Counter) error

LoadIndex loads all index files from the backend in parallel and stores them

func (*Repository) LoadUnpacked

func (r *Repository) LoadUnpacked(ctx context.Context, t restic.FileType, id restic.ID) ([]byte, error)

LoadUnpacked loads and decrypts the file with the given type and ID.

func (*Repository) LookupBlobSize

func (r *Repository) LookupBlobSize(id restic.ID, tpe restic.BlobType) (uint, bool)

LookupBlobSize returns the size of blob id.

func (*Repository) PackSize

func (r *Repository) PackSize() uint

PackSize return the target size of a pack file when uploading

func (*Repository) SaveBlob

func (r *Repository) SaveBlob(ctx context.Context, t restic.BlobType, buf []byte, id restic.ID, storeDuplicate bool) (newID restic.ID, known bool, size int, err error)

SaveBlob saves a blob of type t into the repository. It takes care that no duplicates are saved; this can be overwritten by setting storeDuplicate to true. If id is the null id, it will be computed and returned. Also returns if the blob was already known before. If the blob was not known before, it returns the number of bytes the blob occupies in the repo (compressed or not, including encryption overhead).

func (*Repository) SaveUnpacked

func (r *Repository) SaveUnpacked(ctx context.Context, t restic.FileType, p []byte) (id restic.ID, err error)

SaveUnpacked encrypts data and stores it in the backend. Returned is the storage hash.

func (*Repository) SearchKey

func (r *Repository) SearchKey(ctx context.Context, password string, maxKeys int, keyHint string) error

SearchKey finds a key with the supplied password, afterwards the config is read and parsed. It tries at most maxKeys key files in the repo.

func (*Repository) SetDryRun

func (r *Repository) SetDryRun()

SetDryRun sets the repo backend into dry-run mode.

func (*Repository) SetIndex

func (r *Repository) SetIndex(i restic.MasterIndex) error

SetIndex instructs the repository to use the given index.

func (*Repository) StartPackUploader

func (r *Repository) StartPackUploader(ctx context.Context, wg *errgroup.Group)

func (*Repository) UseCache

func (r *Repository) UseCache(c *cache.Cache)

UseCache replaces the backend with the wrapped cache.

type SavePacker

type SavePacker interface {
	// contains filtered or unexported methods
}

SavePacker implements saving a pack in the repository.

type VersionedBenchmark

type VersionedBenchmark func(b *testing.B, version uint)

type VersionedTest

type VersionedTest func(t *testing.T, version uint)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL