juju: github.com/juju/juju/state/backups Index | Files | Directories

package backups

import "github.com/juju/juju/state/backups"

Package backups contains all the stand-alone backup-related functionality for juju state. That functionality is encapsulated by the backups.Backups type. The package also exposes a few key helpers and components.

Backups are not a part of juju state nor of normal state operations. However, they certainly are tightly coupled with state (the very subject of backups). This puts backups in an odd position, particularly with regard to the storage of backup metadata and archives.

As noted above backups are about state but not a part of state. So exposing backup-related methods on State would imply the wrong thing. Thus most of the functionality here is defined at a high level without relation to state. A few low-level parts or helpers are exposed as functions to which you pass a state value. Those are kept to a minimum.

Note that state (and juju as a whole) currently does not have a persistence layer abstraction to facilitate separating different persistence needs and implementations. As a consequence, state's data, whether about how an model should look or about existing resources within an model, is dumped essentially straight into State's mongo connection. The code in the state package does not make any distinction between the two (nor does the package clearly distinguish between state-related abstractions and state-related data).

Backups add yet another category, merely taking advantage of State's mongo for storage. In the interest of making the distinction clear, among other reasons, backups uses its own database under state's mongo connection.


Package Files

archive.go backups.go backups_linux.go create.go db.go exec.go files.go metadata.go metadata_linux.go params.go restore.go storage.go


const (
    // FilenamePrefix is the prefix used for backup archive files.
    FilenamePrefix = "juju-backup-"

    // FilenameTemplate is used with time.Time.Format to generate a filename.
    FilenameTemplate = FilenamePrefix + "20060102-150405.tar.gz"
const (
    TempFilename = "juju-backup.tar.gz"
const UnknownString = "<unknown>"

UnknownString is a marker value for string fields with unknown values.


var UnknownInt64 = int64(math.MaxInt64)

UnknownInt64 is a marker value for int64 fields with unknown values.

var UnknownVersion = version.MustParse("9999.9999.9999")

UnknownVersion is a marker value for version fields with unknown values.

func GetFilesToBackUp Uses

func GetFilesToBackUp(rootDir string, paths *Paths, oldmachine string) ([]string, error)

GetFilesToBackUp returns the paths that should be included in the backup archive.

func NewStorage Uses

func NewStorage(st DB) filestorage.FileStorage

NewStorage returns a new FileStorage to use for storing backup archives (and metadata).

func PrepareMachineForRestore Uses

func PrepareMachineForRestore(mongoVersion mongo.Version) error

PrepareMachineForRestore deletes all files from the re-bootstrapped machine that are to be replaced by the backup and recreates those directories that are to contain new files; this is to avoid possible mixup from new/old files that lead to an inconsistent restored state machine.

func StoreArchive Uses

func StoreArchive(stor filestorage.FileStorage, meta *Metadata, file io.Reader) error

StoreArchive sends the backup archive and its metadata to storage. It also sets the metadata's ID and Stored values.

type ArchiveData Uses

type ArchiveData struct {
    // contains filtered or unexported fields

ArchiveData is a wrapper around a the uncompressed data in a backup archive file. It provides access to the content of the archive. While ArchiveData provides useful functionality, it may not be appropriate for large archives. The contents of the archive are kept in-memory, so large archives could be too taxing on the host. In that case consider using ArchiveWorkspace instead.

func NewArchiveData Uses

func NewArchiveData(data []byte) *ArchiveData

NewArchiveData builds a new archive data wrapper for the given uncompressed data.

func NewArchiveDataReader Uses

func NewArchiveDataReader(r io.Reader) (*ArchiveData, error)

NewArchiveReader returns a new archive data wrapper for the data in the provided reader. Note that the entire archive will be read into memory and kept there. So for relatively large archives it will often be more appropriate to use ArchiveWorkspace instead.

func (*ArchiveData) Metadata Uses

func (ad *ArchiveData) Metadata() (*Metadata, error)

Metadata returns the metadata stored in the backup archive. If no metadata is there, errors.NotFound is returned.

func (*ArchiveData) NewBuffer Uses

func (ad *ArchiveData) NewBuffer() *bytes.Buffer

NewBuffer wraps the archive data in a Buffer.

func (*ArchiveData) Version Uses

func (ad *ArchiveData) Version() (*version.Number, error)

Version returns the juju version under which the backup archive was created. If no version is found in the archive, it must come from before backup archives included the version. In that case we return version 1.20.

type ArchivePaths Uses

type ArchivePaths struct {
    // ContentDir is the path to the directory within the archive
    // containing all the contents. It is the only file or directory at
    // the top-level of the archive and everything else in the archive
    // is contained in the content directory.
    ContentDir string

    // FilesBundle is the path to the tar file inside the archive
    // containing all the state-related files (with the exception of the
    // DB dump files) gathered in by the backup machinery.
    FilesBundle string

    // DBDumpDir is the path to the directory within the archive
    // contents that contains all the files dumped from the juju state
    // database.
    DBDumpDir string

    // MetadataFile is the path to the metadata file.
    MetadataFile string

ArchivePaths holds the paths to the files and directories in a backup archive.

func NewCanonicalArchivePaths Uses

func NewCanonicalArchivePaths() ArchivePaths

NewCanonicalArchivePaths composes a new ArchivePaths with default values set. These values are relative (un-rooted) and the canonical slash ("/") is the path separator. Thus the paths are suitable for resolving the paths in a backup archive file (which is a tar file).

func NewNonCanonicalArchivePaths Uses

func NewNonCanonicalArchivePaths(rootDir string) ArchivePaths

NonCanonicalArchivePaths builds a new ArchivePaths using default values, rooted at the provided rootDir. The path separator used is platform-dependent. The resulting paths are suitable for locating backup archive contents in a directory into which an archive has been unpacked.

type ArchiveWorkspace Uses

type ArchiveWorkspace struct {
    RootDir string

ArchiveWorkspace is a wrapper around backup archive info that has a concrete root directory and an archive unpacked in it.

func NewArchiveWorkspaceReader Uses

func NewArchiveWorkspaceReader(archive io.Reader) (*ArchiveWorkspace, error)

NewArchiveWorkspaceReader returns a new archive workspace with a new workspace dir populated from the archive. Note that this involves unpacking the entire archive into a directory under the host's "temporary" directory. For relatively large archives this could have adverse effects on hosts with little disk space.

func (*ArchiveWorkspace) Close Uses

func (ws *ArchiveWorkspace) Close() error

Close cleans up the workspace dir.

func (*ArchiveWorkspace) Metadata Uses

func (ws *ArchiveWorkspace) Metadata() (*Metadata, error)

Metadata returns the metadata derived from the JSON file in the archive.

func (*ArchiveWorkspace) OpenBundledFile Uses

func (ws *ArchiveWorkspace) OpenBundledFile(filename string) (io.Reader, error)

OpenBundledFile returns an open ReadCloser for the corresponding file in the archived files bundle.

func (*ArchiveWorkspace) UnpackFilesBundle Uses

func (ws *ArchiveWorkspace) UnpackFilesBundle(targetRoot string) error

UnpackFilesBundle unpacks the archived files bundle into the targeted dir.

type Backups Uses

type Backups interface {
    // Create creates a new juju backup archive. It updates
    // the provided metadata.
    Create(meta *Metadata, paths *Paths, dbInfo *DBInfo, keepCopy, noDownload bool) (string, error)

    // Add stores the backup archive and returns its new ID.
    Add(archive io.Reader, meta *Metadata) (string, error)

    // Get returns the metadata and archive file associated with the ID.
    Get(id string) (*Metadata, io.ReadCloser, error)

    // List returns the metadata for all stored backups.
    List() ([]*Metadata, error)

    // Remove deletes the backup from storage.
    Remove(id string) error

    // Restore updates juju's state to the contents of the backup archive,
    // it returns the tag string for the machine where the backup originated
    // or error if the process fails.
    Restore(backupId string, args RestoreArgs) (names.Tag, error)

Backups is an abstraction around all juju backup-related functionality.

func NewBackups Uses

func NewBackups(stor filestorage.FileStorage) Backups

NewBackups creates a new Backups value using the FileStorage provided.

type ControllerMetadata Uses

type ControllerMetadata struct {
    // UUID contains the controller UUID.
    UUID string

    // MachineID contains controller machine id from which this backup is taken.
    MachineID string

    // MachineInstanceID contains controller machine's instance id from which this backup is taken.
    MachineInstanceID string

    // HANodes contains the number of nodes in this controller's HA configuration.
    HANodes int64

ControllerMetadata contains controller specific metadata.

func UnknownController Uses

func UnknownController() ControllerMetadata

UnknownController returns a new backups origin with unknown values.

type DB Uses

type DB interface {
    // MongoSession returns the underlying mongodb session.
    MongoSession() *mgo.Session

    // ModelTag is the concrete model tag for this database.
    ModelTag() names.ModelTag

    // ModelConfig is the config of the model being backedup.
    ModelConfig() (*config.Config, error)

    // ControllerConfig is the config of the controller being backedup.
    ControllerConfig() (controller.Config, error)

    // StateServingInfo is the secrets of the controller.
    StateServingInfo() (controller.StateServingInfo, error)

DB represents the set of methods required to perform a backup. It exists to break the strict dependency between state and this package, and those that depend on this package.

type DBDumper Uses

type DBDumper interface {
    // Dump something to dumpDir.
    Dump(dumpDir string) error

DBDumper is any type that dumps something to a dump dir.

func NewDBDumper Uses

func NewDBDumper(info *DBInfo) (DBDumper, error)

NewDBDumper returns a new value with a Dump method for dumping the juju state database.

type DBInfo Uses

type DBInfo struct {
    // Address is the DB system's host address.
    Address string
    // Username is used when connecting to the DB system.
    Username string
    // Password is used when connecting to the DB system.
    Password string
    // Targets is a list of databases to dump.
    Targets set.Strings
    // MongoVersion the version of the running mongo db.
    MongoVersion mongo.Version

DBInfo wraps all the DB-specific information backups needs to dump the database. This includes a simplification of the information in authentication.MongoInfo.

func NewDBInfo Uses

func NewDBInfo(mgoInfo *mongo.MongoInfo, session DBSession, version mongo.Version) (*DBInfo, error)

NewDBInfo returns the information needed by backups to dump the database.

type DBRestorer Uses

type DBRestorer interface {
    // Dump something to dumpDir.
    Restore(dumpDir string, dialInfo *mgo.DialInfo) error

DBDumper is any type that dumps something to a dump dir.

func NewDBRestorer Uses

func NewDBRestorer(args RestorerArgs) (DBRestorer, error)

NewDBRestorer returns a new structure that can perform a restore on the db pointed in dialInfo.

type DBSession Uses

type DBSession interface {
    DatabaseNames() ([]string, error)

type Metadata Uses

type Metadata struct {

    // Started records when the backup was started.
    Started time.Time

    // Finished records when the backup was complete.
    Finished *time.Time

    // Origin identifies where the backup was created.
    Origin Origin

    // Notes is an optional user-supplied annotation.
    Notes string

    // FormatVersion stores format version of these metadata.
    FormatVersion int64

    // Controller contains metadata about the controller where the backup was taken.
    Controller ControllerMetadata

    // CACert is the controller CA certificate.
    CACert string

    // CAPrivateKey is the controller CA private key.
    CAPrivateKey string

Metadata contains the metadata for a single state backup archive.

func BuildMetadata Uses

func BuildMetadata(file *os.File) (*Metadata, error)

BuildMetadata generates the metadata for a backup archive file.

func NewMetadata Uses

func NewMetadata() *Metadata

NewMetadata returns a new Metadata for a state backup archive, in the most current format.

func NewMetadataJSONReader Uses

func NewMetadataJSONReader(in io.Reader) (*Metadata, error)

NewMetadataJSONReader extracts a new metadata from the JSON file.

func NewMetadataState Uses

func NewMetadataState(db DB, machine, series string) (*Metadata, error)

NewMetadataState composes a new backup metadata based on the current Juju state.

func (*Metadata) AsJSONBuffer Uses

func (m *Metadata) AsJSONBuffer() (io.Reader, error)

AsJSONBuffer returns a bytes.Buffer containing the JSON-ified metadata. This will always produce latest known format.

func (*Metadata) MarkComplete Uses

func (m *Metadata) MarkComplete(size int64, checksum string) error

MarkComplete populates the remaining metadata values. The default checksum format is used.

type MongoDB Uses

type MongoDB interface {
    UpsertUser(*mgo.User) error

MongoDB represents a mgo.DB.

func GetDB Uses

func GetDB(s string, session MongoSession) MongoDB

GetDB wraps mgo.Session.DB to ease testing.

type MongoSession Uses

type MongoSession interface {
    Run(cmd interface{}, result interface{}) error
    DB(string) *mgo.Database

MongoSession represents mgo.Session.

func NewMongoSession Uses

func NewMongoSession(dialInfo *mgo.DialInfo) (MongoSession, error)

NewMongoSession wraps mgo.DialInfo to ease testing.

type Origin Uses

type Origin struct {
    Model    string
    Machine  string
    Hostname string
    Version  version.Number
    Series   string

Origin identifies where a backup archive came from. While it is more about where and Metadata about what and when, that distinction does not merit special consideration. Instead, Origin exists separately from Metadata due to its use as an argument when requesting the creation of a new backup.

func UnknownOrigin Uses

func UnknownOrigin() Origin

UnknownOrigin returns a new backups origin with unknown values.

type Paths Uses

type Paths struct {
    BackupDir string
    DataDir   string
    LogsDir   string

Paths holds the paths that backups needs.

type RestoreArgs Uses

type RestoreArgs struct {
    PrivateAddress string
    PublicAddress  string
    NewInstId      instance.Id
    NewInstTag     names.Tag
    NewInstSeries  string

RestoreArgs holds the args to be used to call state/backups.Restore

type RestorerArgs Uses

type RestorerArgs struct {
    DialInfo        *mgo.DialInfo
    NewMongoSession func(*mgo.DialInfo) (MongoSession, error)
    Version         mongo.Version
    TagUser         string
    TagUserPassword string
    GetDB           func(string, MongoSession) MongoDB

    RunCommandFn func(string, ...string) error
    StartMongo   func() error
    StopMongo    func() error



Package backups imports 52 packages (graph) and is imported by 146 packages. Updated 2020-04-27. Refresh now. Tools for package owners.