datastore

package
v1.87.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 17, 2020 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package datastore provides data models and datastore persistence abstractions for tracking the state of repository replicas.

See original design discussion: https://gitlab.com/gitlab-org/gitaly/issues/1495

Index

Constants

This section is empty.

Variables

View Source 
var ErrInvalidReplTarget = errors.New("targetStorage repository fails preconditions for replication")

ErrInvalidReplTarget indicates a targetStorage repository cannot be chosen because it fails preconditions for being replicatable

View Source 
var ErrNoPrimaryForStorage = errors.New("no primary for storage")

ErrNoPrimaryForStorage indicates a virtual storage has no primary associated with it

View Source 
var (
	// ErrPrimaryNotSet indicates the primary has not been set in the datastore
	ErrPrimaryNotSet = errors.New("primary is not set")
)
View Source 
var ErrReplicasMissing = errors.New("repository missing secondary replicas")

ErrReplicasMissing indicates the repository does not have any backup replicas

Functions

func CheckPostgresVersion added in v1.76.0 

func CheckPostgresVersion(conf config.Config) error

CheckPostgresVersion checks the server version of the Postgres DB specified in conf. This is a diagnostic for the Praefect Postgres rollout. https://gitlab.com/gitlab-org/gitaly/issues/1755

func Migrate added in v1.81.0 

func Migrate(conf config.Config) (int, error)

Migrate will apply all pending SQL migrations

Types

type ChangeType 

type ChangeType int

ChangeType indicates what kind of change the replication is propagating 

const (
	// UpdateRepo is when a replication updates a repository in place
	UpdateRepo ChangeType = iota + 1
	// DeleteRepo is when a replication deletes a repo
	DeleteRepo
)

type Datastore 

type Datastore interface {
	ReplJobsDatastore
	ReplicasDatastore
}

Datastore is a data persistence abstraction for all of Praefect's persistence needs

type JobState 

type JobState uint8

JobState is an enum that indicates the state of a job 

const (
	// JobStatePending is the initial job state when it is not yet ready to run
	// and may indicate recovery from a failure prior to the ready-state
	JobStatePending JobState = 1 << iota
	// JobStateReady indicates the job is now ready to proceed
	JobStateReady
	// JobStateInProgress indicates the job is being processed by a worker
	JobStateInProgress
	// JobStateComplete indicates the job is now complete
	JobStateComplete
	// JobStateCancelled indicates the job was cancelled. This can occur if the
	// job is no longer relevant (e.g. a node is moved out of a repository)
	JobStateCancelled
	// JobStateFailed indicates the job did not succeed. The Replicator will retry
	// failed jobs.
	JobStateFailed
	// JobStateDead indicates the job was retried up to the maximum retries
	JobStateDead
)

type MemoryDatastore 

type MemoryDatastore struct {
	// contains filtered or unexported fields
}

MemoryDatastore is a simple datastore that isn't persisted to disk. It is only intended for early beta requirements and as a reference implementation for the eventual SQL implementation

func NewInMemory 

func NewInMemory(cfg config.Config) *MemoryDatastore

NewInMemory returns an initialized in-memory datastore

func (*MemoryDatastore) CreateReplicaReplJobs 

func (md *MemoryDatastore) CreateReplicaReplJobs(relativePath string, primaryStorage string, secondaryStorages []string, change ChangeType) ([]uint64, error)

CreateReplicaReplJobs creates a replication job for each secondary that backs the specified repository. Upon success, the job IDs will be returned.

func (*MemoryDatastore) GetJobs 

func (md *MemoryDatastore) GetJobs(state JobState, targetNodeStorage string, count int) ([]ReplJob, error)

GetJobs is a more general method to retrieve jobs of a certain state from the datastore

func (*MemoryDatastore) GetPrimary 

func (md *MemoryDatastore) GetPrimary(virtualStorage string) (models.Node, error)

GetPrimary returns the primary configured in the config file

func (*MemoryDatastore) GetReplicas 

func (md *MemoryDatastore) GetReplicas(relativePath string) ([]models.Node, error)

GetReplicas gets the secondaries for a repository based on the relative path

func (*MemoryDatastore) GetSecondaries added in v1.86.0 

func (md *MemoryDatastore) GetSecondaries(virtualStorage string) ([]models.Node, error)

GetSecondaries gets the secondary nodes associated with a virtual storage

func (*MemoryDatastore) GetStorageNode 

func (md *MemoryDatastore) GetStorageNode(nodeStorage string) (models.Node, error)

GetStorageNode gets all storage nodes

func (*MemoryDatastore) GetStorageNodes 

func (md *MemoryDatastore) GetStorageNodes() ([]models.Node, error)

GetStorageNodes gets all storage nodes

func (*MemoryDatastore) IncrReplJobAttempts added in v1.87.0 

func (md *MemoryDatastore) IncrReplJobAttempts(jobID uint64) error

IncrReplJobAttempts updates an existing replication job's state

func (*MemoryDatastore) UpdateReplJobState added in v1.87.0 

func (md *MemoryDatastore) UpdateReplJobState(jobID uint64, newState JobState) error

UpdateReplJobState updates an existing replication job's state

type ReplJob 

type ReplJob struct {
	Change                 ChangeType
	ID                     uint64      // autoincrement ID
	TargetNode, SourceNode models.Node // which node to replicate to?
	RelativePath           string      // source for replication
	State                  JobState
	Attempts               int
}

ReplJob is an instance of a queued replication job. A replication job is meant for updating the repository so that it is synced with the primary copy. Scheduled indicates when a replication job should be performed.

type ReplJobsDatastore 

type ReplJobsDatastore interface {
	// GetJobs fetches a list of chronologically ordered replication
	// jobs for the given storage replica. The returned list will be at most
	// count-length.
	GetJobs(flag JobState, nodeStorage string, count int) ([]ReplJob, error)

	// CreateReplicaReplJobs will create replication jobs for each secondary
	// replica of a repository known to the datastore. A set of replication job
	// ID's for the created jobs will be returned upon success.
	CreateReplicaReplJobs(relativePath string, primaryStorage string, secondaryStorages []string, change ChangeType) ([]uint64, error)

	// UpdateReplJobState updates the state of an existing replication job
	UpdateReplJobState(jobID uint64, newState JobState) error

	IncrReplJobAttempts(jobID uint64) error
}

ReplJobsDatastore represents the behavior needed for fetching and updating replication jobs from the datastore

type ReplicasDatastore 

type ReplicasDatastore interface {
	GetPrimary(virtualStorage string) (models.Node, error)

	GetSecondaries(virtualStorage string) ([]models.Node, error)

	GetReplicas(relativePath string) ([]models.Node, error)

	GetStorageNode(nodeStorage string) (models.Node, error)

	GetStorageNodes() ([]models.Node, error)
}

ReplicasDatastore manages accessing and setting which secondary replicas backup a repository

Source Files

View all Source files

Directories

Path Synopsis
Package glsql (Gitaly SQL) is a helper package to work with plain SQL queries.
 Package glsql (Gitaly SQL) is a helper package to work with plain SQL queries.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.