Documentation ¶
Overview ¶
Package golembic is intended to provide tooling for managing SQL migrations in Go.
The underlying principles of golembic are as follows:
- Migrations should follow a straight line (from some root); this line should be verified to avoid merge conflicts causing "branching"
- The "current" state of the world will be tracked in a `migrations` table containing all migrations that have been applied.
- A series of migrations should be easy to use both in a script or as part of a larger piece of Go code
- Down migrations are not supported. The idea being that the risk of data loss from a down migration is not worth it and that writing down migrations can be more challenging than writing up migrations.
The design allows for running "arbitrary" code inside migrations so that even non-SQL tasks can be tracked as a "run-once" migration.
The name is a portmanteau of Go (the programming language) and `alembic`, the Python migrations package. The name `alembic` itself is motivated by the foundational ORM SQLAlchemy (an alembic is a distilling apparatus used by alchemists).
Index ¶
- Constants
- Variables
- func ApplyDynamic(ctx context.Context, s *migration.Suite, c *db.Connection) (err error)
- func GenerateSuite(m *Manager) (*migration.Suite, error)
- func PlanEventWrite(ctx context.Context, log logger.Log, revision, body string, status PlanStatus)
- func QuoteIdentifier(name string) string
- type ApplyConfig
- type ApplyOption
- type CreateTableParameters
- type Manager
- func (m *Manager) ApplyMigration(ctx context.Context, pool *db.Connection, tx *sql.Tx, migration Migration) (err error)
- func (m *Manager) InsertMigration(ctx context.Context, pool *db.Connection, tx *sql.Tx, migration Migration) error
- func (m *Manager) Latest(ctx context.Context, pool *db.Connection, tx *sql.Tx) (revision string, createdAt time.Time, err error)
- func (m *Manager) Plan(ctx context.Context, pool *db.Connection, tx *sql.Tx, opts ...ApplyOption) ([]Migration, error)
- type ManagerOption
- type Migration
- type MigrationOption
- func OptAlwaysError(err error) MigrationOption
- func OptDescription(description string) MigrationOption
- func OptMilestone(milestone bool) MigrationOption
- func OptPrevious(previous string) MigrationOption
- func OptRevision(revision string) MigrationOption
- func OptUp(up UpMigration) MigrationOption
- func OptUpConn(up UpMigrationConn) MigrationOption
- func OptUpConnFromFile(filename string) MigrationOption
- func OptUpConnFromSQL(statement string) MigrationOption
- func OptUpFromFile(filename string) MigrationOption
- func OptUpFromSQL(statement string) MigrationOption
- type Migrations
- func (m *Migrations) All() []Migration
- func (m *Migrations) Get(revision string) *Migration
- func (m *Migrations) Register(migration Migration) error
- func (m *Migrations) RegisterMany(ms ...Migration) error
- func (m *Migrations) RegisterManyOpt(manyOpts ...[]MigrationOption) error
- func (m *Migrations) Revisions() []string
- func (m *Migrations) Root() Migration
- func (m *Migrations) Since(revision string) (int, []Migration, error)
- type PlanEvent
- type PlanStatus
- type UpMigration
- type UpMigrationConn
Constants ¶
const ( // DefaultMetadataTable is the default name for the table used to store // metadata about migrations. DefaultMetadataTable = "golembic_migrations" )
Variables ¶
var ( // ErrNotRoot is the error returned when attempting to start a sequence of // migrations with a non-root migration. ErrNotRoot = ex.Class("Root migration cannot have a previous migration set") // ErrMissingRevision is the error returned when attempting to register a migration // with no revision. ErrMissingRevision = ex.Class("A migration must have a revision") // ErrNoPrevious is the error returned when attempting to register a migration // with no previous. ErrNoPrevious = ex.Class("Cannot register a migration with no previous migration") // ErrPreviousNotRegistered is the error returned when attempting to register // a migration with a previous that is not yet registered. ErrPreviousNotRegistered = ex.Class("Cannot register a migration until previous migration is registered") // ErrAlreadyRegistered is the error returned when a migration has already been // registered. ErrAlreadyRegistered = ex.Class("Migration has already been registered") // ErrNilInterface is the error returned when a value satisfying an interface // is nil in a context where it is not allowed. ErrNilInterface = ex.Class("Value satisfying interface was nil") // ErrMigrationNotRegistered is the error returned when no migration has been // registered for a given revision. ErrMigrationNotRegistered = ex.Class("No migration registered for revision") // ErrMigrationMismatch is the error returned when the migration stored in // SQL does not match the registered migration. ErrMigrationMismatch = ex.Class("Migration stored in SQL doesn't match sequence") // ErrCannotInvokeUp is the error returned when a migration cannot invoke the // up function (e.g. if it is `nil`). ErrCannotInvokeUp = ex.Class("Cannot invoke up function for a migration") // ErrCannotPassMilestone is the error returned when a migration sequence // contains a milestone migration that is **NOT** the last step. ErrCannotPassMilestone = ex.Class("If a migration sequence contains a milestone, it must be the last migration") )
Functions ¶
func ApplyDynamic ¶
ApplyDynamic applies a migrations suite. Rather than using a `range` over `s.Groups`, it uses a length check, which allows `s.Groups` to change dynamically during the iteration.
func GenerateSuite ¶
GenerateSuite generates a suite of migrations from a sequence of golembic migrations.
func PlanEventWrite ¶
func QuoteIdentifier ¶
QuoteIdentifier quotes an identifier, such as a table name, for usage in a query.
This implementation is vendored in here to avoid the side effects of importing `github.com/lib/pq`.
See: - https://github.com/lib/pq/blob/v1.8.0/conn.go#L1564-L1581 - https://www.sqlite.org/lang_keywords.html - https://github.com/ronsavage/SQL/blob/a67e7eaefae89ed761fa4dcbc5431ec9a235a6c8/sql-99.bnf#L412
Types ¶
type ApplyConfig ¶
type ApplyConfig struct {
VerifyHistory bool
}
ApplyConfig provides configurable fields for "up" commands that will apply migrations.
func NewApplyConfig ¶
func NewApplyConfig(opts ...ApplyOption) (*ApplyConfig, error)
NewApplyConfig creates a new `ApplyConfig` and applies options.
type ApplyOption ¶
type ApplyOption = func(*ApplyConfig) error
ApplyOption describes options used to create an apply configuration.
func OptApplyVerifyHistory ¶
func OptApplyVerifyHistory(verify bool) ApplyOption
OptApplyVerifyHistory sets `VerifyHistory` on an `ApplyConfig`.
type CreateTableParameters ¶
type CreateTableParameters struct { SerialID string Revision string Previous string CreatedAt string }
CreateTableParameters specifies a set of parameters that are intended to be used in a `CREATE TABLE` statement.
type Manager ¶
type Manager struct { // MetadataTable is the name of the table that stores migration metadata. // The expected default value (`DefaultMetadataTable`) is // "golembic_migrations". MetadataTable string // Sequence is the collection of registered migrations to be applied, // verified, described, etc. by this manager. Sequence *Migrations // VerifyHistory indicates that the rows **stored** in the migration metadata // table should be verified during planning. VerifyHistory bool // DevelopmentMode is a flag indicating that this manager is currently // being run in development mode, so things like extra validation should // intentionally be disabled. This is intended for use in testing and // development, where an entire database is spun up locally (e.g. in Docker) // and migrations will be applied from scratch (including milestones that // may not come at the end). DevelopmentMode bool // Log is used for printing output Log logger.Log }
Manager orchestrates database operations done via `Up` / `UpConn` as well as supporting operations such as writing rows into a migration metadata table during a migration.
func NewManager ¶
func NewManager(opts ...ManagerOption) (*Manager, error)
NewManager creates a new manager for orchestrating migrations.
func (*Manager) ApplyMigration ¶
func (m *Manager) ApplyMigration(ctx context.Context, pool *db.Connection, tx *sql.Tx, migration Migration) (err error)
ApplyMigration creates a transaction that runs the "Up" migration.
func (*Manager) InsertMigration ¶
func (m *Manager) InsertMigration(ctx context.Context, pool *db.Connection, tx *sql.Tx, migration Migration) error
InsertMigration inserts a migration into the migrations metadata table.
func (*Manager) Latest ¶
func (m *Manager) Latest(ctx context.Context, pool *db.Connection, tx *sql.Tx) (revision string, createdAt time.Time, err error)
Latest determines the revision and timestamp of the most recently applied migration.
NOTE: This assumes, but does not check, that the migrations metadata table exists.
type ManagerOption ¶
ManagerOption describes options used to create a new manager.
func OptDevelopmentMode ¶
func OptDevelopmentMode(mode bool) ManagerOption
OptDevelopmentMode sets the development mode flag on a manager.
func OptManagerLog ¶
func OptManagerLog(log logger.Log) ManagerOption
OptManagerLog sets the logger interface on a manager. If `log` is `nil`code man the option will return an error.
func OptManagerMetadataTable ¶
func OptManagerMetadataTable(table string) ManagerOption
OptManagerMetadataTable sets the metadata table name on a manager.
func OptManagerSequence ¶
func OptManagerSequence(migrations *Migrations) ManagerOption
OptManagerSequence sets the migrations sequence on a manager.
func OptManagerVerifyHistory ¶
func OptManagerVerifyHistory(verify bool) ManagerOption
OptManagerVerifyHistory sets `VerifyHistory` on a manager.
type Migration ¶
type Migration struct { // Previous is the revision identifier for the migration immediately // preceding this one. If absent, this indicates that this migration is // the "base" or "root" migration. Previous string // Revision is an opaque name that uniquely identifies a migration. It // is required for a migration to be valid. Revision string // Description is a long form description of why the migration is being // performed. It is intended to be used in "describe" scenarios where // a long form "history" of changes is presented. Description string // Milestone is a flag indicating if the current migration is a milestone. // A milestone is a special migration that **must** be the last migration // in a sequence whenever applied. This is intended to be used in situations // where a change must be "staged" in two (or more parts) and one part // must run and "stabilize" before the next migration runs. For example, in // a rolling update deploy strategy some changes may not be compatible with // "old" and "new" versions of the code that may run simultaneously, so a // milestone marks the last point where old / new versions of application // code should be expected to be able to interact with the current schema. Milestone bool // Up is the function to be executed when a migration is being applied. Either // this field or `UpConn` are required (not both) and this field should be // the default choice in most cases. This function will be run in a transaction // that also writes a row to the migrations metadata table to signify that // this migration was applied. Up UpMigration // UpConn is the non-transactional form of `Up`. This should be used in // rare situations where a migration cannot run inside a transaction, e.g. // a `CREATE UNIQUE INDEX CONCURRENTLY` statement. UpConn UpMigrationConn // contains filtered or unexported fields }
Migration represents an individual migration to be applied; typically as a set of SQL statements.
func NewMigration ¶
func NewMigration(opts ...MigrationOption) (*Migration, error)
NewMigration creates a new migration from a variadic slice of options.
func (Migration) ExtendedDescription ¶
ExtendedDescription is an extended form of `m.Description` that also incorporates other information like whether `m` is a milestone.
func (Migration) InvokeUp ¶
InvokeUp dispatches to `Up` or `UpConn`, depending on which is set. If both or neither is set, that is considered an error. If `UpConn` needs to be invoked, this lazily creates a new connection from a pool. It's crucial that the pool sets the relevant timeouts when creating a new connection to make sure migrations don't cause disruptions in application performance due to accidentally holding locks for an extended period.
type MigrationOption ¶
MigrationOption describes options used to create a new migration.
func OptAlwaysError ¶
func OptAlwaysError(err error) MigrationOption
OptAlwaysError returns an option that always returns an error.
func OptDescription ¶
func OptDescription(description string) MigrationOption
OptDescription sets the description on a migration.
func OptMilestone ¶
func OptMilestone(milestone bool) MigrationOption
OptMilestone sets the milestone flag on a migration.
func OptPrevious ¶
func OptPrevious(previous string) MigrationOption
OptPrevious sets the previous on a migration.
func OptRevision ¶
func OptRevision(revision string) MigrationOption
OptRevision sets the revision on a migration.
func OptUp ¶
func OptUp(up UpMigration) MigrationOption
OptUp sets the `up` function on a migration.
func OptUpConn ¶
func OptUpConn(up UpMigrationConn) MigrationOption
OptUpConn sets the non-transactional `up` function on a migration.
func OptUpConnFromFile ¶
func OptUpConnFromFile(filename string) MigrationOption
OptUpConnFromFile returns an option that sets the non-transctional `up` function to execute a SQL statement that is stored in a file.
func OptUpConnFromSQL ¶
func OptUpConnFromSQL(statement string) MigrationOption
OptUpConnFromSQL returns an option that sets the non-transctional `up` function to execute a SQL statement.
func OptUpFromFile ¶
func OptUpFromFile(filename string) MigrationOption
OptUpFromFile returns an option that sets the `up` function to execute a SQL statement that is stored in a file.
func OptUpFromSQL ¶
func OptUpFromSQL(statement string) MigrationOption
OptUpFromSQL returns an option that sets the `up` function to execute a SQL statement.
type Migrations ¶
type Migrations struct {
// contains filtered or unexported fields
}
Migrations represents a sequence of migrations to be applied.
func NewSequence ¶
func NewSequence(root Migration) (*Migrations, error)
NewSequence creates a new sequence of migrations rooted in a single base / root migration.
func (*Migrations) All ¶
func (m *Migrations) All() []Migration
All produces the migrations in the sequence, in order.
NOTE: This does not verify or enforce the invariant that there must be
exactly one migration without a previous migration. This invariant is enforced by the exported methods such as `Register()` and `RegisterMany()` and the constructor `NewSequence()`.
func (*Migrations) Get ¶
func (m *Migrations) Get(revision string) *Migration
Get retrieves a revision from the sequence, if present. If not, returns `nil`.
func (*Migrations) Register ¶
func (m *Migrations) Register(migration Migration) error
Register adds a new migration to an existing sequence of migrations, if possible. The new migration must have a previous migration and have a valid revision that is not already registered.
func (*Migrations) RegisterMany ¶
func (m *Migrations) RegisterMany(ms ...Migration) error
RegisterMany attempts to register multiple migrations (in order) with an existing sequence.
func (*Migrations) RegisterManyOpt ¶
func (m *Migrations) RegisterManyOpt(manyOpts ...[]MigrationOption) error
RegisterManyOpt attempts to register multiple migrations (in order) with an existing sequence. It differs from `RegisterMany()` in that the construction of `Migration` objects is handled directly here by taking a slice of option slices.
func (*Migrations) Revisions ¶
func (m *Migrations) Revisions() []string
Revisions produces the revisions in the sequence, in order.
This utilizes `All()` and just extracts the revisions.
func (*Migrations) Root ¶
func (m *Migrations) Root() Migration
Root does a linear scan of every migration in the sequence and returns the root migration. In the "general" case such a scan would be expensive, but the number of migrations should always be a small number.
NOTE: This does not verify or enforce the invariant that there must be exactly one migration without a previous migration. This invariant is enforced by the exported methods such as `Register()` and `RegisterMany()` and the constructor `NewSequence()`.
func (*Migrations) Since ¶
func (m *Migrations) Since(revision string) (int, []Migration, error)
Since returns the migrations that occur **after** `revision`.
This utilizes `All()` and returns all migrations after the one that matches `revision`. If none match, an error will be returned. If `revision` is the **last** migration, the migrations returned will be an empty slice.
type PlanEvent ¶
type PlanEvent struct { Revision string Body string Status PlanStatus Labels []string }
type PlanStatus ¶
type PlanStatus string
const ( PlanStatusUnset PlanStatus = "" PlanStatusFailed PlanStatus = migration.StatFailed PlanStatusApplied PlanStatus = migration.StatApplied )
type UpMigration ¶
UpMigration defines a function interface to be used for up / forward migrations. The SQL transaction will be started **before** `UpMigration` is invoked and will be committed **after** the `UpMigration` exits without error. In addition to the contents of `UpMigration`, a row will be written to the migrations metadata table as part of the transaction.
The expectation is that the migration runs SQL statements within the transaction. If a migration cannot run inside a transaction, e.g. a `CREATE UNIQUE INDEX CONCURRENTLY` statement, then the `UpMigration` interface should be used.
type UpMigrationConn ¶
type UpMigrationConn = func(context.Context, *db.Connection) error
UpMigrationConn defines a function interface to be used for up / forward migrations. This is the non-transactional form of `UpMigration` and should only be used in rare situations.