public-migratex

command module
v0.0.0-...-2597955 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 15, 2021 License: MIT Imports: 24 Imported by: 0

README

LocalCover Engineering - Migratex

Wrapper script to add support for DML database migrations to migrate.

We use the excellent migrate tool for Google GCP Spanner database migrations however it does not support DML. DML is useful when a new feauture requires some non-transactional data. We work around this with a wrapper tool migratex which interleaves DML migrations with migrate's DDL migrations.

migratexuses the Go Cloud Spanner client library and so can cache the Spanner session and leverage things like batch DML processing.

migratex requires a naming convention to recognize migrations. In the following examples _[SOME_BUINSESS_DOMAIN]_[SOME_FEATURE] can be anything, [REVISION] must be an integer, optionally prefixed by zeros, and [ENV_ID] is and environment ID for which the migrations should be applied. The environment ID is passed to migratex at runtime:

[REVISION]_[SOME_BUINSESS_DOMAIN]_[SOME_FEATURE].ddl.up.sql
[REVISION]_[SOME_BUINSESS_DOMAIN]_[SOME_FEATURE].[ENV_ID].dml.sql
[REVISION]_[SOME_BUINSESS_DOMAIN]_[SOME_FEATURE].[ENV_ID].[ENV_ID].dml.sql
[REVISION]_[SOME_BUINSESS_DOMAIN]_[SOME_FEATURE].all.dml.sql

DML can contain tokens and if so the tokens will be resolved if a JSON token definition file exists. JSON token definition files are optional but there can only be one per DML file:

[REVISION]_[SOME_BUINSESS_DOMAIN]_[SOME_FEATURE].[ENV_ID].dml.json
[REVISION]_[SOME_BUINSESS_DOMAIN]_[SOME_FEATURE].[ENV_ID].[ENV_ID].dml.json
[REVISION]_[SOME_BUINSESS_DOMAIN]_[SOME_FEATURE].all.dml.json

Note that there can only be one DML file for a revision for each environment. Note that DML migration revision history is maintained in the table DataMigrations.

Usage

gcloud config configurations activate [CONFIGURATION_NAME]

gcloud spanner databases create [SPANNER_DATABASE_ID] --instance=[SPANNER_INSTANCE_ID]

# DDL only using 'migrate'
migrate -path . -database spanner://projects/[GCP_PROJECT_ID]/instances/[SPANNER_INSTANCE_ID]/databases/[SPANNER_DATABASE_ID] up

go get -u github.com/localcover/public-migratex

# DDL and DML using 'migratex'
migratex -env_id=[ENV_ID] -gcp_project_id=[GCP_PROJECT_ID] -spanner_instance_id=[SPANNER_INSTANCE_ID] -spanner_database_id=[SPANNER_DATABASE_ID]

Contributors

Contributors in alphabetical order are Leonie Farmer, Jordan Felix, Billy Fung, Cai Gwatkin, Xiaona Jia, Rick Lee, Ling Liang, Parker Lin, Bruno Monteiro, Tom Wang, Steven Zhang, Yixin Zhang et al.

Documentation

The Go Gopher

There is no documentation for this package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL