README
¶
Migrations
The migration package provides the tools to detect and manage the changes made to application models, store them in version control and apply them to the database schema.
Quick start
Example usage:
package main
import (
"fmt"
_ "github.com/gwenn/gosqlite" // Imports SQLite driver.
"github.com/moiseshiraldo/gomodel"
"github.com/moiseshiraldo/gomodel/migration"
"os"
)
var User = gomodel.New(
"User",
gomodel.Fields{
"email": gomodel.CharField{MaxLength: 100, Index: true},
"active": gomodel.BooleanField{DefaultFalse: true},
"created": gomodel.DateTimeField{AutoNowAdd: true},
},
gomodel.Options{},
)
var app = gomodel.NewApp("main", "", User.Model)
func setup() {
gomodel.Register(app)
gomodel.Start(map[string]gomodel.Database{
"default": {
Driver: "sqlite3",
Name: ":memory:",
},
})
}
func main() {
setup()
// Detects changes and writes migration files.
if _, err := migration.Make("main", migration.MakeOptions{}); err != nil {
fmt.Println(err)
os.Exit(1)
}
// Runs all pending migrations.
if err := migration.Run(migration.RunOptions{}); err != nil {
fmt.Println("something went wrong:", err)
os.Exit(1)
}
fmt.Println("Changes applied to database schema!")
}
Check the MakeOptions and RunOptions documentation for more details.
If the database schema is not going to change, or you don't need to keep track of the changes (e.g. in-memory SQLite database), you can detect and apply all the changes directly using the MakeAndRun function:
func main() {
setup()
if err := migration.MakeAndRun("default"); err != nil {
fmt.Println("something went wrong:", err)
}
}
Managing changes
When your project contains a considerable number of models that change over time, it's probably a good idea to keep track of the database schema changes along with the code ones.
Migration files
GoModel will read and write the schema changes to the directory specified when you create and register an application:
var app = gomodel.NewApp("main", "/home/dev/project/main/migrations", User.Model)
If you specify a relative path, the full one will be constructed from
$GOPATH/src. Each application must have a unique migrations path, that only
contains JSON files following the naming convention {number}_{name}.json,
where number is a four digit number representing the order of the node in the
graph of changes and name can be any arbitrary name (e.g. 0001_initial.json).
You can create a new empty migration using the Make function:
_, err := migration.Make("main", migration.MakeOptions{Empty: true})
if err != nil {
fmt.Println("something went wrong:", err)
}
The migration file will look something like this:
{
"App": "main",
"Dependencies": [
[
"main",
"0001_initial"
]
],
"Operations": []
}
The App attribute is the name of the application. Dependencies is the list
of migrations (application and full name) that should be applied before the
described one. And Operations is the list of changes to be applied to the
database schema (see supported operations).
Automatic detection
The Make function can be used to automatically detect any changes on the application models and write them to migration files:
if _, err := migration.Make("main", migration.MakeOptions{}); err != nil {
fmt.Println(err)
}
The function will load and process the existing migration files, compare the resulted state with the model definitions and write any changes to new migration files.
Applying migrations
The Run function can be used to apply changes from migration files to the database schema. For example:
options := migration.RunOptions{
App: "main",
Node: "0002",
Database: "default",
}
if _, err := migration.Run(options); err != nil {
fmt.Println(err)
}
The code above would apply the changes up to and including the second migration. If any migration greater than the second one was already applied, it would be reverted.
Supported operations
CreateModel
{
"CreateModel": {
"Name": "User",
"Table": "users",
"Fields": {
"id": {
"IntegerField": {
"PrimaryKey": true,
"Auto": true
}
},
"email": {
"CharField": {
"MaxLength": 100,
"Index": true
}
}
}
}
}
DeleteModel
{
"DeleteModel": {
"Name": "User",
}
}
AddFields
{
"AddFields": {
"Model": "User",
"Fields": {
"active": {
"BooleanField": {
"DefaultFalse": true
}
},
"created": {
"DateTimeField": {
"AutoNowAdd": true
}
}
}
}
}
RemoveFields
{
"RemoveFields": {
"Model": "User",
"Fields": [
"active",
"created"
]
}
}
AddIndex
{
"AddIndex": {
"Model": "User",
"Name": "users_user_email_auto_idx",
"Fields": [
"email"
]
}
}
RemoveIndex
{
"RemoveIndex": {
"Model": "User",
"Name": "users_user_email_auto_idx"
}
}
Documentation
¶
Overview ¶
Package migration provides the tools to detect and manage the changes made to application models, store them in version control and apply them to the database schema.
The Make function detects and returns all the changes made to the models in the specified application, and optionally writes them to the project source in JSON format.
The Run function loads the changes from the project source and apply them to the database schema.
Alternatively, the MakeAndRun function can be used to directly create all the models on the database schema without storing changes in the project source.
Index ¶
- Variables
- func MakeAndRun(database string) error
- func RegisterOperation(op Operation) error
- func Run(options RunOptions) error
- type AddFields
- type AddIndex
- type AppNotFoundError
- type AppState
- type CircularDependencyError
- type CreateModel
- type DeleteModel
- type DuplicateNumberError
- type ErrorTrace
- type InvalidDependencyError
- type LoadError
- type MakeOptions
- type NameError
- type NoAppMigrationsError
- type Node
- type Operation
- type OperationList
- type OperationRunError
- type OperationStateError
- type PathError
- type RemoveFields
- type RemoveIndex
- type RunOptions
- type SaveError
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var FileNameRegex = regexp.MustCompile(`^([0-9]{4})_\w+\.json$`)
FileNameRegex is a regular expression to validate migration file names (e.g. 0001_initial.json).
var Migration = gomodel.New( "Migration", gomodel.Fields{ "app": gomodel.CharField{MaxLength: 50}, "number": gomodel.IntegerField{}, "name": gomodel.CharField{MaxLength: 100}, "applied": gomodel.DateTimeField{AutoNowAdd: true}, }, gomodel.Options{}, )
Migration holds the model definition to store applied nodes in the database.
var NodeNameRegex = regexp.MustCompile(`^([0-9]{4})_\w+$`)
NodeNameRegex is a regular expression to validate node names (e.g. 0001_initial).
Functions ¶
func MakeAndRun ¶
MakeAndRun propagates all the application models to the database schema.
func RegisterOperation ¶
RegisterOperation registers a custom operation. Returns an error if the operation name returned by the OpName method already exists.
func Run ¶
func Run(options RunOptions) error
Run applies the migrations specified by RunOptions to the database schema.
Example ¶
User := gomodel.New(
"User",
gomodel.Fields{
"email": gomodel.CharField{Unique: true, MaxLength: 100},
"active": gomodel.BooleanField{DefaultFalse: true},
"created": gomodel.DateTimeField{AutoNowAdd: true},
},
gomodel.Options{},
)
app := gomodel.NewApp("users", "myproject/users/migrations", User.Model)
gomodel.Register(app)
gomodel.Start(map[string]gomodel.Database{
"default": {
Driver: "postgres",
Name: "gomodeltest",
User: "local",
Password: "1234",
},
})
if err := Run(RunOptions{App: "users", Node: "0001_initial"}); err != nil {
fmt.Println(err)
}
Output:
Types ¶
type AddFields ¶
AddFields implements the Operation interface to add new fields.
type AddIndex ¶
AddIndex implements the Operation interface to add an index.
type AppNotFoundError ¶
type AppNotFoundError struct {
Name string // Application name.
Trace ErrorTrace
}
AppNotFoundError is raised when the named application is not registered.
func (*AppNotFoundError) Error ¶
func (e *AppNotFoundError) Error() string
Error implements the error interface.
type AppState ¶
type AppState struct {
// Models holds the model definitions for this application state.
Models map[string]*gomodel.Model
// contains filtered or unexported fields
}
AppState holds the application state for a certain node of the changes graph.
func Make ¶
func Make(appName string, options MakeOptions) (*AppState, error)
Make detects the changes between the gomodel.Model definitions and the migration files for the application named by appName.
It returns the *AppState containing the migrations and writes them to files depending on MakeOptions.
Example ¶
User := gomodel.New(
"User",
gomodel.Fields{
"email": gomodel.CharField{Unique: true, MaxLength: 100},
"active": gomodel.BooleanField{DefaultFalse: true},
"created": gomodel.DateTimeField{AutoNowAdd: true},
},
gomodel.Options{},
)
app := gomodel.NewApp("users", "myproject/users/migrations", User.Model)
gomodel.Register(app)
gomodel.Start(map[string]gomodel.Database{
"default": {
Driver: "postgres",
Name: "gomodeltest",
User: "local",
Password: "1234",
},
})
if _, err := Make("users", MakeOptions{}); err != nil {
fmt.Println(err)
}
Output:
func (AppState) Fake ¶
Fake fakes the changes up to and including the node named by nodeName, for the db schema named by database.
func (*AppState) MakeMigrations ¶
MakeMigrations returns a list of nodes containing the changes between the gomodel.Model definitions and the migrations files for this application.
type CircularDependencyError ¶
type CircularDependencyError struct {
Trace ErrorTrace
}
CircularDependencyError is raised when the migration files contain circular dependencies.
func (*CircularDependencyError) Error ¶
func (e *CircularDependencyError) Error() string
Error implements the error interface.
type CreateModel ¶
type CreateModel struct {
Name string
// Table is used to set a custom name for the database table. If blank,
// the table will be created as {app_name}_{model_name} all lowercase.
Table string `json:",omitempty"`
Fields gomodel.Fields
}
CreateModel implements the Operation interface to create a new model.
func (CreateModel) Backwards ¶
func (op CreateModel) Backwards( engine gomodel.Engine, state *AppState, prevState *AppState, ) error
Backwards drops the table from the database.
func (CreateModel) OpName ¶
func (op CreateModel) OpName() string
OpName returns the operation name.
func (CreateModel) SetState ¶
func (op CreateModel) SetState(state *AppState) error
SetState adds the new model to the given application state.
type DeleteModel ¶
type DeleteModel struct {
Name string
}
DeleteModel implements the operation to delete a model.
func (DeleteModel) Backwards ¶
func (op DeleteModel) Backwards( engine gomodel.Engine, state *AppState, prevState *AppState, ) error
Backwards creates the table on the database.
func (DeleteModel) OpName ¶
func (op DeleteModel) OpName() string
OpName returns the operation name.
func (DeleteModel) SetState ¶
func (op DeleteModel) SetState(state *AppState) error
SetState removes the model from the given application state.
type DuplicateNumberError ¶
type DuplicateNumberError struct {
Trace ErrorTrace
}
DuplicateNumberError is raised if an application contains any migration file with a duplicate number.
func (*DuplicateNumberError) Error ¶
func (e *DuplicateNumberError) Error() string
Error implements the error interface.
type ErrorTrace ¶
ErrorTrace holds the context information of a migration error.
func (ErrorTrace) String ¶
func (e ErrorTrace) String() string
String implements the fmt.Stringer interface.
type InvalidDependencyError ¶
type InvalidDependencyError struct {
Trace ErrorTrace
}
InvalidDependencyError is raised when a migration file contains invalid dependencies.
func (*InvalidDependencyError) Error ¶
func (e *InvalidDependencyError) Error() string
Error implements the error interface.
type LoadError ¶
type LoadError struct {
Trace ErrorTrace
}
LoadError is raised when a node fails to read from file.
type MakeOptions ¶
type MakeOptions struct {
Empty bool // Empty option is used to create an empty migration file.
OmitWrite bool // OmitWrite options is used to skip writing the file.
}
MakeOptions holds the options for the Make function.
type NameError ¶
type NameError struct {
Name string // Node name.
Trace ErrorTrace
}
NameError is raised if an invalid migration node name is provided.
type NoAppMigrationsError ¶
type NoAppMigrationsError struct {
Name string // Node name.
Trace ErrorTrace
}
NoAppMigrationsError is raised when trying to migrate an application with no migrations.
func (*NoAppMigrationsError) Error ¶
func (e *NoAppMigrationsError) Error() string
Error implements the error interface.
type Node ¶
type Node struct {
// App is the application name.
App string
// Dependencies is the list of dependencies for this node. Each dependency
// is a list of two elements, application name and migration name
// (e.g. ["users", "0001_initial"])
Dependencies [][]string
// Operations is the list of operations describing the changes.
Operations OperationList
// contains filtered or unexported fields
}
A Node represents a point in the application changes graph.
func (*Node) Backwards ¶
Backwards reverses the node changes (and the applied nodes depending on it) on the database schema given by db. All the queries will be run inside a transaction if supported by the database.
func (*Node) Fake ¶
Fake marks the node as applied without making any changes to the database schema.
func (*Node) FakeBackwards ¶
FakeBackwards marks the node as unapplied without reversing any change on the database schema.
func (*Node) Load ¶
Load reads the node details from the corresponding file in the migrations folder.
type Operation ¶
type Operation interface {
// OpName returns the operation name, which must be unique.
OpName() string
// SetState applies the operation changes to the given application state.
SetState(state *AppState) error
// Run applies the operation changes to the database schema. The method
// is automatically called when a Node is migrating forwards.
//
// The engine is the database (or transaction if supported) Engine.
//
// The state is the application state resulted from the operation changes.
//
// The prevState is the application state previous to the operation changes.
Run(engine gomodel.Engine, state *AppState, prevState *AppState) error
// Backwards reverse the operation changes on the database schema. The
// method is automatically called when a Node is migrating backwards.
//
// The engine is the database (or transaction if supported) Engine.
//
// The state is the application state resulted from the operation changes.
//
// The prevState is the application state previous to the operation changes.
Backwards(engine gomodel.Engine, state *AppState, prevState *AppState) error
}
The Operation interface represents a change in the application state (models, fields, indexes...) and how to propagate it to the database schema.
Custom operations can be registered using the RegisterOperation function.
type OperationList ¶
type OperationList []Operation
OperationList represents the list of operations of a migration Node.
func (OperationList) MarshalJSON ¶
func (opList OperationList) MarshalJSON() ([]byte, error)
MarshalJSON implements the json.Marshaler interface for the OperationList type. The type is serialized into a list of JSON objects, where the key is the name of the Operation returned by the OpName method and the value is the serliazed type implementing the Operation interface.
func (*OperationList) UnmarshalJSON ¶
func (opList *OperationList) UnmarshalJSON(data []byte) error
UnmarshalJSON implements the json.Unmarshaler interface. An error is returned if the name of the operation is not registered.
type OperationRunError ¶
type OperationRunError struct {
Trace ErrorTrace
}
OperationRunError is raised if an operation fails to apply to the database schema.
func (*OperationRunError) Error ¶
func (e *OperationRunError) Error() string
Error implements the error interface.
type OperationStateError ¶
type OperationStateError struct {
Trace ErrorTrace
}
OperationStateError is raised if a migration file contains invalid operations.
func (*OperationStateError) Error ¶
func (e *OperationStateError) Error() string
Error implements the error interface.
type PathError ¶
type PathError struct {
App string // Application name.
Trace ErrorTrace
}
PathError is raised if reaading from the applicatoin path or writing to it is not possible.
type RemoveFields ¶
RemoveFields implements the Operation interface to remove fields.
func (RemoveFields) Backwards ¶
func (op RemoveFields) Backwards( engine gomodel.Engine, state *AppState, prevState *AppState, ) error
Backwards adds the columns to the table on the database.
func (RemoveFields) OpName ¶
func (op RemoveFields) OpName() string
OpName returns the operation name.
func (RemoveFields) SetState ¶
func (op RemoveFields) SetState(state *AppState) error
SetState removes the fields from the model in the given the application state.
type RemoveIndex ¶
RemoveIndex implements the Operation interface to remove an index.
func (RemoveIndex) Backwards ¶
func (op RemoveIndex) Backwards( engine gomodel.Engine, state *AppState, prevState *AppState, ) error
Backwards creates the index on the database.
func (RemoveIndex) OpName ¶
func (op RemoveIndex) OpName() string
OpName returns the operation name.
func (RemoveIndex) SetState ¶
func (op RemoveIndex) SetState(state *AppState) error
SetState removes the index from the model in the given application state.
type RunOptions ¶
type RunOptions struct {
App string // Application name or blank for all applications.
Node string // Node name (e.g. 0001_initial or just 0001).
Fake bool // Fake is used to apply the migration without DB changes.
Database string // Database name or blank for default.
}
RunOptions holds the options for the Run function.
Source Files