redux

package module
v0.0.0-...-02aa62a Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 10, 2018 License: BSD-2-Clause Imports: 14 Imported by: 0

README

Description

redux is top down software build tool similar to make but simpler and more reliable.

It implements and extends the redo concept somewhat sparsely described by DJ Bernstein.

I implemented a minimal set of redo tools some years ago and used them enough to become convinced that the idea was worthwhile. However, they needed to be better, sharper, and faster before they could replace Make in my toolbox, thus, this tool.

Installation

redux is written in Go and requires the Go compiler to be installed. Go can be fetched from the Go site or your favorite distribution channel.

Assuming Go is installed, the command:

$ go get github.com/gyepisam/redux/redux

will fetch, build and install redux into a $GOBIN directory. To complete the installation, run the command:

$ redux install

which installs the associated links to the binary and the man pages in a related directory. By default, the links are created in the same directory as the executable, but this can be changed with the environment variable: $BINDIR.

The redux man page documentation can be installed separately with the command

$ redux install man

There are several options for where to install the manual pages. Please see 'redux help install' for details.

redux supports the following commands:

  •  init -- Creates or reinitializes one or more redo root directories.
  • ifchange -- Creates dependency on targets and ensure that targets are up to date.
  • ifcreate -- Creates dependency on non-existence of targets.
  •  redo -- Builds files atomically.
  • install -- Installs links and manual pages

The install links command creates links for each of these commands so they can be invoked as:

Overview

A redo based system must have a directory named .redo at the top level project directory.

It is created with the redo-init command, whose effects are idempotent.

The database contains file status and metadata. Its format is not specified, but it must be capable of supporting multiple readers and multiple writers (though not necessarily writing to the same sections). Redux implements a file based database and a null database. The current file based database is probably about the slowest. Fortunately, new databases can be easily plugged in.

In the redo system, there are three kinds of files.

  • do files are scripts that build targets. In addition, they establish file dependencies by calling redo-ifcreate and redo-ifchange. do files are invoked by the redo commands directly and indirectly by redo-ifchange.

  • target files are generated or otherwise composed, likely from other files, by do scripts

  • static or source files are manually created/edited. They are tracked for changes, but are not generated. As such, they have no do files.

There are two kinds of dependencies

  • ifchange denotes a relationship where a change in one file marks the other as out of date. A ifchange B implies that a change in B invalidates A. Change includes creation, modification or deletion.

  • ifcreate denotes a dependency on the creation or deletion of a file. A ifcreate B implies that when B comes into existence or ceases to exist, A will be outdated.

Please see the individual command documentation for further details.

#Credits

redux is written by Gyepi Sam self-redux@gyepi.com.

The redo concept is DJ Bernstein's and is described here.

Thanks to Szabolcs Szasz, Mateusz Czapliński, and dontdieych for their requests, feedback, suggestions, ideas, and bug reports.

I am interested in any and all feedback on this software.

Documentation

Overview

redux is a library for redux, a top down software build tool which implements and extends the redo concept somewhat sparsely described by [DJ Bernstein](http://cr.yp.to/redo.html).

To install it, run 

$ go get github.com/gyepisam/redux/redux

The command 

$ redux help

provides built-in help, which is augmented by the html, man pages and the README file.

Index

Constants

View Source 
const (
	// TASK_PREFIX is a marker for scripts that don't produce content.
	TASK_PREFIX = `@`

	// REDO_DIR names the hidden directory used for data and configuration.
	REDO_DIR = ".redo"

	// REDO_DIR_ENV_NAME names the environment variable for the REDO_DIR hidden directory.
	REDO_DIR_ENV_NAME = "REDO_DIR"

	// KEY_SEPARATOR is used to join the parts of the database key.
	KEY_SEPARATOR = "/"

	// AUTO marks system generated event records.
	AUTO = "auto"

	// Directory creation permission mode
	DIR_PERM = 0755

	// Extension for do scripts
	DO_EXT = "do"

	// Basename for default script
	DO_BASENAME = "default"

	// Extension separator
	EXT_SEP = "."

	DEFAULT_TARGET = "all"

	DEFAULT_DO = DEFAULT_TARGET + EXT_SEP + DO_EXT
)
View Source 
const (
	// Where is data kept?
	DATA_DIR = "data"
)

Variables

View Source 
var (
	Verbosity = len(os.Getenv("REDO_VERBOSE"))
	Debug     = len(os.Getenv("REDO_DEBUG")) > 0
	ShellArgs = os.Getenv("REDO_SHELL_ARGS")
)

Options default to env values, to be overriden by main() if necessary.

View Source 
var NullKeyErr = errors.New("Key cannot be empty.")
View Source 
var NullPrefixErr = errors.New("Prefix cannot be empty.")

Functions

func InitDir 

func InitDir(dirname string) error

InitDir creates a redo directory in the specified project root directory.

func RecordRelation 

func RecordRelation(dependent *File, target *File, event Event, m *Metadata) error

func Verbose 

func Verbose() bool

func WithDB 

func WithDB(arg string, f func(DB) error) error

Types

type Config 

type Config struct {
	DBType string
}

Configuration container

type DB 

type DB interface {
	IsNull() bool

	// Put stores value under key
	Put(key string, value []byte) error

	// Get returns the value stored under key and a boolean indicating
	// whether the returned value was actually found in the database.
	Get(key string) ([]byte, bool, error)

	// Delete removes the value stored under key.
	// If key does not exist, the operation is a noop.
	Delete(key string) error

	// GetKeys returns a list of keys which have the specified key as a prefix.
	GetKeys(prefix string) ([]string, error)

	// GetValues returns a list of values whose keys matching the specified key prefix.
	GetValues(prefix string) ([][]byte, error)

	// GetRecords returns a list of records (keys and data) matchign the specified key prefix.
	GetRecords(prefix string) ([]Record, error)

	Close() error
}

DB allows allows for multiple implementations of the redo database.

func FileDbOpen 

func FileDbOpen(rootdir string) (DB, error)

Open requires a project root argument

func NullDbOpen 

func NullDbOpen(ignored string) (DB, error)

Open requires a project root argument

type Dependent 

type Dependent struct {
	Path string
}

Dependent is the inverse of Prerequisite

func (Dependent) File 

func (d Dependent) File(dir string) (*File, error)

type DoInfo 

type DoInfo struct {
	Dir     string
	Name    string
	Arg2    string   //do file arg2. Depends on target and do file names.
	RelDir  string   //relative directory to target from do script.
	Missing []string //more specific do scripts that were not found.
}

A DoInfo represents an active do file.

func (*DoInfo) Path 

func (do *DoInfo) Path() string

func (*DoInfo) RelPath 

func (do *DoInfo) RelPath(path string) string

type Event 

type Event string

Event names a change or create event 

const (
	IFCREATE Event = "ifcreate"
	IFCHANGE Event = "ifchange"

	AUTO_IFCREATE Event = AUTO + KEY_SEPARATOR + "ifcreate"
	AUTO_IFCHANGE Event = AUTO + KEY_SEPARATOR + "ifchange"
)

Events of note

func (Event) String 

func (event Event) String() string

String representation.

type File 

type File struct {
	Target string // file name argument to redo, redo-ifchange, redo-ifcreate, etc

	RootDir string // contains .redo directory
	Path    string // Relative to RootDir

	Dir  string
	Name string
	Ext  string // File extension. Could be empty. Includes preceding dot.

	PathHash     Hash   // SHA1 hash of Path. Used as database key.
	FullPathHash Hash   //SHA1 hash of RootDir/Path. Used for loop detection.
	DoFile       string // Do script used to generate target output.

	Config Config
	// contains filtered or unexported fields
}

File represents a source or target file..

func NewFile 

func NewFile(dir, path string) (f *File, err error)

NewFile creates and returns a File instance for the given path. If the path is not fully qualified, it is made relative to dir. The newly created instance is initialized with the database specified by the configuration file found in its root directory or the default database. If a file does not have a root directory, it is initialized with a NullDb and HasNullDb will return true.

func (*File) Abs 

func (f *File) Abs(path string) string

Abs returns a cleaned up fullpath by joining f.RootDir to path.

func (*File) AllDependents 

func (f *File) AllDependents() ([]*File, error)

func (*File) AsDependent 

func (f *File) AsDependent(dir string) Dependent

func (*File) AsPrerequisite 

func (f *File) AsPrerequisite(dir string, m *Metadata) Prerequisite

func (*File) ContentHash 

func (f *File) ContentHash() (Hash, error)

ContentHash returns a cryptographic hash of the file contents.

func (*File) Debug 

func (f *File) Debug(format string, args ...interface{})

Debug prints out messages to stderr when the debug flag is enabled.

func (*File) Delete 

func (f *File) Delete(key string) error

func (*File) DeleteAllDependencies 

func (f *File) DeleteAllDependencies() (err error)

func (*File) DeleteAllPrerequisites 

func (f *File) DeleteAllPrerequisites() error

DeleteAllPrerequisites removed all of the file's prerequisites.

func (*File) DeleteAutoPrerequisites 

func (f *File) DeleteAutoPrerequisites() error

DeleteAutoPrerequisites removes all of the file's system generated prerequisites.

func (*File) DeleteDependency 

func (f *File) DeleteDependency(event Event, hash Hash) error

func (*File) DeleteMetadata 

func (f *File) DeleteMetadata() error

DeleteMetadata removes the metadata record, if any, from the database.

func (*File) DeleteMustRebuild 

func (f *File) DeleteMustRebuild() error

DeleteMustRebuild removed the database record.

func (*File) DeletePrerequisite 

func (f *File) DeletePrerequisite(event Event, hash Hash) error

DeletePrerequisite removes a single prerequisite.

func (*File) DeleteRecords 

func (f *File) DeleteRecords() error

Delete removes a target's database records

func (*File) DependentFiles 

func (f *File) DependentFiles(prefix string) ([]*File, error)

func (*File) DoInfoCandidates 

func (f *File) DoInfoCandidates() []*DoInfo

DoInfoCandidates generates a list of possible do filenames for this file

func (*File) ErrNotFound 

func (f *File) ErrNotFound(m string) error

ErrNotFound is used when the current file is expected to exists and does not.

func (*File) ErrUninitialized 

func (f *File) ErrUninitialized() error

ErrUninitialized denotes an uninitialized directory.

func (*File) Errorf 

func (f *File) Errorf(format string, args ...interface{}) error

Errorf formats errors for the current file.

func (*File) EventDependents 

func (f *File) EventDependents(event Event) ([]*File, error)

func (*File) Exists 

func (f *File) Exists() (bool, error)

Exist verifies that the file exists on disk.

func (*File) Fullpath 

func (f *File) Fullpath() string

Fullpath returns the fully qualified path to the target file.

func (*File) GenerateNotifications 

func (f *File) GenerateNotifications(oldMeta, newMeta *Metadata) error

func (*File) Get 

func (f *File) Get(key string, obj interface{}) (bool, error)

Get returns a database record decoded into the specified type.

func (*File) GetMetadata 

func (f *File) GetMetadata() (Metadata, bool, error)

GetMetadata returns a record as a metadata structure found denotes whether the record was found.

func (*File) GetPrerequisite 

func (f *File) GetPrerequisite(event Event, hash Hash) (prereq Prerequisite, found bool, err error)

GetPrerequisite returns the prerequisite for the event and hash. If the record does not exist, found is false and err is nil.

func (*File) HasDoFile 

func (f *File) HasDoFile() bool

HasDoFile returns true if the receiver has been assigned a .do script.

func (*File) HasNullDb 

func (f *File) HasNullDb() bool

HasNullDb specifies whether the File receiver uses a NullDb.

func (*File) IsCurrent 

func (f *File) IsCurrent() (bool, error)

func (*File) IsTask 

func (f *File) IsTask() bool

IsTask denotes when the current target is a task script, either explicitly (-task argument to redo) or implicitly (name begins with @ or name is default task).

func (*File) Log 

func (f *File) Log(format string, args ...interface{})

Log prints out messages to stderr when the verbosity is greater than N.

func (*File) MustRebuild 

func (f *File) MustRebuild() bool

MustRebuild returns a boolean denoting whether the target must be rebuilt.

func (*File) NewMetadata 

func (f *File) NewMetadata() (m *Metadata, err error)

NewMetadata computes and returns the file metadata.

func (*File) NewOutput 

func (f *File) NewOutput() (*Output, error)

NewOutput returns an initialized Output

func (*File) NotifyDependents 

func (f *File) NotifyDependents(event Event) (err error)

NotifyDependents flags dependents as out of date because target has been created, modified, or deleted.

func (*File) PrerequisiteFiles 

func (f *File) PrerequisiteFiles(events ...Event) ([]*File, error)

PrerequisiteFiles returns a slice of *File objects for the file's prerequisites for the list of events.

func (*File) Prerequisites 

func (f *File) Prerequisites(events ...Event) (out []*Prerequisite, err error)

Prerequisites returns a slice of prerequisites for the file.

func (*File) Put 

func (f *File) Put(key string, obj interface{}) (err error)

Put stores a database record.

func (*File) PutDependency 

func (f *File) PutDependency(event Event, hash Hash, dep Dependent) error

func (*File) PutMetadata 

func (f *File) PutMetadata(m *Metadata) error

PutMetadata stores the file's metadata in the database.

func (*File) PutMustRebuild 

func (f *File) PutMustRebuild() error

PutMustRebuild sets the eponymous database record value.

func (*File) PutPrerequisite 

func (f *File) PutPrerequisite(event Event, hash Hash, prereq Prerequisite) error

PutPrerequisite stores the given prerequisite using a key based on the event and hash.

func (*File) Redo 

func (target *File) Redo() error

Redo finds and executes the .do file for the given target.

func (*File) RedoDir 

func (f *File) RedoDir() string

RedoDir returns the path to the .redo directory.

func (*File) RedoIfChange 

func (target *File) RedoIfChange(dependent *File) error

RedoIfChange runs redo on the target if it is out of date or its current state disagrees with its dependent's version of its state.

func (*File) RedoIfCreate 

func (target *File) RedoIfCreate(dependent *File) error

RedoIfCreate records a dependency record on a file that does not yet exist

func (*File) Rel 

func (f *File) Rel(path string) string

Rel makes path relative to f.RootDir.

func (*File) RunDoFile 

func (target *File) RunDoFile(doInfo *DoInfo) (err error)

RunDoFile executes the do file script, records the metadata for the resulting output, then saves the resulting output to the target file, if applicable. The execution is equivalent to: sh target.ext.do target.ext target outfn > out0 A well behaved .do file writes to stdout (out0) or to the $3 file (outfn), but not both.

func (*File) SetTaskFlag 

func (f *File) SetTaskFlag(isTask bool)

type FileDb 

type FileDb struct {
	DataDir string
}

FileDb is a file based DB for storing Redo relationships and metadata.

func (*FileDb) Close 

func (db *FileDb) Close() error

Close is a noop for this DB type

func (*FileDb) Delete 

func (db *FileDb) Delete(key string) error

func (*FileDb) Get 

func (db *FileDb) Get(key string) ([]byte, bool, error)

func (*FileDb) GetKeys 

func (db *FileDb) GetKeys(prefix string) ([]string, error)

GetKeys returns an array of keys that are prefixes of the specified key.

func (*FileDb) GetRecords 

func (db *FileDb) GetRecords(prefix string) ([]Record, error)

func (*FileDb) GetValues 

func (db *FileDb) GetValues(prefix string) ([][]byte, error)

GetValues returns an array of data values for keys with the specified prefix.

func (*FileDb) IsNull 

func (db *FileDb) IsNull() bool

func (*FileDb) Put 

func (db *FileDb) Put(key string, value []byte) error

type Hash 

type Hash string

func ContentHash 

func ContentHash(path string) (hash Hash, err error)

func MakeHash 

func MakeHash(content interface{}) Hash

type Metadata 

type Metadata struct {
	Path        string //not used for comparison
	ContentHash Hash
	DoFile      string
}

File Metadata.

func NewMetadata 

func NewMetadata(path string, storedPath string) (*Metadata, error)

NewMetadata returns a metadata instance for the given path. If the file is not found, nil is returned.

func (*Metadata) Equal 

func (m *Metadata) Equal(other *Metadata) bool

Equal compares metadata instances for equality.

func (Metadata) HasDoFile 

func (m Metadata) HasDoFile() bool

HasDoFile returns true if the metadata has a non-empty DoField field.

func (Metadata) IsCreated 

func (m Metadata) IsCreated(other Metadata) bool

IsCreated compares m to other to determine m represents a newly created file.

type NullDb 

type NullDb struct {
}

NullDb is a blackhole database, used for source files outside the redo project directory structure.. It never fails, all writes disappear, and reads return nothing.

func (*NullDb) Close 

func (db *NullDb) Close() error

func (*NullDb) Delete 

func (db *NullDb) Delete(key string) error

Delete removes the value stored under key. If key does not exist, the operation is a noop.

func (*NullDb) Get 

func (db *NullDb) Get(key string) ([]byte, bool, error)

Get returns the value stored under key and a boolean indicating whether the returned value was actually found in the database.

func (*NullDb) GetKeys 

func (db *NullDb) GetKeys(prefix string) ([]string, error)

GetKeys returns a list of keys which have the specified key as a prefix.

func (*NullDb) GetRecords 

func (db *NullDb) GetRecords(prefix string) ([]Record, error)

GetRecords returns a list of records (keys and data) matchign the specified key prefix.

func (*NullDb) GetValues 

func (db *NullDb) GetValues(prefix string) ([][]byte, error)

GetValues returns a list of values whose keys matching the specified key prefix.

func (*NullDb) IsNull 

func (db *NullDb) IsNull() bool

func (*NullDb) Put 

func (db *NullDb) Put(key string, value []byte) error

Put stores value under key

type Output 

type Output struct {
	*os.File
	IsArg3 bool
}

An Output is the output of a .do scripts, either through stdout or $3 (Arg3) If the .do script invocation is equivalent to the sh command, 

sh target.ext.do target.ext target tmp0 > tmp1

tmp0 and tmp1 would be outputs.

func NewOutput 

func NewOutput(file *os.File) *Output

func (*Output) Cleanup 

func (out *Output) Cleanup()

func (*Output) Copy 

func (out *Output) Copy(destDir string) (destPath string, err error)

func (*Output) SetupArg3 

func (out *Output) SetupArg3() error

type Prerequisite 

type Prerequisite struct {
	Path      string // path back to target of prerequisite.
	*Metadata        // target's metadata upon record creation.
}

Prerequisite from a source to a target.

func (Prerequisite) File 

func (p Prerequisite) File(dir string) (*File, error)

func (*Prerequisite) IsCurrent 

func (p *Prerequisite) IsCurrent(rootDir string) (isCurrent bool, err error)

type Record 

type Record struct {
	Key   string
	Value []byte
}

type RelPath 

type RelPath struct {
	// contains filtered or unexported fields
}

RelPath is a structure to simplify paths from directory upwards traversal

func (*RelPath) Add 

func (r *RelPath) Add(s string)

Add an entry

func (*RelPath) Join 

func (r *RelPath) Join() string

Reverse and Join

type Relation 

type Relation string

Dependency Relations 

const (
	SATISFIES Relation = "satisfies"
	REQUIRES  Relation = "requires"
)

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.