nin

package module
v0.0.0-...-f35891e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 12, 2022 License: Apache-2.0 Imports: 22 Imported by: 0

README

nin

Go Reference codecov

Little nin' is ninja's little sibling.

Nin is an experimental fork of ninja translated in Go.

Installation

Install go1.17 or later.

go install github.com/maruel/nin/cmd/nin@latest

Use nin where you would have used ninja (or create a symlink).

Are you serious?

Yeah.

The reason it's possible at all is because ninja is well written and has a reasonable amount of unit tests.

Why?

  • The parser can be used as a library
    • This opens the door to a lot of opportunity and a real ecosystem
  • Refactoring Go >> refactoring C++
    • As I made progress, I saw opportunities for simplification
  • Making the code concurrent (e.g. making the parser parallel) is easier
  • Plans to have it be stateful and/or do RPCs and change fundamental parts
    • E.g. call directly the RBE backend instead of shelling out reclient?
  • It's easier to keep the performance promise in check, and keep it maintainable
    • Go has native benchmarking
    • Go has native CPU and memory profiling
    • Go has native code coverage
    • Go has native documentation
  • Since it's GC, and the program runs as a one shot, we can just disable GC and save a significant amount of memory management (read: CPU) overhead.

I'll write a better roadmap if the project doesn't crash and burn.

Some people did advent of code 2021, I did a brain teaser instead.

Concerns
  • Go's slice and string bound checking slow things down, I'll have to micro optimize a bit.
  • Go's function calls are more expensive and the Go compiler inlines less often than the C++ compiler. I'll reduce the number of function calls.
  • Go's Windows layer is likely slower than raw C++, so I'll probably call raw syscall functions on Windows.
  • Staying up to date changes done upstream, especially to the file format and correctness checks.

Current state

  • Manifest (build.ninja) parsing: 5% faster on average! 📉
  • Latency: nin is in the same ballpark (-5%) for building ninja itself.
  • CPU usage: about 15% higher, has to be optimized.
  • 16 test cases out of 394 (5%) have to be fixed. git grep Skip..TODO | wc -l versus git grep "^func Test" | wc -l.
  • Closely tracking upstream as-is.
  • Flag parsing is not 100% compatible yet.

See PERF.md to learn how to measure performance yourself, since I know myself enough that I will forget to update the stats above and it will get better over time.

ninja

Ninja is a small build system with a focus on speed. https://ninja-build.org/

See the manual or doc/manual.asciidoc included in the distribution for background and more details.

Documentation

Index

Constants

View Source 
const NinjaVersion = "1.10.2.git"

NinjaVersion is the version number of the current Ninja release.

This will always be "git" on trunk.

TODO(maruel): Figure out our versioning convention.

Variables

View Source 
var (
	DefaultPool = NewPool("", 0)
	ConsolePool = NewPool("console", 1)
	PhonyRule   = NewRule("phony")
)

Well known default pools and rules.

View Source 
var Debug struct {
	// Explaining enables debug print of reason while a command is run.
	Explaining bool
	// KeepDepfile enables keeping gcc-style dependency files.
	KeepDepfile bool
	// KeepRsp enables keeping response file after commands.
	KeepRsp bool
}

Debug contains debug functionality.

Functions

func CanonicalizePath 

func CanonicalizePath(path string) string

CanonicalizePath canonicalizes a path like "foo/../bar.h" into just "bar.h".

func CanonicalizePathBits 

func CanonicalizePathBits(path string) (string, uint64)

CanonicalizePathBits canonicalizes a path like "foo/../bar.h" into just "bar.h".

Returns a bits set starting from lowest for a backslash that was normalized to a forward slash. (only used on Windows)

func GetTimeMillis 

func GetTimeMillis() int64

GetTimeMillis gets the current time as relative to some epoch.

Epoch varies between platforms; only useful for measuring elapsed time.

func HashCommand 

func HashCommand(command string) uint64

HashCommand hashes a command using the MurmurHash2 algorithm by Austin Appleby.

func IsReservedBinding 

func IsReservedBinding(v string) bool

IsReservedBinding returns true if the binding name is reserved by ninja.

func MakeDirs 

func MakeDirs(d DiskInterface, path string) error

MakeDirs create all the parent directories for path; like mkdir -p `basename path`.

func ParseDyndep 

func ParseDyndep(state *State, dyndepFile DyndepFile, filename string, input []byte) error

ParseDyndep parses a dyndep file provided as an input with null terminated string.

It updates state and dyndepFile.

func ParseManifest 

func ParseManifest(state *State, fr FileReader, options ParseManifestOpts, filename string, input []byte) error

ParseManifest parses a manifest file (i.e. build.ninja).

The input must contain a trailing terminating zero byte.

func PathDecanonicalized 

func PathDecanonicalized(path string, slashBits uint64) string

PathDecanonicalized does the reverse process of CanonicalizePath().

Only does anything on Windows.

func SpellcheckString 

func SpellcheckString(text string, words ...string) string

SpellcheckString provides the closest match to a misspelled string, given a list of correct spellings.

Returns "" if there is no close enough match.

Types

type BindingEnv 

type BindingEnv struct {
	Bindings map[string]string
	Rules    map[string]*Rule
	Parent   *BindingEnv
}

BindingEnv is an Env which contains a mapping of variables to values as well as a pointer to a parent scope.

func NewBindingEnv 

func NewBindingEnv(parent *BindingEnv) *BindingEnv

NewBindingEnv returns an initialized BindingEnv.

func (*BindingEnv) LookupRule 

func (b *BindingEnv) LookupRule(ruleName string) *Rule

LookupRule returns a rule by name.

func (*BindingEnv) LookupVariable 

func (b *BindingEnv) LookupVariable(v string) string

LookupVariable returns a variable's value.

func (*BindingEnv) String 

func (b *BindingEnv) String() string

String serializes the bindings.

type BuildConfig 

type BuildConfig struct {
	Verbosity       Verbosity
	DryRun          bool
	Parallelism     int
	FailuresAllowed int
	// The maximum load average we must not exceed. A negative or zero value
	// means that we do not have any limit.
	MaxLoadAvg float64
}

BuildConfig are the options (e.g. verbosity, parallelism) passed to a build.

func NewBuildConfig 

func NewBuildConfig() BuildConfig

NewBuildConfig returns the default build configuration.

type BuildLog 

type BuildLog struct {
	Entries map[string]*LogEntry
	// contains filtered or unexported fields
}

BuildLog stores a log of every command ran for every build.

It has a few uses:

1) (hashes of) command lines for existing output files, so we know when we need to rebuild due to the command changing.

2) timing information, perhaps for generating reports.

3) restat information.

func NewBuildLog 

func NewBuildLog() BuildLog

NewBuildLog returns an initialized BuidLog.

func (*BuildLog) Close 

func (b *BuildLog) Close() error

Close closes the file handle.

func (*BuildLog) Load 

func (b *BuildLog) Load(path string) (LoadStatus, error)

Load the on-disk log.

It can return a warning with success and an error.

LoadNotFound is only returned when os.IsNotExist(err) is true.

func (*BuildLog) OpenForWrite 

func (b *BuildLog) OpenForWrite(path string, user BuildLogUser) error

OpenForWrite prepares writing to the log file without actually opening it - that will happen when/if it's needed.

func (*BuildLog) Recompact 

func (b *BuildLog) Recompact(path string, user BuildLogUser) error

Recompact rewrites the known log entries, throwing away old data.

func (*BuildLog) RecordCommand 

func (b *BuildLog) RecordCommand(edge *Edge, startTime, endTime int32, mtime TimeStamp) error

RecordCommand records an edge.

func (*BuildLog) Restat 

func (b *BuildLog) Restat(path string, di DiskInterface, outputs []string) error

Restat recompacts but stat()'s all outputs in the log.

type BuildLogUser 

type BuildLogUser interface {
	IsPathDead(s string) bool
}

BuildLogUser answers questions about the manifest for the BuildLog.

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

Builder wraps the build process: starting commands, updating status.

func NewBuilder 

func NewBuilder(state *State, config *BuildConfig, buildLog *BuildLog, depsLog *DepsLog, di DiskInterface, status Status, startTimeMillis int64) *Builder

NewBuilder returns an initialized Builder.

func (*Builder) AddTarget 

func (b *Builder) AddTarget(target *Node) (bool, error)

AddTarget adds a target to the build, scanning dependencies.

Returns true if the target is dirty. Returns false and no error if the target is up to date.

func (*Builder) AlreadyUpToDate 

func (b *Builder) AlreadyUpToDate() bool

AlreadyUpToDate returns true if the build targets are already up to date.

func (*Builder) Build 

func (b *Builder) Build() error

Build runs the build.

It is an error to call this function when AlreadyUpToDate() is true.

type CLParser 

type CLParser struct {
	// contains filtered or unexported fields
}

CLParser parses Visual Studio's cl.exe dependency output.

It requires some massaging to work with Ninja; for example, it emits include information on stderr in a funny format when building with /showIncludes. This class parses this output.

func NewCLParser 

func NewCLParser() CLParser

NewCLParser returns an initialized CLParser.

func (*CLParser) Parse 

func (c *CLParser) Parse(output, depsPrefix string, filteredOutput *string) error

Parse the full output of cl, filling filteredOutput with the text that should be printed (if any). Returns true on success, or false with err filled. output must not be the same object as filteredObject.

type Cleaner 

type Cleaner struct {
	// contains filtered or unexported fields
}

Cleaner cleans a build directory.

func NewCleaner 

func NewCleaner(state *State, config *BuildConfig, di DiskInterface) *Cleaner

NewCleaner returns an initialized cleaner.

func (*Cleaner) CleanAll 

func (c *Cleaner) CleanAll(generator bool) int

CleanAll cleans all built files, except for files created by generator rules.

If generator is set, also clean files created by generator rules.

Return non-zero if an error occurs.

func (*Cleaner) CleanDead 

func (c *Cleaner) CleanDead(entries map[string]*LogEntry) int

CleanDead cleans the files produced by previous builds that are no longer in the manifest.

Returns non-zero if an error occurs.

func (*Cleaner) CleanRule 

func (c *Cleaner) CleanRule(rule *Rule) int

CleanRule cleans the file produced by the given rule.

Returns non-zero if an error occurs.

func (*Cleaner) CleanRuleName 

func (c *Cleaner) CleanRuleName(rule string) int

CleanRuleName cleans the file produced by the given rule.

Returns non-zero if an error occurs.

func (*Cleaner) CleanRules 

func (c *Cleaner) CleanRules(rules []string) int

CleanRules cleans the file produced by the given rules.

Returns non-zero if an error occurs.

func (*Cleaner) CleanTargets 

func (c *Cleaner) CleanTargets(targets []string) int

CleanTargets cleans the given target targets.

Return non-zero if an error occurs.

func (*Cleaner) Reset 

func (c *Cleaner) Reset()

Reset reinitializes the cleaner stats.

type DependencyScan 

type DependencyScan struct {
	// contains filtered or unexported fields
}

DependencyScan manages the process of scanning the files in a graph and updating the dirty/outputsReady state of all the nodes and edges.

func NewDependencyScan 

func NewDependencyScan(state *State, buildLog *BuildLog, depsLog *DepsLog, di DiskInterface) DependencyScan

NewDependencyScan returns an initialized DependencyScan.

func (*DependencyScan) LoadDyndeps 

func (d *DependencyScan) LoadDyndeps(node *Node, ddf DyndepFile) error

LoadDyndeps loads a dyndep file from the given node's path and update the build graph with the new information.

The 'DyndepFile' object stores the information loaded from the dyndep file.

func (*DependencyScan) RecomputeDirty 

func (d *DependencyScan) RecomputeDirty(initialNode *Node) ([]*Node, error)

RecomputeDirty updates the |dirty| state of the given Node by transitively inspecting their input edges.

Examine inputs, outputs, and command lines to judge whether an edge needs to be re-run, and update OutputsReady and each outputs' Dirty state accordingly.

Appends any validation nodes found to the nodes parameter.

type DepfileParser 

type DepfileParser struct {
	// contains filtered or unexported fields
}

DepfileParser is the parser for the dependency information emitted by gcc's -M flags.

func (*DepfileParser) Parse 

func (d *DepfileParser) Parse(content []byte) error

Parse parses a dependency file.

content must contain a terminating zero byte.

Warning: mutate the slice content in-place.

A note on backslashes in Makefiles, from reading the docs: Backslash-newline is the line continuation character. Backslash-# escapes a # (otherwise meaningful as a comment start). Backslash-% escapes a % (otherwise meaningful as a special). Finally, quoting the GNU manual, "Backslashes that are not in danger of quoting ‘%’ characters go unmolested." How do you end a line with a backslash? The netbsd Make docs suggest reading the result of a shell command echoing a backslash!

Rather than implement all of above, we follow what GCC/Clang produces: Backslashes escape a space or hash sign. When a space is preceded by 2N+1 backslashes, it is represents N backslashes followed by space. When a space is preceded by 2N backslashes, it represents 2N backslashes at the end of a filename. A hash sign is escaped by a single backslash. All other backslashes remain unchanged.

If anyone actually has depfiles that rely on the more complicated behavior we can adjust this.

type Deps 

type Deps struct {
	MTime TimeStamp
	Nodes []*Node
}

Deps is the reading (startup-time) struct.

func NewDeps 

func NewDeps(mtime TimeStamp, nodeCount int) *Deps

NewDeps returns an initialized Deps.

type DepsLog 

type DepsLog struct {
	// Maps id -> Node.
	Nodes []*Node
	// Maps id -> Deps of that id.
	Deps []*Deps
	// contains filtered or unexported fields
}

DepsLog represents a .ninja_deps log file to accelerate incremental build.

As build commands run they can output extra dependency information (e.g. header dependencies for C source) dynamically. DepsLog collects that information at build time and uses it for subsequent builds.

The on-disk format is based on two primary design constraints:

- it must be written to as a stream (during the build, which may be interrupted);

- it can be read all at once on startup. (Alternative designs, where it contains indexing information, were considered and discarded as too complicated to implement; if the file is small than reading it fully on startup is acceptable.)

Here are some stats from the Windows Chrome dependency files, to help guide the design space. The total text in the files sums to 90mb so some compression is warranted to keep load-time fast. There's about 10k files worth of dependencies that reference about 40k total paths totalling 2mb of unique strings.

Based on these stats, here's the current design.

The file is structured as version header followed by a sequence of records. Each record is either a path string or a dependency list. Numbering the path strings in file order gives them dense integer ids. A dependency list maps an output id to a list of input ids.

Concretely, a record is: 

four bytes record length, high bit indicates record type
  (but max record sizes are capped at 512kB)
path records contain the string name of the path, followed by up to 3
  padding bytes to align on 4 byte boundaries, followed by the
  one's complement of the expected index of the record (to detect
  concurrent writes of multiple ninja processes to the log).
dependency records are an array of 4-byte integers
  [output path id,
   output path mtime (lower 4 bytes), output path mtime (upper 4 bytes),
   input path id, input path id...]
  (The mtime is compared against the on-disk output path mtime
  to verify the stored data is up-to-date.)

If two records reference the same output the latter one in the file wins, allowing updates to just be appended to the file. A separate repacking step can run occasionally to remove dead records.

func (*DepsLog) Close 

func (d *DepsLog) Close() error

Close closes the file handle.

func (*DepsLog) GetDeps 

func (d *DepsLog) GetDeps(node *Node) *Deps

GetDeps returns the Deps for this node ID.

Silently ignore invalid node ID.

func (*DepsLog) GetFirstReverseDepsNode 

func (d *DepsLog) GetFirstReverseDepsNode(node *Node) *Node

GetFirstReverseDepsNode returns something?

TODO(maruel): Understand better.

func (*DepsLog) IsDepsEntryLiveFor 

func (d *DepsLog) IsDepsEntryLiveFor(node *Node) bool

IsDepsEntryLiveFor returns if the deps entry for a node is still reachable from the manifest.

The deps log can contain deps entries for files that were built in the past but are no longer part of the manifest. This function returns if this is the case for a given node. This function is slow, don't call it from code that runs on every build.

func (*DepsLog) Load 

func (d *DepsLog) Load(path string, state *State) (LoadStatus, error)

Load loads a .ninja_deps to accelerate incremental build.

Note: For version differences, this should migrate to the new format. But the v1 format could sometimes (rarely) end up with invalid data, so don't migrate v1 to v3 to force a rebuild. (v2 only existed for a few days, and there was no release with it, so pretend that it never happened.)

Warning: the whole file content is kept alive.

TODO(maruel): Make it an option so that when used as a library it doesn't become a memory bloat. This is especially important when recompacting.

func (*DepsLog) OpenForWrite 

func (d *DepsLog) OpenForWrite(path string) error

OpenForWrite prepares writing to the log file without actually opening it - that will happen when/if it's needed.

func (*DepsLog) Recompact 

func (d *DepsLog) Recompact(path string) error

Recompact rewrites the known log entries, throwing away old data.

type DiskInterface 

type DiskInterface interface {
	FileReader
	// Stat stat()'s a file, returning the mtime, or 0 if missing and -1 on
	// other errors.
	Stat(path string) (TimeStamp, error)

	// MakeDir creates a directory, returning false on failure.
	MakeDir(path string) error

	// WriteFile creates a file, with the specified name and contents
	WriteFile(path, contents string) error

	// RemoveFile removes the file named path.
	//
	// It should return an error that matches os.IsNotExist() if the file was not
	// present.
	RemoveFile(path string) error
}

DiskInterface is an interface for accessing the disk.

Abstract so it can be mocked out for tests. The real implementation is RealDiskInterface.

type DyndepFile 

type DyndepFile map[*Edge]*Dyndeps

DyndepFile stores data loaded from one dyndep file.

Map from an edge to its dynamically-discovered dependency information.

type DyndepLoader 

type DyndepLoader struct {
	// contains filtered or unexported fields
}

DyndepLoader loads dynamically discovered dependencies, as referenced via the "dyndep" attribute in build files.

func NewDyndepLoader 

func NewDyndepLoader(state *State, di DiskInterface) DyndepLoader

NewDyndepLoader returns an initialized DyndepLoader.

func (*DyndepLoader) LoadDyndeps 

func (d *DyndepLoader) LoadDyndeps(node *Node, ddf DyndepFile) error

LoadDyndeps loads a dyndep file from the given node's path and update the build graph with the new information.

Caller can optionally provide a 'DyndepFile' object in which to store the information loaded from the dyndep file.

type Dyndeps 

type Dyndeps struct {
	// contains filtered or unexported fields
}

Dyndeps stores dynamically-discovered dependency information for one edge.

func (*Dyndeps) String 

func (d *Dyndeps) String() string

type Edge 

type Edge struct {
	Inputs      []*Node
	Outputs     []*Node
	Validations []*Node
	Rule        *Rule
	Pool        *Pool
	Dyndep      *Node
	Env         *BindingEnv
	Mark        VisitMark
	ID          int32

	// There are three types of inputs.
	// 1) explicit deps, which show up as $in on the command line;
	// 2) implicit deps, which the target depends on implicitly (e.g. C headers),
	//                   and changes in them cause the target to rebuild;
	// 3) order-only deps, which are needed before the target builds but which
	//                     don't cause the target to rebuild.
	// These are stored in Inputs in that order, and we keep counts of
	// #2 and #3 when we need to access the various subsets.
	ImplicitDeps  int32
	OrderOnlyDeps int32

	// There are two types of outputs.
	// 1) explicit outs, which show up as $out on the command line;
	// 2) implicit outs, which the target generates but are not part of $out.
	// These are stored in Outputs in that order, and we keep a count of
	// #2 to use when we need to access the various subsets.
	ImplicitOuts int32

	OutputsReady         bool
	DepsLoaded           bool
	DepsMissing          bool
	GeneratedByDepLoader bool
}

Edge is an edge in the dependency graph; links between Nodes using Rules.

func (*Edge) Dump 

func (e *Edge) Dump(prefix string)

Dump prints the Edge details to stdout.

func (*Edge) EvaluateCommand 

func (e *Edge) EvaluateCommand(inclRspFile bool) string

EvaluateCommand expands all variables in a command and return it as a string.

If inclRspFile is enabled, the string will also contain the full contents of a response file (if applicable)

func (*Edge) GetBinding 

func (e *Edge) GetBinding(key string) string

GetBinding returns the shell-escaped value of |key|.

func (*Edge) GetUnescapedDepfile 

func (e *Edge) GetUnescapedDepfile() string

GetUnescapedDepfile returns like GetBinding("depfile"), but without shell escaping.

func (*Edge) GetUnescapedDyndep 

func (e *Edge) GetUnescapedDyndep() string

GetUnescapedDyndep returns like GetBinding("dyndep"), but without shell escaping.

func (*Edge) GetUnescapedRspfile 

func (e *Edge) GetUnescapedRspfile() string

GetUnescapedRspfile returns like GetBinding("rspfile"), but without shell escaping.

func (*Edge) IsImplicit 

func (e *Edge) IsImplicit(index int) bool

IsImplicit returns if the inputs at the specified index is implicit and not for ordering only.

func (*Edge) IsOrderOnly 

func (e *Edge) IsOrderOnly(index int) bool

IsOrderOnly returns if the input at the specified index is only used for ordering.

type EdgeSet 

type EdgeSet struct {
	// contains filtered or unexported fields
}

EdgeSet acts as a sorted set of *Edge, so map[*Edge]struct{} but with sorted pop.

func NewEdgeSet 

func NewEdgeSet() *EdgeSet

NewEdgeSet returns an initialized EdgeSet.

func (*EdgeSet) Add 

func (e *EdgeSet) Add(ed *Edge)

Add the edge to the set.

func (*EdgeSet) IsEmpty 

func (e *EdgeSet) IsEmpty() bool

IsEmpty return true if the set is empty.

func (*EdgeSet) Pop 

func (e *EdgeSet) Pop() *Edge

Pop returns the lowest ID.

type Env 

type Env interface {
	LookupVariable(v string) string
}

Env is an interface for a scope for variable (e.g. "$foo") lookups.

type EvalString 

type EvalString struct {
	Parsed []EvalStringToken
}

EvalString is a tokenized string that contains variable references.

Can be evaluated relative to an Env.

func (*EvalString) Evaluate 

func (e *EvalString) Evaluate(env Env) string

Evaluate returns the evaluated string with variable expanded using value found in environment env.

func (*EvalString) Serialize 

func (e *EvalString) Serialize() string

Serialize constructs a human-readable representation of the parsed state.

Used in tests.

func (*EvalString) String 

func (e *EvalString) String() string

func (*EvalString) Unparse 

func (e *EvalString) Unparse() string

Unparse returns the string with variables not expanded.

Used for diagnostics.

type EvalStringToken 

type EvalStringToken struct {
	Value     string
	IsSpecial bool
}

EvalStringToken is one token in the EvalString list of items.

func (*EvalStringToken) String 

func (t *EvalStringToken) String() string

type ExistenceStatus 

type ExistenceStatus int32

ExistenceStatus represents the knowledge of the file's existence. 

const (
	// ExistenceStatusUnknown means the file hasn't been examined.
	ExistenceStatusUnknown ExistenceStatus = iota
	// ExistenceStatusMissing means the file doesn't exist. MTime will be the
	// latest mtime of its dependencies.
	ExistenceStatusMissing
	// ExistenceStatusExists means the path is an actual file. MTime will be the
	// file's mtime.
	ExistenceStatusExists
)

type ExitStatus 

type ExitStatus = int

ExitStatus is well known process exit code. 

const (
	ExitSuccess ExitStatus = iota
	ExitFailure
	ExitInterrupted
)

Well know process exit codes.

type FileReader 

type FileReader interface {
	// ReadFile reads a file and returns its content.
	//
	// Unlike os.ReadFile(), if the content is not empty, it appends a zero byte
	// at the end of the slice.
	ReadFile(path string) ([]byte, error)
}

FileReader is an interface for reading files from disk.

See DiskInterface for details. This base offers the minimum interface needed just to read files.

type GraphViz 

type GraphViz struct {
	// contains filtered or unexported fields
}

GraphViz is the object to initialize the parameters to create GraphViz .dot file output.

func NewGraphViz 

func NewGraphViz(state *State, di DiskInterface) GraphViz

NewGraphViz returns an initialized GraphViz.

func (*GraphViz) AddTarget 

func (g *GraphViz) AddTarget(node *Node)

AddTarget adds a node to include in the graph.

func (*GraphViz) Finish 

func (g *GraphViz) Finish()

Finish prints out the footer.

func (*GraphViz) Start 

func (g *GraphViz) Start()

Start prints out the header.

type LoadStatus 

type LoadStatus int32

LoadStatus is the return code when loading a build log or a deps log. 

const (
	LoadError LoadStatus = iota
	LoadSuccess
	LoadNotFound
)

Valid LoadStatus values.

type LogEntry 

type LogEntry struct {
	// contains filtered or unexported fields
}

LogEntry is an entry in BuildLog.

func (*LogEntry) Equal 

func (l *LogEntry) Equal(r *LogEntry) bool

Equal compares two LogEntry.

func (*LogEntry) Serialize 

func (l *LogEntry) Serialize(w io.Writer) error

Serialize writes an entry into a log file as a text form.

type MetricsCollection 

type MetricsCollection struct {
	// contains filtered or unexported fields
}

MetricsCollection collects metrics. 

var Metrics MetricsCollection

Metrics is the singleton that stores metrics for this package.

func (*MetricsCollection) Enable 

func (m *MetricsCollection) Enable()

Enable enables metrics collection.

Must be called before using any other functionality in this package.

func (*MetricsCollection) Report 

func (m *MetricsCollection) Report()

Report prints a summary report to stdout.

type MissingDependencyScanner 

type MissingDependencyScanner struct {
	// contains filtered or unexported fields
}

MissingDependencyScanner is a scanner for missing dependencies.

func NewMissingDependencyScanner 

func NewMissingDependencyScanner(delegate MissingDependencyScannerDelegate, depsLog *DepsLog, state *State, di DiskInterface) MissingDependencyScanner

NewMissingDependencyScanner returns an initialized MissingDependencyScanner.

func (*MissingDependencyScanner) HadMissingDeps 

func (m *MissingDependencyScanner) HadMissingDeps() bool

HadMissingDeps return true if there were any missing dependencies found.

func (*MissingDependencyScanner) PrintStats 

func (m *MissingDependencyScanner) PrintStats()

PrintStats prints statistics to stdout.

func (*MissingDependencyScanner) ProcessNode 

func (m *MissingDependencyScanner) ProcessNode(node *Node)

ProcessNode does something?

TODO(maruel): Figure out.

type MissingDependencyScannerDelegate 

type MissingDependencyScannerDelegate interface {
	OnMissingDep(node *Node, path string, generator *Rule)
}

MissingDependencyScannerDelegate is a callback when a missing dependency is found.

type Node 

type Node struct {

	// Path is the path of the file that this node represents.
	Path string

	// Set bits starting from lowest for backslashes that were normalized to
	// forward slashes by CanonicalizePathBits. See |PathDecanonicalized|.
	SlashBits uint64

	// The Edge that produces this Node, or NULL when there is no
	// known edge to produce it.
	InEdge *Edge

	// All Edges that use this Node as an input.
	OutEdges []*Edge

	// All Edges that use this Node as a validation.
	ValidationOutEdges []*Edge

	// Possible values of MTime:
	//   -1: file hasn't been examined
	//   0:  we looked, and file doesn't exist
	//   >0: actual file's mtime, or the latest mtime of its dependencies if it doesn't exist
	MTime TimeStamp

	// A dense integer id for the node, assigned and used by DepsLog.
	ID int32

	Exists ExistenceStatus

	// Dirty is true when the underlying file is out-of-date.
	// But note that Edge.OutputsReady is also used in judging which
	// edges to build.
	Dirty bool

	// Store whether dyndep information is expected from this node but
	// has not yet been loaded.
	DyndepPending bool
}

Node represents information about a node in the dependency graph: the file, whether it's dirty, mtime, etc.

func (*Node) Dump 

func (n *Node) Dump(prefix string)

Dump prints out Node's details to stdout.

func (*Node) PathDecanonicalized 

func (n *Node) PathDecanonicalized() string

PathDecanonicalized return |Path| but use SlashBits to convert back to original slash styles.

func (*Node) Stat 

func (n *Node) Stat(di DiskInterface) error

Stat stat's the file.

type ParseManifestConcurrency 

type ParseManifestConcurrency int32

ParseManifestConcurrency defines the concurrency parameters when parsing manifest (build.ninja files). 

const (
	// ParseManifestSerial parses files serially in the same way that the C++
	// ninja implementation does. It is the most compatible way.
	//
	// This reduces CPU usage, at the cost of higher latency.
	ParseManifestSerial ParseManifestConcurrency = iota
	// ParseManifestPrewarmSubninja loads files serially except that subninjas
	// are processed at the very end. This gives a latency improvement when a
	// significant number of subninjas are processed because subninja files can
	// be read from disk concurrently. This causes subninja files to be processed
	// out of order.
	ParseManifestPrewarmSubninja
	// ParseManifestConcurrentParsing parses all subninjas concurrently.
	ParseManifestConcurrentParsing
)

func (ParseManifestConcurrency) String 

func (p ParseManifestConcurrency) String() string

type ParseManifestOpts 

type ParseManifestOpts struct {
	// ErrOnDupeEdge causes duplicate rules for one target to print an error,
	// otherwise warns.
	ErrOnDupeEdge bool
	// ErrOnPhonyCycle causes phony cycles to print an error, otherwise warns.
	ErrOnPhonyCycle bool
	// Quiet silences warnings.
	Quiet bool
	// Concurrency defines the parsing concurrency.
	Concurrency ParseManifestConcurrency
}

ParseManifestOpts are the options when parsing a build.ninja file.

type Pool 

type Pool struct {
	Name string
	// contains filtered or unexported fields
}

Pool is a pool for delayed edges.

Pools are scoped to a State. Edges within a State will share Pools. A Pool will keep a count of the total 'weight' of the currently scheduled edges. If a Plan attempts to schedule an Edge which would cause the total weight to exceed the depth of the Pool, the Pool will enqueue the Edge instead of allowing the Plan to schedule it. The Pool will relinquish queued Edges when the total scheduled weight diminishes enough (i.e. when a scheduled edge completes).

func NewPool 

func NewPool(name string, depth int) *Pool

NewPool returns an initialized Pool.

func (*Pool) Dump 

func (p *Pool) Dump()

Dump the Pool and its edges (useful for debugging).

type RealDiskInterface 

type RealDiskInterface struct {
	// contains filtered or unexported fields
}

RealDiskInterface is the implementation of DiskInterface that actually hits the disk.

func (*RealDiskInterface) AllowStatCache 

func (r *RealDiskInterface) AllowStatCache(allow bool)

AllowStatCache sets whether stat information can be cached.

Only has an effect on Windows.

func (*RealDiskInterface) MakeDir 

func (r *RealDiskInterface) MakeDir(path string) error

MakeDir implements DiskInterface.

func (*RealDiskInterface) ReadFile 

func (r *RealDiskInterface) ReadFile(path string) ([]byte, error)

ReadFile implements DiskInterface.

func (*RealDiskInterface) RemoveFile 

func (r *RealDiskInterface) RemoveFile(path string) error

RemoveFile implements DiskInterface.

func (*RealDiskInterface) Stat 

func (r *RealDiskInterface) Stat(path string) (TimeStamp, error)

Stat implements DiskInterface.

func (*RealDiskInterface) WriteFile 

func (r *RealDiskInterface) WriteFile(path string, contents string) error

WriteFile implements DiskInterface.

type Result 

type Result struct {
	Edge     *Edge
	ExitCode ExitStatus
	Output   string
}

Result is the result of waiting for a command.

type Rule 

type Rule struct {
	Name     string
	Bindings map[string]*EvalString
}

Rule is an invocable build command and associated metadata (description, etc.).

func NewRule 

func NewRule(name string) *Rule

NewRule returns an initialized Rule.

func (*Rule) String 

func (r *Rule) String() string

type State 

type State struct {
	// Mapping of path -> Node.
	Paths map[string]*Node

	// All the Pools used in the graph.
	Pools map[string]*Pool

	// All the Edges of the graph.
	Edges []*Edge

	Bindings *BindingEnv
	Defaults []*Node
}

State is the global state (file status) for a single run.

func NewState 

func NewState() State

NewState returns an initialized State.

It is preloaded with PhonyRule, and DefaultPool and ConsolePool.

func (*State) DefaultNodes 

func (s *State) DefaultNodes() []*Node

DefaultNodes returns the default nodes to build.

If none are defined, returns all the root nodes.

func (*State) Dump 

func (s *State) Dump()

Dump the nodes and Pools (useful for debugging).

func (*State) GetNode 

func (s *State) GetNode(path string, slashBits uint64) *Node

GetNode returns the existing node for this path.

If the node doesn't exist, create it and return it.

func (*State) Reset 

func (s *State) Reset()

Reset state. Keeps all nodes and edges, but restores them to the state where we haven't yet examined the disk for dirty state.

func (*State) RootNodes 

func (s *State) RootNodes() []*Node

RootNodes return the root node(s) of the graph.

Root nodes have no output edges.

Returns an empty slice if no root node is found.

func (*State) SpellcheckNode 

func (s *State) SpellcheckNode(path string) *Node

SpellcheckNode returns the node with the closest name.

type Status 

type Status interface {
	PlanHasTotalEdges(total int)
	BuildEdgeStarted(edge *Edge, startTimeMillis int32)
	BuildEdgeFinished(edge *Edge, endTimeMillis int32, success bool, output string)
	BuildLoadDyndeps()
	BuildStarted()
	BuildFinished()

	Info(msg string, i ...interface{})
	Warning(msg string, i ...interface{})
	Error(msg string, i ...interface{})
}

Status is the interface that tracks the status of a build: completion fraction, printing updates.

type TimeStamp 

type TimeStamp int64

TimeStamp is the timestamp of a file.

When considering file modification times we only care to compare them against one another -- we never convert them to an absolute real time. On POSIX we use timespec (seconds&nanoseconds since epoch) and on Windows we use a different value. Both fit in an int64.

type Token 

type Token int32
const (
	ERROR Token = iota
	BUILD
	COLON
	DEFAULT
	EQUALS
	IDENT
	INCLUDE
	INDENT
	NEWLINE
	PIPE
	PIPE2
	PIPEAT
	POOL
	RULE
	SUBNINJA
	TEOF
)

func (Token) String 

func (t Token) String() string

String() returns a human-readable form of a token, used in error messages.

type Verbosity 

type Verbosity int32

Verbosity controls the verbosity of the status updates. 

const (
	// Quiet means no output -- used when testing.
	Quiet Verbosity = iota
	// NoStatusUpdate means just regular output but suppress status update.
	NoStatusUpdate
	// Normal provides regular output and status update.
	Normal
	// Verbose prints out commands executed.
	Verbose
)

type VisitMark 

type VisitMark int32

VisitMark is a market to determine if an edge is visited. 

const (
	VisitNone VisitMark = iota
	VisitInStack
	VisitDone
)

Valid VisitMark values.

type Want 

type Want int32

Want enumerates possible steps we want for an edge. 

const (
	// WantNothing means we do not want to build the edge, but we might want to
	// build one of its dependents.
	WantNothing Want = iota
	// WantToStart means we want to build the edge, but have not yet scheduled
	// it.
	WantToStart
	// WantToFinish means we want to build the edge, have scheduled it, and are
	// waiting for it to complete.
	WantToFinish
)

Source Files

View all Source files

Directories

Path Synopsis
cmd
build_log_perftest
depfile_parser_perftest
hash_collision_bench
manifest_parser_perftest
Tests manifest parser performance.
 Tests manifest parser performance.
nin

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.