luci: Index | Files | Directories

package lucicfg

import ""

Package lucicfg contains LUCI config generator.

All Starlark code is executed sequentially in a single goroutine from inside Generate function, thus this package doesn't used any mutexes or other synchronization primitives. It is safe to call Generate concurrently though, since there's no global shared state, each Generate call operates on its own state.


Package Files

configset.go duration.go errors.go genctx.go generator.go io.go meta.go native.go protos.go sequence.go state.go strutil.go tracked.go version.go


const (
    TOK_LB    = iota // non-escaped '{'
    TOK_RB           // non-escaped '}'
    TOK_NUM          // a sequence of digits
    TOK_RUNES        // an arbitrary sequence of non-special runes
    TOK_COMMA        // ','
    TOK_DOTS         // '..'
const (
    // Version is the version of lucicfg tool.
    // It ends up in CLI output and in User-Agent headers.
    Version = "1.3.1"

    // UserAgent is used for User-Agent header in HTTP requests from lucicfg.
    UserAgent = "lucicfg v" + Version

func FindTrackedFiles Uses

func FindTrackedFiles(dir string, patterns []string) ([]string, error)

FindTrackedFiles recursively discovers all regular files in the given directory whose names match given patterns.

See TrackedSet for the format of `patterns`. If the directory doesn't exist, returns empty slice.

Returned file names are sorted, slash-separated and relative to `dir`.

func TrackedSet Uses

func TrackedSet(patterns []string) func(string) (bool, error)

TrackedSet returns a predicate that classifies whether a slash-separated path belongs to a tracked set or not.

Each entry in `patterns` is either `<glob pattern>` (a "positive" glob) or `!<glob pattern>` (a "negative" glob). A path is considered tracked if its base name matches any of the positive globs and none of the negative globs. If `patterns` is empty, no paths are considered tracked. If all patterns are negative, single `*` positive pattern is implied as well.

The predicate returns an error if some pattern is malformed.

type BacktracableError Uses

type BacktracableError interface {

    // Backtrace returns a user-friendly error message describing the stack
    // of calls that led to this error, along with the error message itself.
    Backtrace() string

BacktracableError is an error that has a starlark backtrace attached to it.

Implemented by Error here, by starlark.EvalError and graph errors.

type ConfigSet Uses

type ConfigSet map[string][]byte

ConfigSet is an in-memory representation of a config set.

Keys are slash-separated filenames, values are corresponding file bodies.

func ReadConfigSet Uses

func ReadConfigSet(dir string) (ConfigSet, error)

ReadConfigSet reads all regular files in the given directory (recursively) and returns them as a ConfigSet.

func (ConfigSet) Compare Uses

func (cs ConfigSet) Compare(dir string) (changed, unchanged []string)

Compare compares files on disk to what's in the config set.

Returns names of files that are different ('changed') and same ('unchanged').

Files on disk that are not in the config set are totally ignored. Files that cannot be read are considered outdated.

func (ConfigSet) DebugDump Uses

func (cs ConfigSet) DebugDump()

DebugDump writes the config files to stdout in a format useful for debugging.

func (ConfigSet) Digests Uses

func (cs ConfigSet) Digests() map[string]string

Digests returns a map "config name -> hex SHA256 of its body".

func (ConfigSet) DiscardChangesToUntracked Uses

func (cs ConfigSet) DiscardChangesToUntracked(ctx context.Context, tracked []string, dir string) error

DiscardChangesToUntracked replaces bodies of the files that are in the config set, but not in the `tracked` set (per TrackedSet semantics) with what's on disk in the given `dir`.

This allows to construct partially generated config set: some configs (the ones in the tracked set) are generated, others are loaded from disk.

If `dir` is "-" (which indicates that the config set is going to be dumped to stdout rather then to disk), just removes untracked files from the config set.

func (ConfigSet) Files Uses

func (cs ConfigSet) Files() []string

Files returns a sorted list of file names in the config set.

func (ConfigSet) Validate Uses

func (cs ConfigSet) Validate(ctx context.Context, name string, val ConfigSetValidator) (*ValidationResult, error)

Validate sends the config set for validation to LUCI Config service.

'name' is a name of this config set from LUCI Config point of view, e.g. "projects/<something>" or "services/<something>". It tells LUCI Config how to validate files in the set.

Returns an error only if the validation call itself failed (e.g. LUCI Config was unreachable). Otherwise returns ValidationResult with a list of validation message (errors, warnings, etc). The list of messages may be empty if the config set is 100% valid.

func (ConfigSet) Write Uses

func (cs ConfigSet) Write(dir string) (changed, unchanged []string, err error)

Write updates files on disk to match the config set.

Returns a list of updated files and a list of files that are already up-to-date, same as Compare.

Creates missing directories. Not atomic. All files have mode 0666.

type ConfigSetValidator Uses

type ConfigSetValidator interface {
    // Validate sends the validation request to the service.
    Validate(ctx context.Context, req *ValidationRequest) (*ValidationResult, error)

ConfigSetValidator is primarily implemented through config.Service, but can also be mocked in tests.

func RemoteValidator Uses

func RemoteValidator(svc *config.Service) ConfigSetValidator

RemoteValidator returns ConfigSetValidator that makes RPCs to LUCI Config.

type Error Uses

type Error struct {
    Msg   string
    Stack *builtins.CapturedStacktrace

Error is a single error message emitted by the config generator.

It holds a stack trace responsible for the error.

func (*Error) Backtrace Uses

func (e *Error) Backtrace() string

Backtrace is part of BacktracableError interface.

func (*Error) Error Uses

func (e *Error) Error() string

Error is part of 'error' interface.

type Inputs Uses

type Inputs struct {
    Code  interpreter.Loader // a package with the user supplied code
    Entry string             // a name of the entry point script in this package

    TextPBHeader string // a header to put into generated textpb files
    // contains filtered or unexported fields

Inputs define all inputs for the config generator.

type Meta Uses

type Meta struct {
    ConfigServiceHost string   `json:"config_service_host"` // LUCI config host name
    ConfigSet         string   `json:"config_set"`          // e.g. "project/<name>"
    ConfigDir         string   `json:"config_dir"`          // output directory to place generated files or '-' for stdout
    TrackedFiles      []string `json:"tracked_files"`       // e.g. ["*.cfg", "!*-dev.cfg"]
    FailOnWarnings    bool     `json:"fail_on_warnings"`    // true to treat validation warnings as errors
    // contains filtered or unexported fields

Meta contains configuration for the configuration generator itself.

It influences how generator produces output configs. It is settable through lucicfg.config(...) statements on the Starlark side or through command line flags. Command line flags override what was set via lucicfg.config(...).

See @stdlib//internal/ for full meaning of fields.

func (*Meta) AddFlags Uses

func (m *Meta) AddFlags(fs *flag.FlagSet)

AddFlags registers command line flags that correspond to Meta fields.

func (*Meta) Log Uses

func (m *Meta) Log(ctx context.Context)

Log logs the values of the meta parameters to Debug logger.

func (*Meta) PopulateFromTouchedIn Uses

func (m *Meta) PopulateFromTouchedIn(t *Meta)

PopulateFromTouchedIn takes all touched values in `t` and copies them to `m`, overriding what's in `m`.

func (*Meta) RebaseConfigDir Uses

func (m *Meta) RebaseConfigDir(root string)

RebaseConfigDir changes ConfigDir, if it is set, to be absolute by appending it to the given root.

Doesn't touch "-", which indicates "stdout".

func (*Meta) WasTouched Uses

func (m *Meta) WasTouched(field string) bool

WasTouched returns true if the field (given by its Starlark snake_case name) was explicitly set via CLI flags or via lucicfg.config(...) in Starlark.

Panics if the field is unrecognized.

type State Uses

type State struct {
    Inputs  Inputs    // all inputs, exactly as passed to Generate.
    Configs ConfigSet // all generated config files, populated at the end
    Meta    Meta      // lucicfg parameters, settable through Starlark
    // contains filtered or unexported fields

State is mutated throughout execution of the script and at the end contains the final execution result.

It is available in the implementation of native functions exposed to the Starlark side. Starlark code operates with the state exclusively through these functions.

All Starlark code is executed sequentially in a single goroutine, thus the state is not protected by any mutexes.

func Generate Uses

func Generate(ctx context.Context, in Inputs) (*State, error)

Generate interprets the high-level config.

Returns a multi-error with all captured errors. Some of them may implement BacktracableError interface.

type ValidationMessage Uses

type ValidationMessage = config.ComponentsConfigEndpointValidationMessage

Alias some ridiculously long type names that we round-trip in the public API.

type ValidationRequest Uses

type ValidationRequest = config.LuciConfigValidateConfigRequestMessage

Alias some ridiculously long type names that we round-trip in the public API.

type ValidationResult Uses

type ValidationResult struct {
    Failed   bool                 `json:"failed"`   // true if the config is bad
    Messages []*ValidationMessage `json:"messages"` // errors, warning, infos, etc.

ValidationResult is what we get after validating a config set.

func (*ValidationResult) Log Uses

func (vr *ValidationResult) Log(ctx context.Context)

Log all messages in the result to the logger at an appropriate logging level.

func (*ValidationResult) OverallError Uses

func (vr *ValidationResult) OverallError(failOnWarnings bool) error

OverallError is nil if the validation succeeded or non-nil if failed.

Beware: mutates Failed field accordingly.


cliPackage cli contains command line interface for lucicfg tool.
cli/basePackage base contains code shared by other CLI subpackages.
cli/cmds/generatePackage generate implements 'generate' subcommand.
cli/cmds/validatePackage validate implements 'validate' subcommand.
docgo:generate go run ../cmd/docgen/main.go -templates templates -out .
docgenPackage docgen generates documentation from Starlark code.
docgen/astPackage ast defines AST relevant for the documentation generation.
docgen/docstringPackage docstring parses docstrings into more structured representation.
docgen/symbolsPackage symbols defines a data model representing Starlark symbols.
graphPackage graph implements a DAG used internally to represent config objects.
starlarkPackage starlark is generated by
testprotoPackage testproto contains proto messages used exclusively in unit tests.
varsPackage vars implement lucicfg.var() support.

Package lucicfg imports 45 packages (graph) and is imported by 5 packages. Updated 2019-02-22. Refresh now. Tools for package owners.