package config

import "go.chromium.org/luci/config"

Package config contains a client to access LUCI configuration service.

WARNING: Large chunks of this package tree are deprecated and will be removed at some point. In particular 'appengine/*' and 'server/cfgclient/*' will not be made compatible with GAE v2 and will be deleted.

The existing API (exposed via server/cfgclient package) is "pull-based": users "pull" configs via cfgclient.Get(...) calls whenever they want. In particular they may do it from the data plane (which generally handles a lot of QPS). This requires cfgclient.Get(...) to be very fast and very reliable. "Very fast and very reliable" doesn't apply to LUCI Config service itself at all (nor should it). As a result cfgclient.Get(...) implementation had to incorporate some pretty complicated multi-layered cache (that no one fully understands anymore).

But turns out the majority of users (but not all) use cfgclient.Get(...) from their control plane, i.e. they run some cron job that periodically pulls most recent configs, processes them in some way, and stores them somewhere where the data plane can reliably reach them. This also gives them flexibility to react to diffs in the configs.

For such users the complexity of cfgclient.Get(...) implementation is completely unnecessary. It just adds more work to them to configure the caching layer.

We are migrating to GAE v2 and GKE, and we no longer assume Cloud Datastore and Memcache are ubiquitously present. The existing complicated cfgclient code will not work in such environments.

Instead of refactoring it we are planning to remove it completely and change the public package API to be "push-based", i.e. we want to *force* users to register some callback which receives configs whenever they change. Then users may store them whereever they want (like most of them already do in the cron jobs).

This approach would result in a vastly simpler config client code and will guarantee LUCI Config service is not a dependency of the data plane (so it's not a big deal if it goes down).

Index ¶

• Variables
• func ValidateProjectName(p string) error
• type Config
• type FormatSpec
• func (fs *FormatSpec) Unformatted() bool
• type Interface
• type Meta
• type Project
• type RepoType
• type Set
• func ProjectSet(project string) Set
• func RefSet(project string, ref string) Set
• func ServiceSet(service string) Set
• func (cs Set) Project() string
• func (cs Set) ProjectAndRef() (project, ref string)
• func (cs Set) Ref() string
• func (cs Set) Service() string
• func (cs Set) Split() (domain, target, ref string)

Package Files ¶

doc.go interface.go project_name.go set.go

Variables ¶

var ErrNoConfig = errors.New("no such config")

ErrNoConfig is returned if requested config does not exist.

func ValidateProjectName ¶ Uses

func ValidateProjectName(p string) error

ValidateProjectName returns an error if the supplied string is not a valid project name.

A valid project name may only include:

- Lowercase letters [a-z]
- Numbers [0-9]
- Hyphen (-)
- Underscore (_)

It also must begin with a letter.

See: https://github.com/luci/luci-py/blob/8e594074929871a9761d27e814541bc0d7d84744/appengine/components/components/config/common.py#L41

type Config ¶ Uses

type Config struct {
    Meta

    // FormatSpec, if non-empty, qualifies the format of the Content.
    //
    // It is set only if config was obtained through cfgclient guts that does
    // reformatting (e.g. formats text protos into binary protos).
    //
    // DEPRECATED.
    FormatSpec FormatSpec `json:"formatSpec,omitempty"`

    // Error is not nil if there where troubles fetching this config. Used only
    // by functions that operate with multiple configs at once, such as
    // GetProjectConfigs and GetRefConfigs.
    Error error `json:"error,omitempty"`

    // Content is the actual body of the config file.
    Content string `json:"content,omitempty"`
}

Config is a configuration file along with its metadata.

type FormatSpec ¶ Uses

type FormatSpec struct {
    // Formatter is the name of the formatter that produces the content data.
    //
    // An empty string means the original config service format.
    Formatter string

    // Data is additional format data describing the type. It may be empty.
    Data string
}

FormatSpec is a specification for formatted data.

DEPRECATED.

func (*FormatSpec) Unformatted ¶ Uses

func (fs *FormatSpec) Unformatted() bool

Unformatted returns true if fs does not specify a format.

type Interface ¶ Uses

type Interface interface {
    // GetConfig returns a config at a path in a config set or ErrNoConfig
    // if missing. If metaOnly is true, returned Config struct has only Meta set
    // (and the call is faster).
    GetConfig(ctx context.Context, configSet Set, path string, metaOnly bool) (*Config, error)

    // GetConfigByHash returns the contents of a config, as identified by its
    // content hash, or ErrNoConfig if missing.
    GetConfigByHash(ctx context.Context, contentHash string) (string, error)

    // GetConfigSetLocation returns the URL location of a config set.
    GetConfigSetLocation(ctx context.Context, configSet Set) (*url.URL, error)

    // GetProjectConfigs returns all the configs at the given path in all
    // projects that have such config. If metaOnly is true, returned Config
    // structs have only Meta set (and the call is faster).
    GetProjectConfigs(ctx context.Context, path string, metaOnly bool) ([]Config, error)

    // GetProjects returns all the registered projects in the configuration
    // service.
    GetProjects(ctx context.Context) ([]Project, error)

    // GetRefConfigs returns the config at the given path in all refs of all
    // projects that have such config. If metaOnly is true, returned Config
    // structs have only Meta set (and the call is faster).
    GetRefConfigs(ctx context.Context, path string, metaOnly bool) ([]Config, error)

    // GetRefs returns the list of refs for a project.
    GetRefs(ctx context.Context, projectID string) ([]string, error)

    // ListFiles returns the list of files for a config set.
    ListFiles(ctx context.Context, configSet Set) ([]string, error)
}

Interface represents low-level pull-based LUCI Config API.

This is roughly a wrapper over LUCI Config RPC interface, and all methods are generally slow and depend on available of LUCI Config service. They *must not* be used in a hot path of requests.

Transient errors are tagged with transient.Tag.

type Meta ¶ Uses

type Meta struct {
    // ConfigSet is the config set name (e.g. "projects/<id>") this config
    // belongs to.
    ConfigSet Set `json:"configSet,omitempty"`

    // Path is the filename relative to the root of the config set,
    // without leading slash, e.g. "luci-scheduler.cfg".
    Path string `json:"path,omitempty"`

    // ContentHash can be used to quickly check that content didn't change.
    ContentHash string `json:"contentHash,omitempty"`

    // Revision is git SHA1 of a repository the config was fetched from.
    Revision string `json:"revision,omitempty"`

    // ViewURL is the URL surfaced for viewing the config.
    ViewURL string `json:"view_url,omitempty"`
}

Meta is metadata about a single configuration file.

type Project ¶ Uses

type Project struct {
    // ID is unique project identifier.
    ID  string

    // Name is a short friendly display name of the project.
    Name string

    // RepoType specifies in what kind of storage projects configs are stored.
    RepoType RepoType

    // RepoUrl is the location of this project code repository. May be nil if
    // unknown or cannot be parsed.
    RepoURL *url.URL
}

Project is a project registered in the luci-config service.

type RepoType ¶ Uses

type RepoType string

RepoType is the type of the repo the Project is stored in.

const (
    // GitilesRepo means a repo is backed by the Gitiles service.
    GitilesRepo RepoType = "GITILES"

    // UnknownRepo means a repo is backed by an unknown service.
    // It may be an invalid repo.
    UnknownRepo RepoType = "UNKNOWN"
)

type Set ¶ Uses

type Set string

Set is a name of a configuration set: a bunch of config files versioned and stored as a single unit in a same repository.

A config set name consists of a domain and a series of path components: domain/target[/ref].

- Service config sets are config sets in the "services" domain, with the
  service name as the target.
- Project config sets are config sets in the "projects" domain. The target
  is the project name.

func ProjectSet ¶ Uses

func ProjectSet(project string) Set

ProjectSet returns the config set for the specified project.

func RefSet ¶ Uses

func RefSet(project string, ref string) Set

RefSet returns the config set for the specified project and ref. If ref is empty, this will equal the ProjectSet value.

func ServiceSet ¶ Uses

func ServiceSet(service string) Set

ServiceSet returns the name of a config set for the specified service.

func (Set) Project ¶ Uses

func (cs Set) Project() string

Project returns a project name for a project-rooted config set or empty string for all other sets.

Use ProjectAndRef if you need to get both the project name and the ref.

func (Set) ProjectAndRef ¶ Uses

func (cs Set) ProjectAndRef() (project, ref string)

ProjectAndRef splits a project-rooted config set (projects/<name>[/...]) into its project name and ref components.

For example, "projects/foo/bar/baz" is a config set that belongs to project "foo". It will be parsed into ("foo", "bar/baz").

If configSet is not a project config set, empty strings will be returned.

func (Set) Ref ¶ Uses

func (cs Set) Ref() string

Ref returns a ref component of a project-rooted config set or empty string for all other sets.

Use ProjectAndRef if you need to get both the project name and the ref.

func (Set) Service ¶ Uses

func (cs Set) Service() string

Service returns a service name for a service-rooted config set or empty string for all other sets.

func (Set) Split ¶ Uses

func (cs Set) Split() (domain, target, ref string)

Split splits a Set into its domain, target, and ref components.

Directories ¶