evergreen

package module
v0.0.0-...-29e6843 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 20, 2018 License: Apache-2.0 Imports: 29 Imported by: 0

README

Evergreen

Evergreen is a distributed continuous integration system built by MongoDB. It dynamically allocates hosts (via AWS, GCP, etc) to run tasks in parallel across many machines at once to decrease the total amount of time needed to complete a test workload.

Using Evergreen, we've significantly enhanced the productivity of our engineers.

Features

Elastic Host Allocation

Use only the computing resources you need.

Clean UI

Easily navigate the state of your tests, logs, and commit history.

Multiplatform Support

Run jobs on Linux (including PowerPC and ZSeries), Windows, OSX, Solaris, and BSD.

Spawn Hosts

Spin up a copy of any machine in your test infrastructure for debugging.

Patch Builds

See test results for your code changes before committing.

Stepback on Failure

Automatically run past commits to pinpoint the origin of a test failure

See the documentation for a full feature list!

Usage

Evergreen requires the configuration of a main server along with a cloud provider or static servers. Please refer to our tutorial for full installation instructions.

System Requirements

The Evergreen Agent and Command Line Tool are supported on Linux, OSX, Windows, and Solaris operating systems. However, the Evergreen API Server, UI Server, and Runner program are currently only supported and tested on Linux and OSX.

Go Requirements

Vendoring Dependencies

Our dependencies live in the vendor subdirectory of the repo, and are managed using glide. To add a new dependency, add the package to glide.yaml and the package name and revision to glide.lock, and run glide install -s. To refresh the entire vendor tree, run make vendor-sync

Building the Binaries

Setup:

  • ensure that your GOPATH environment variable is set.
  • check out a copy of the repo into your gopath. You can use: go get github.com/evergreen-ci/evergreen. If you have an existing checkout of the evergreen repository that is not in $GOPATH/src/github.com/evergreen-ci/ move or create a symlink.
  • run make vendor to set up the vendoring environment.

Possible Targets:

  • run make build to compile all server binaries for your local system.
  • run make agent cli to compile the agent and cli for your local platform.
  • run make agents clis to compile and cross compile all agent and command line interface binaries.
  • run make dist to compile all server, commandline, and agent binaries and create a dist tarball with all artifacts.

Terminology

  • distro: A platform type (e.g. Windows or OSX) plus large-scale architectural details. One of the targets that we produce executables for.
  • host: One of the machines used to run tasks on; typically an instance of a distro that does the actual work (see task). Equivalently, any machine in our system other than MOTU.
  • revision: A revision SHA; e.g. ad8a1d4678122bada9d4479b114cf68d39c27724.
  • project: A branch in a given repository. e.g. master in the mongodb/mongo repository
  • version: (project + revision). A version of a project.
  • buildvariant: build specific information.
  • build: (version + buildvariant) = (project + revision + buildvariant).
  • task: “compile”, “test”, or “push”. The kinds of things that we want to do on a host.

Running Evergreen Processes

A single configuration file drives most of how Evergreen works. Each Evergreen process must be supplied this settings file to run properly. For development, tweak the sample configuration file here.

All Evergreen programs accept a configuration file with the -conf flag.

How It Works

  • A MongoDB server must be already running on the port specified in the configuration file.
  • Both the API and UI server processes are started manually and listen for connections.
  • All other processes are run by the runner process.
  • For detailed instructions, please see the wiki.

Documentation

Index

Constants

View Source
const (
	Announcement BannerTheme = "announcement"
	Information              = "information"
	Warning                  = "warning"
	Important                = "important"
)
View Source
const (
	User            = "mci"
	GithubPatchUser = "github_pull_request"

	HostRunning         = "running"
	HostTerminated      = "terminated"
	HostUninitialized   = "initializing"
	HostStarting        = "starting"
	HostProvisioning    = "provisioning"
	HostProvisionFailed = "provision failed"
	HostQuarantined     = "quarantined"
	HostDecommissioned  = "decommissioned"

	HostStatusSuccess = "success"
	HostStatusFailed  = "failed"

	// Task Statuses used in the database models
	TaskStarted      = "started"
	TaskUnstarted    = "unstarted"
	TaskUndispatched = "undispatched"
	TaskDispatched   = "dispatched"
	TaskFailed       = "failed"
	TaskSucceeded    = "success"
	TaskInactive     = "inactive"
	TaskSystemFailed = "system-failed"
	TaskTestTimedOut = "test-timed-out"

	// Task Command Types
	CommandTypeTest   = "test"
	CommandTypeSystem = "system"
	CommandTypeSetup  = "setup"

	// Task Statuses used only in TaskEndDetails
	// TaskFailed and TaskSucceeded are also used here
	TaskSetupFailed      = "setup-failed"
	TaskTimedOut         = "task-timed-out"
	TaskSystemUnresponse = "system-unresponsive"
	TaskSystemTimedOut   = "system-timed-out"

	// TaskConflict is used only in communication with the Agent
	TaskConflict = "task-conflict"

	TestFailedStatus         = "fail"
	TestSilentlyFailedStatus = "silentfail"
	TestSkippedStatus        = "skip"
	TestSucceededStatus      = "pass"

	BuildStarted   = "started"
	BuildCreated   = "created"
	BuildFailed    = "failed"
	BuildSucceeded = "success"

	VersionStarted   = "started"
	VersionCreated   = "created"
	VersionFailed    = "failed"
	VersionSucceeded = "success"

	PatchCreated   = "created"
	PatchStarted   = "started"
	PatchSucceeded = "succeeded"
	PatchFailed    = "failed"

	PushLogPushing = "pushing"
	PushLogSuccess = "success"

	HostTypeStatic = "static"

	CompileStage = "compile"
	PushStage    = "push"

	// maximum task (zero based) execution number
	MaxTaskExecution = 3

	// maximum task priority
	MaxTaskPriority = 100

	// LogMessage struct versions
	LogmessageFormatTimestamp = 1
	LogmessageCurrentVersion  = LogmessageFormatTimestamp

	EvergreenHome = "EVGHOME"
	MongodbUrl    = "MONGO_URL"

	// Special logging output targets
	LocalLoggingOverride          = "LOCAL"
	StandardOutputLoggingOverride = "STDOUT"

	DefaultTaskActivator   = ""
	StepbackTaskActivator  = "stepback"
	APIServerTaskActivator = "apiserver"

	RestRoutePrefix = "rest"
	APIRoutePrefix  = "api"

	AgentAPIVersion = 2

	DegradedLoggingPercent = 10

	SetupScriptName    = "setup.sh"
	TeardownScriptName = "teardown.sh"

	RoutePaginatorNextPageHeaderKey = "Link"
)
View Source
const (
	UIPackage      = "EVERGREEN_UI"
	RESTV2Package  = "EVERGREEN_REST_V2"
	MonitorPackage = "EVERGREEN_MONITOR"
)

evergreen package names

View Source
const (
	AuthTokenCookie     = "mci-token"
	TaskSecretHeader    = "Task-Secret"
	HostHeader          = "Host-Id"
	HostSecretHeader    = "Host-Secret"
	ContentTypeHeader   = "Content-Type"
	ContentTypeValue    = "application/json"
	ContentLengthHeader = "Content-Length"
	APIUserHeader       = "Api-User"
	APIKeyHeader        = "Api-Key"
)
View Source
const (
	ProviderNameEc2Auto     = "ec2-auto"
	ProviderNameEc2OnDemand = "ec2-ondemand"
	ProviderNameEc2Spot     = "ec2-spot"
	ProviderNameDocker      = "docker"
	ProviderNameDockerMock  = "docker-mock"
	ProviderNameGce         = "gce"
	ProviderNameStatic      = "static"
	ProviderNameOpenstack   = "openstack"
	ProviderNameVsphere     = "vsphere"
	ProviderNameMock        = "mock"

	// TODO: This can be removed when no more hosts with provider ec2 are running.
	ProviderNameEc2Legacy = "ec2"
)

cloud provider related constants

View Source
const (
	DefaultServiceConfigurationFileName = "/etc/mci_settings.yml"
	DefaultDatabaseUrl                  = "localhost:27017"
	DefaultDatabaseName                 = "mci"

	// database and config directory, set to the testing version by default for safety
	NotificationsFile = "mci-notifications.yml"
	ClientDirectory   = "clients"

	// version requester types
	PatchVersionRequester       = "patch_request"
	GithubPRRequester           = "github_pull_request"
	RepotrackerVersionRequester = "gitter_request"
)
View Source
const (
	GenerateTasksCommandName = "generate.tasks"
	CreateHostCommandName    = "host.create"
)
View Source
const (
	SenderGithubStatus = SenderKey(iota)
	SenderEvergreenWebhook
	SenderSlack
	SenderJIRAIssue
	SenderJIRAComment
	SenderEmail
)
View Source
const NameTimeFormat = "20060102150405"

NameTimeFormat is the format in which to log times like instance start time.

Variables

View Source
var (
	// Should be specified with -ldflags at build time
	BuildRevision = ""

	// Commandline Version String; used to control auto-updating.
	ClientVersion = "2018-06-05"
)
View Source
var (
	ConfigCollection = "admin"
	ConfigDocID      = "global"
)
View Source
var (
	PatchRequesters = []string{
		PatchVersionRequester,
		GithubPRRequester,
	}

	// UphostStatus is a list of all host statuses that are considered "up."
	// This is used for query building.
	UphostStatus = []string{
		HostRunning,
		HostUninitialized,
		HostStarting,
		HostProvisioning,
		HostProvisionFailed,
	}

	// Hosts in "initializing" status aren't actually running yet:
	// they're just intents, so this list omits that value.
	ActiveStatus = []string{
		HostRunning,
		HostStarting,
		HostProvisioning,
		HostProvisionFailed,
	}

	// constant arrays for db update logic
	AbortableStatuses = []string{TaskStarted, TaskDispatched}
	CompletedStatuses = []string{TaskSucceeded, TaskFailed}

	ValidCommandTypes = []string{CommandTypeSetup, CommandTypeSystem, CommandTypeTest}
)
View Source
var (

	// ContainerPool keys
	ContainerPoolIdKey = bsonutil.MustHaveTag(ContainerPool{}, "Id")
)

nolint: deadcode, megacheck

View Source
var (
	// Providers where hosts can be created and terminated automatically.
	ProviderSpawnable = []string{
		ProviderNameDocker,
		ProviderNameEc2Legacy,
		ProviderNameEc2OnDemand,
		ProviderNameEc2Spot,
		ProviderNameEc2Auto,
		ProviderNameGce,
		ProviderNameOpenstack,
		ProviderNameVsphere,
	}
)

Functions

func CreateSession

func CreateSession(settings DBSettings) *db.SessionFactory

func FindEvergreenHome

func FindEvergreenHome() string

FindEvergreenHome finds the directory of the EVGHOME environment variable.

func IsPatchRequester

func IsPatchRequester(requester string) bool

func IsSystemActivator

func IsSystemActivator(caller string) bool

IsSystemActivator returns true when the task activator is Evergreen.

func ResetEnvironment

func ResetEnvironment()

func SetBanner

func SetBanner(bannerText string) error

SetBanner sets the text of the Evergreen site-wide banner. Setting a blank string here means that there is no banner

func SetBannerTheme

func SetBannerTheme(theme BannerTheme) error

SetBanner sets the text of the Evergreen site-wide banner. Setting a blank string here means that there is no banner

func SetServiceFlags

func SetServiceFlags(flags ServiceFlags) error

SetServiceFlags sets whether each of the runner/API server processes is enabled

func UpdateConfig

func UpdateConfig(config *Settings) error

UpdateConfig updates all evergreen settings documents in DB

Types

type APIConfig

type APIConfig struct {
	HttpListenAddr      string `bson:"http_listen_addr" json:"http_listen_addr" yaml:"httplistenaddr"`
	GithubWebhookSecret string `bson:"github_webhook_secret" json:"github_webhook_secret" yaml:"github_webhook_secret"`
}

APIConfig holds relevant log and listener settings for the API server.

func (*APIConfig) Get

func (c *APIConfig) Get() error

func (*APIConfig) SectionId

func (c *APIConfig) SectionId() string

func (*APIConfig) Set

func (c *APIConfig) Set() error

func (*APIConfig) ValidateAndDefault

func (c *APIConfig) ValidateAndDefault() error

type AWSConfig

type AWSConfig struct {
	Secret string `bson:"aws_secret" json:"aws_secret" yaml:"aws_secret"`
	Id     string `bson:"aws_id" json:"aws_id" yaml:"aws_id"`
}

AWSConfig stores auth info for Amazon Web Services.

type AlertsConfig

type AlertsConfig struct {
	SMTP SMTPConfig `bson:"smtp" json:"smtp" yaml:"smtp"`
}

func (*AlertsConfig) Get

func (c *AlertsConfig) Get() error

func (*AlertsConfig) SectionId

func (c *AlertsConfig) SectionId() string

func (*AlertsConfig) Set

func (c *AlertsConfig) Set() error

func (*AlertsConfig) ValidateAndDefault

func (c *AlertsConfig) ValidateAndDefault() error

type AmboyConfig

type AmboyConfig struct {
	Name           string `bson:"name" json:"name" yaml:"name"`
	DB             string `bson:"database" json:"database" yaml:"database"`
	PoolSizeLocal  int    `bson:"pool_size_local" json:"pool_size_local" yaml:"pool_size_local"`
	PoolSizeRemote int    `bson:"pool_size_remote" json:"pool_size_remote" yaml:"pool_size_remote"`
	LocalStorage   int    `bson:"local_storage_size" json:"local_storage_size" yaml:"local_storage_size"`
}

func (*AmboyConfig) Get

func (c *AmboyConfig) Get() error

func (*AmboyConfig) SectionId

func (c *AmboyConfig) SectionId() string

func (*AmboyConfig) Set

func (c *AmboyConfig) Set() error

func (*AmboyConfig) ValidateAndDefault

func (c *AmboyConfig) ValidateAndDefault() error

type AuthConfig

type AuthConfig struct {
	Crowd  *CrowdConfig      `bson:"crowd" json:"crowd" yaml:"crowd"`
	Naive  *NaiveAuthConfig  `bson:"naive" json:"naive" yaml:"naive"`
	Github *GithubAuthConfig `bson:"github" json:"github" yaml:"github"`
}

AuthConfig has a pointer to either a CrowConfig or a NaiveAuthConfig.

func (*AuthConfig) Get

func (c *AuthConfig) Get() error

func (*AuthConfig) SectionId

func (c *AuthConfig) SectionId() string

func (*AuthConfig) Set

func (c *AuthConfig) Set() error

func (*AuthConfig) ValidateAndDefault

func (c *AuthConfig) ValidateAndDefault() error

type AuthUser

type AuthUser struct {
	Username    string `bson:"username" json:"username" yaml:"username"`
	DisplayName string `bson:"display_name" json:"display_name" yaml:"display_name"`
	Password    string `bson:"password" json:"password" yaml:"password"`
	Email       string `bson:"email" json:"email" yaml:"email"`
}

AuthUser configures a user for our Naive authentication setup.

type BannerTheme

type BannerTheme string

supported banner themes in Evergreen

func IsValidBannerTheme

func IsValidBannerTheme(input string) (bool, BannerTheme)

type BuildBaronProject

type BuildBaronProject struct {
	TicketCreateProject  string   `mapstructure:"ticket_create_project" bson:"ticket_create_project"`
	TicketSearchProjects []string `mapstructure:"ticket_search_projects" bson:"ticket_search_projects"`

	// The BF Suggestion server as a source of suggestions is only enabled for projects where BFSuggestionServer isn't the empty string.
	BFSuggestionServer      string `mapstructure:"bf_suggestion_server" bson:"bf_suggestion_server"`
	BFSuggestionUsername    string `mapstructure:"bf_suggestion_username" bson:"bf_suggestion_username"`
	BFSuggestionPassword    string `mapstructure:"bf_suggestion_password" bson:"bf_suggestion_password"`
	BFSuggestionTimeoutSecs int    `mapstructure:"bf_suggestion_timeout_secs" bson:"bf_suggestion_timeout_secs"`
}

type ClientBinary

type ClientBinary struct {
	Arch string `yaml:"arch" json:"arch"`
	OS   string `yaml:"os" json:"os"`
	URL  string `yaml:"url" json:"url"`
}

type ClientConfig

type ClientConfig struct {
	ClientBinaries []ClientBinary `yaml:"client_binaries" json:"ClientBinaries"`
	LatestRevision string         `yaml:"latest_revision" json:"LatestRevision"`
}

type CloudProviders

type CloudProviders struct {
	AWS       AWSConfig       `bson:"aws" json:"aws" yaml:"aws"`
	Docker    DockerConfig    `bson:"docker" json:"docker" yaml:"docker"`
	GCE       GCEConfig       `bson:"gce" json:"gce" yaml:"gce"`
	OpenStack OpenStackConfig `bson:"openstack" json:"openstack" yaml:"openstack"`
	VSphere   VSphereConfig   `bson:"vsphere" json:"vsphere" yaml:"vsphere"`
}

CloudProviders stores configuration settings for the supported cloud host providers.

func (*CloudProviders) Get

func (c *CloudProviders) Get() error

func (*CloudProviders) SectionId

func (c *CloudProviders) SectionId() string

func (*CloudProviders) Set

func (c *CloudProviders) Set() error

func (*CloudProviders) ValidateAndDefault

func (c *CloudProviders) ValidateAndDefault() error

type ConfigSection

type ConfigSection interface {
	// SectionId() returns the ID of the section to be used in the database document and struct tag
	SectionId() string
	// Get() populates the section from the DB
	Get() error
	// Set() upserts the section document into the DB
	Set() error
	// ValidateAndDefault() validates input and sets defaults
	ValidateAndDefault() error
}

ConfigSection defines a sub-document in the evegreen config any config sections must also be added to registry.go

type ConfigSectionRegistry

type ConfigSectionRegistry struct {
	// contains filtered or unexported fields
}
var ConfigRegistry *ConfigSectionRegistry

func (*ConfigSectionRegistry) GetSection

func (r *ConfigSectionRegistry) GetSection(id string) ConfigSection

func (*ConfigSectionRegistry) GetSections

func (r *ConfigSectionRegistry) GetSections() map[string]ConfigSection

type ContainerPool

type ContainerPool struct {
	// Distro of parent host that hosts containers
	Distro string `bson:"distro" json:"distro" yaml:"distro"`
	// ID of container pool
	Id string `bson:"id" json:"id" yaml:"id"`
	// Maximum number of containers per parent host with this container pool
	MaxContainers int `bson:"max_containers" json:"max_containers" yaml:"max_containers"`
	// Port number to start at for SSH connections
	Port uint16 `bson:"port" json:"port" yaml:"port"`
}

ContainerPool holds settings for each container pool

type ContainerPoolsConfig

type ContainerPoolsConfig struct {
	Pools []ContainerPool `bson:"pools" json:"pools" yaml:"pools"`
}

func (*ContainerPoolsConfig) Get

func (c *ContainerPoolsConfig) Get() error

func (*ContainerPoolsConfig) GetContainerPool

func (c *ContainerPoolsConfig) GetContainerPool(id string) *ContainerPool

GetContainerPool retrieves the container pool with a given id from a ContainerPoolsConfig struct

func (*ContainerPoolsConfig) SectionId

func (c *ContainerPoolsConfig) SectionId() string

func (*ContainerPoolsConfig) Set

func (c *ContainerPoolsConfig) Set() error

func (*ContainerPoolsConfig) ValidateAndDefault

func (c *ContainerPoolsConfig) ValidateAndDefault() error

type CrowdConfig

type CrowdConfig struct {
	Username string `bson:"username" json:"username" yaml:"username"`
	Password string `bson:"password" json:"password" yaml:"password"`
	Urlroot  string `bson:"url_root" json:"url_root" yaml:"urlroot"`
}

CrowdConfig holds settings for interacting with Atlassian Crowd.

type DBSettings

type DBSettings struct {
	Url                  string       `yaml:"url"`
	SSL                  bool         `yaml:"ssl"`
	DB                   string       `yaml:"db"`
	WriteConcernSettings WriteConcern `yaml:"write_concern"`
}

type DockerConfig

type DockerConfig struct {
	APIVersion string `bson:"api_version" json:"api_version" yaml:"api_version"`
}

DockerConfig stores auth info for Docker.

type Environment

type Environment interface {
	// Configure initializes the object. Some implementations may
	// not allow the same instance to be configured more than
	// once.
	//
	// If Configure returns without an error, you should assume
	// that the queues have been started, there was no issue
	// establishing a connection to the database, and that the
	// local and remote queues have started.
	Configure(context.Context, string, *DBSettings) error

	// Returns the settings object. The settings object is not
	// necessarily safe for concurrent access.
	Settings() *Settings
	Session() db.Session

	// The Environment provides access to two queue's, a
	// local-process level queue that is not persisted between
	// runs, and a remote shared queue that all processes can use
	// to distribute work amongst the application tier.
	//
	// The LocalQueue is not durable, and results aren't available
	// between process restarts. The RemoteQueue is not
	// (generally) started by default.
	LocalQueue() amboy.Queue
	RemoteQueue() amboy.Queue

	// ClientConfig provides access to a list of the latest evergreen
	// clients, that this server can serve to users
	ClientConfig() *ClientConfig

	// GetSender provides a grip Sender configured with the environment's
	// settings. These Grip senders must be used with Composers that specify
	// all message details.
	GetSender(SenderKey) (send.Sender, error)

	// RegisterCloser adds a function object to an internal
	// tracker to be called by the Close method before process
	// termination. The ID is used in reporting, but must be
	// unique or a new closer could overwrite an existing closer
	// in some implementations.
	RegisterCloser(string, func(context.Context) error)
	// Close calls all registered closers in the environment.
	Close(context.Context) error
}

Environment provides application-level services (e.g. databases, configuration, queues.

func GetEnvironment

func GetEnvironment() Environment

GetEnvironment returns the global application level environment. This implementation is thread safe, but must be configured before use.

In general you should call this operation once per process execution and pass the Environment interface through your application like a context, although there are cases in legacy code (e.g. models) and in the implementation of amboy jobs where it is necessary to access the global environment. There is a mock implementation for use in testing.

type GCEConfig

type GCEConfig struct {
	ClientEmail  string `bson:"client_email" json:"client_email" yaml:"client_email"`
	PrivateKey   string `bson:"private_key" json:"private_key" yaml:"private_key"`
	PrivateKeyID string `bson:"private_key_id" json:"private_key_id" yaml:"private_key_id"`
	TokenURI     string `bson:"token_uri" json:"token_uri" yaml:"token_uri"`
}

GCEConfig stores auth info for Google Compute Engine. Can be retrieved from: https://developers.google.com/identity/protocols/application-default-credentials

type GithubAuthConfig

type GithubAuthConfig struct {
	ClientId     string   `bson:"client_id" json:"client_id" yaml:"client_id"`
	ClientSecret string   `bson:"client_secret" json:"client_secret" yaml:"client_secret"`
	Users        []string `bson:"users" json:"users" yaml:"users"`
	Organization string   `bson:"organization" json:"organization" yaml:"organization"`
}

GithubAuthConfig holds settings for interacting with Github Authentication including the ClientID, ClientSecret and CallbackUri which are given when registering the application Furthermore,

type HostInitConfig

type HostInitConfig struct {
	SSHTimeoutSeconds int64 `bson:"ssh_timeout_secs" json:"ssh_timeout_secs" yaml:"sshtimeoutseconds"`
}

HostInitConfig holds logging settings for the hostinit process.

func (*HostInitConfig) Get

func (c *HostInitConfig) Get() error

func (*HostInitConfig) SectionId

func (c *HostInitConfig) SectionId() string

func (*HostInitConfig) Set

func (c *HostInitConfig) Set() error

func (*HostInitConfig) ValidateAndDefault

func (c *HostInitConfig) ValidateAndDefault() error

type JIRACustomFieldsByProject

type JIRACustomFieldsByProject []JIRANotificationsProject

func (*JIRACustomFieldsByProject) FromMap

func (j *JIRACustomFieldsByProject) FromMap(m map[string]map[string]string)

func (JIRACustomFieldsByProject) ToMap

func (j JIRACustomFieldsByProject) ToMap() (map[string]map[string]string, error)

type JIRANotificationsConfig

type JIRANotificationsConfig struct {
	// CustomFields is a map[string]map[string]string. The key of the first
	// map is the JIRA project (ex: EVG), the key of the second map is
	// the custom field name, and the inner most value is the template
	// for the custom field
	CustomFields JIRACustomFieldsByProject `bson:"custom_fields"`
}

func (*JIRANotificationsConfig) Get

func (c *JIRANotificationsConfig) Get() error

func (*JIRANotificationsConfig) SectionId

func (c *JIRANotificationsConfig) SectionId() string

func (*JIRANotificationsConfig) Set

func (c *JIRANotificationsConfig) Set() error

func (*JIRANotificationsConfig) ValidateAndDefault

func (c *JIRANotificationsConfig) ValidateAndDefault() error

type JIRANotificationsCustomField

type JIRANotificationsCustomField struct {
	Field    string `bson:"field"`
	Template string `bson:"template"`
}

type JIRANotificationsProject

type JIRANotificationsProject struct {
	Project string                         `bson:"project"`
	Fields  []JIRANotificationsCustomField `bson:"fields"`
}

type JiraConfig

type JiraConfig struct {
	Host           string `yaml:"host" bson:"host" json:"host"`
	Username       string `yaml:"username" bson:"username" json:"username"`
	Password       string `yaml:"password" bson:"password" json:"password"`
	DefaultProject string `yaml:"default_project" bson:"default_project" json:"default_project"`
}

JiraConfig stores auth info for interacting with Atlassian Jira.

func (*JiraConfig) Get

func (c *JiraConfig) Get() error

func (JiraConfig) GetHostURL

func (c JiraConfig) GetHostURL() string

func (*JiraConfig) SectionId

func (c *JiraConfig) SectionId() string

func (*JiraConfig) Set

func (c *JiraConfig) Set() error

func (*JiraConfig) ValidateAndDefault

func (c *JiraConfig) ValidateAndDefault() error

type LogBuffering

type LogBuffering struct {
	DurationSeconds int `bson:"duration_seconds" json:"duration_seconds" yaml:"duration_seconds"`
	Count           int `bson:"count" json:"count" yaml:"count"`
}

type LoggerConfig

type LoggerConfig struct {
	Buffer         LogBuffering `bson:"buffer" json:"buffer" yaml:"buffer"`
	DefaultLevel   string       `bson:"default_level" json:"default_level" yaml:"default_level"`
	ThresholdLevel string       `bson:"threshold_level" json:"threshold_level" yaml:"threshold_level"`
}

func (*LoggerConfig) Get

func (c *LoggerConfig) Get() error

func (LoggerConfig) Info

func (c LoggerConfig) Info() send.LevelInfo

func (*LoggerConfig) SectionId

func (c *LoggerConfig) SectionId() string

func (*LoggerConfig) Set

func (c *LoggerConfig) Set() error

func (*LoggerConfig) ValidateAndDefault

func (c *LoggerConfig) ValidateAndDefault() error

type NaiveAuthConfig

type NaiveAuthConfig struct {
	Users []*AuthUser `bson:"users" json:"users" yaml:"users"`
}

NaiveAuthConfig contains a list of AuthUsers from the settings file.

type NotifyConfig

type NotifyConfig struct {
	BufferTargetPerInterval int        `bson:"buffer_target_per_interval" json:"buffer_target_per_interval" yaml:"buffer_target_per_interval"`
	BufferIntervalSeconds   int        `bson:"buffer_interval_seconds" json:"buffer_interval_seconds" yaml:"buffer_interval_seconds"`
	SMTP                    SMTPConfig `bson:"smtp" json:"smtp" yaml:"smtp"`
}

NotifyConfig hold logging and email settings for the notify package.

func (*NotifyConfig) Get

func (c *NotifyConfig) Get() error

func (*NotifyConfig) SectionId

func (c *NotifyConfig) SectionId() string

func (*NotifyConfig) Set

func (c *NotifyConfig) Set() error

func (*NotifyConfig) ValidateAndDefault

func (c *NotifyConfig) ValidateAndDefault() error

type OpenStackConfig

type OpenStackConfig struct {
	IdentityEndpoint string `bson:"identity_endpoint" json:"identity_endpoint" yaml:"identity_endpoint"`

	Username   string `bson:"username" json:"username" yaml:"username"`
	Password   string `bson:"password" json:"password" yaml:"password"`
	DomainName string `bson:"domain_name" json:"domain_name" yaml:"domain_name"`

	ProjectName string `bson:"project_name" json:"project_name" yaml:"project_name"`
	ProjectID   string `bson:"project_id" json:"project_id" yaml:"project_id"`

	Region string `bson:"region" json:"region" yaml:"region"`
}

OpenStackConfig stores auth info for Linaro using Identity V3. All fields required.

The config is NOT compatible with Identity V2.

type PluginConfig

type PluginConfig map[string]map[string]interface{}

PluginConfig holds plugin-specific settings, which are handled. manually by their respective plugins

type RepoTrackerConfig

type RepoTrackerConfig struct {
	NumNewRepoRevisionsToFetch int `bson:"revs_to_fetch" json:"revs_to_fetch" yaml:"numnewreporevisionstofetch"`
	MaxRepoRevisionsToSearch   int `bson:"max_revs_to_search" json:"max_revs_to_search" yaml:"maxreporevisionstosearch"`
	MaxConcurrentRequests      int `bson:"max_con_requests" json:"max_con_requests" yaml:"maxconcurrentrequests"`
}

RepoTrackerConfig holds settings for polling project repositories.

func (*RepoTrackerConfig) Get

func (c *RepoTrackerConfig) Get() error

func (*RepoTrackerConfig) SectionId

func (c *RepoTrackerConfig) SectionId() string

func (*RepoTrackerConfig) Set

func (c *RepoTrackerConfig) Set() error

func (*RepoTrackerConfig) ValidateAndDefault

func (c *RepoTrackerConfig) ValidateAndDefault() error

type SMTPConfig

type SMTPConfig struct {
	Server     string   `bson:"server" json:"server" yaml:"server"`
	Port       int      `bson:"port" json:"port" yaml:"port"`
	UseSSL     bool     `bson:"use_ssl" json:"use_ssl" yaml:"use_ssl"`
	Username   string   `bson:"username" json:"username" yaml:"username"`
	Password   string   `bson:"password" json:"password" yaml:"password"`
	From       string   `bson:"from" json:"from" yaml:"from"`
	AdminEmail []string `bson:"admin_email" json:"admin_email" yaml:"admin_email"`
}

SMTPConfig holds SMTP email settings.

type SchedulerConfig

type SchedulerConfig struct {
	TaskFinder       string  `bson:"task_finder" json:"task_finder" yaml:"task_finder"`
	HostAllocator    string  `bson:"host_allocator" json:"host_allocator" yaml:"host_allocator"`
	FreeHostFraction float64 `bson:"free_host_fraction" json:"free_host_fraction" yaml:"free_host_fraction"`
}

SchedulerConfig holds relevant settings for the scheduler process.

func (*SchedulerConfig) Get

func (c *SchedulerConfig) Get() error

func (*SchedulerConfig) SectionId

func (c *SchedulerConfig) SectionId() string

func (*SchedulerConfig) Set

func (c *SchedulerConfig) Set() error

func (*SchedulerConfig) ValidateAndDefault

func (c *SchedulerConfig) ValidateAndDefault() error

type SenderKey

type SenderKey int

type ServiceFlags

type ServiceFlags struct {
	TaskDispatchDisabled         bool `bson:"task_dispatch_disabled" json:"task_dispatch_disabled"`
	HostinitDisabled             bool `bson:"hostinit_disabled" json:"hostinit_disabled"`
	MonitorDisabled              bool `bson:"monitor_disabled" json:"monitor_disabled"`
	AlertsDisabled               bool `bson:"alerts_disabled" json:"alerts_disabled"`
	TaskrunnerDisabled           bool `bson:"taskrunner_disabled" json:"taskrunner_disabled"`
	RepotrackerDisabled          bool `bson:"repotracker_disabled" json:"repotracker_disabled"`
	SchedulerDisabled            bool `bson:"scheduler_disabled" json:"scheduler_disabled"`
	GithubPRTestingDisabled      bool `bson:"github_pr_testing_disabled" json:"github_pr_testing_disabled"`
	RepotrackerPushEventDisabled bool `bson:"repotracker_push_event_disabled" json:"repotracker_push_event_disabled"`
	CLIUpdatesDisabled           bool `bson:"cli_updates_disabled" json:"cli_updates_disabled"`
	BackgroundStatsDisabled      bool `bson:"background_stats_disabled" json:"background_stats_disabled"`
	TaskLoggingDisabled          bool `bson:"task_logging_disabled" json:"task_logging_disabled"`

	// Notification Flags
	EventProcessingDisabled      bool `bson:"event_processing_disabled" json:"event_processing_disabled"`
	JIRANotificationsDisabled    bool `bson:"jira_notifications_disabled" json:"jira_notifications_disabled"`
	SlackNotificationsDisabled   bool `bson:"slack_notifications_disabled" json:"slack_notifications_disabled"`
	EmailNotificationsDisabled   bool `bson:"email_notifications_disabled" json:"email_notifications_disabled"`
	WebhookNotificationsDisabled bool `bson:"webhook_notifications_disabled" json:"webhook_notifications_disabled"`
	GithubStatusAPIDisabled      bool `bson:"github_status_api_disabled" json:"github_status_api_disabled"`
}

ServiceFlags holds the state of each of the runner/API processes

func GetServiceFlags

func GetServiceFlags() (*ServiceFlags, error)

func (*ServiceFlags) Get

func (c *ServiceFlags) Get() error

func (*ServiceFlags) SectionId

func (c *ServiceFlags) SectionId() string

func (*ServiceFlags) Set

func (c *ServiceFlags) Set() error

func (*ServiceFlags) ValidateAndDefault

func (c *ServiceFlags) ValidateAndDefault() error

type Settings

type Settings struct {
	Id                 string                    `bson:"_id" json:"id"`
	Alerts             AlertsConfig              `yaml:"alerts" bson:"alerts" json:"alerts" id:"alerts"`
	Amboy              AmboyConfig               `yaml:"amboy" bson:"amboy" json:"amboy" id:"amboy"`
	Api                APIConfig                 `yaml:"api" bson:"api" json:"api" id:"api"`
	ApiUrl             string                    `yaml:"api_url" bson:"api_url" json:"api_url"`
	AuthConfig         AuthConfig                `yaml:"auth" bson:"auth" json:"auth" id:"auth"`
	Banner             string                    `bson:"banner" json:"banner"`
	BannerTheme        BannerTheme               `bson:"banner_theme" json:"banner_theme"`
	ClientBinariesDir  string                    `yaml:"client_binaries_dir" bson:"client_binaries_dir" json:"client_binaries_dir"`
	ConfigDir          string                    `yaml:"configdir" bson:"configdir" json:"configdir"`
	ContainerPools     ContainerPoolsConfig      `yaml:"container_pools" bson:"container_pools" json:"container_pools" id:"container_pools"`
	Credentials        map[string]string         `yaml:"credentials" bson:"credentials" json:"credentials"`
	CredentialsNew     util.KeyValuePairSlice    `yaml:"credentials_new" bson:"credentials_new" json:"credentials_new"`
	Database           DBSettings                `yaml:"database"`
	Expansions         map[string]string         `yaml:"expansions" bson:"expansions" json:"expansions"`
	ExpansionsNew      util.KeyValuePairSlice    `yaml:"expansions_new" bson:"expansions_new" json:"expansions_new"`
	GithubPRCreatorOrg string                    `yaml:"github_pr_creator_org" bson:"github_pr_creator_org" json:"github_pr_creator_org"`
	HostInit           HostInitConfig            `yaml:"hostinit" bson:"hostinit" json:"hostinit" id:"hostinit"`
	Jira               JiraConfig                `yaml:"jira" bson:"jira" json:"jira" id:"jira"`
	JIRANotifications  JIRANotificationsConfig   `yaml:"jira_notifications" json:"jira_notifications" bson:"jira_notifications" id:"jira_notifications"`
	Keys               map[string]string         `yaml:"keys" bson:"keys" json:"keys"`
	KeysNew            util.KeyValuePairSlice    `yaml:"keys_new" bson:"keys_new" json:"keys_new"`
	LoggerConfig       LoggerConfig              `yaml:"logger_config" bson:"logger_config" json:"logger_config" id:"logger_config"`
	LogPath            string                    `yaml:"log_path" bson:"log_path" json:"log_path"`
	Notify             NotifyConfig              `yaml:"notify" bson:"notify" json:"notify" id:"notify"`
	Plugins            PluginConfig              `yaml:"plugins" bson:"plugins" json:"plugins"`
	PluginsNew         util.KeyValuePairSlice    `yaml:"plugins_new" bson:"plugins_new" json:"plugins_new"`
	PprofPort          string                    `yaml:"pprof_port" bson:"pprof_port" json:"pprof_port"`
	Providers          CloudProviders            `yaml:"providers" bson:"providers" json:"providers" id:"providers"`
	RepoTracker        RepoTrackerConfig         `yaml:"repotracker" bson:"repotracker" json:"repotracker" id:"repotracker"`
	Scheduler          SchedulerConfig           `yaml:"scheduler" bson:"scheduler" json:"scheduler" id:"scheduler"`
	ServiceFlags       ServiceFlags              `bson:"service_flags" json:"service_flags" id:"service_flags"`
	Slack              SlackConfig               `yaml:"slack" bson:"slack" json:"slack" id:"slack"`
	Splunk             send.SplunkConnectionInfo `yaml:"splunk" bson:"splunk" json:"splunk"`
	SuperUsers         []string                  `yaml:"superusers" bson:"superusers" json:"superusers"`
	Ui                 UIConfig                  `yaml:"ui" bson:"ui" json:"ui" id:"ui"`
}

Settings contains all configuration settings for running Evergreen.

func GetConfig

func GetConfig() (*Settings, error)

GetConfig retrieves the Evergreen config document. If no document is present in the DB, it will return the defaults

func NewSettings

func NewSettings(filename string) (*Settings, error)

NewSettings builds an in-memory representation of the given settings file.

func (*Settings) Get

func (c *Settings) Get() error

func (*Settings) GetGithubOauthToken

func (s *Settings) GetGithubOauthToken() (string, error)

func (*Settings) GetSender

func (s *Settings) GetSender(env Environment) (send.Sender, error)

func (*Settings) SectionId

func (c *Settings) SectionId() string

func (*Settings) SessionFactory

func (settings *Settings) SessionFactory() *db.SessionFactory

SessionFactory creates a usable SessionFactory from the Evergreen settings.

func (*Settings) Set

func (c *Settings) Set() error

func (*Settings) Validate

func (settings *Settings) Validate() error

Validate checks the settings and returns nil if the config is valid, or an error with a message explaining why otherwise.

func (*Settings) ValidateAndDefault

func (c *Settings) ValidateAndDefault() error

type SlackConfig

type SlackConfig struct {
	Options *send.SlackOptions `bson:"options" json:"options" yaml:"options"`
	Token   string             `bson:"token" json:"token" yaml:"token"`
	Level   string             `bson:"level" json:"level" yaml:"level"`
}

func (*SlackConfig) Get

func (c *SlackConfig) Get() error

func (*SlackConfig) SectionId

func (c *SlackConfig) SectionId() string

func (*SlackConfig) Set

func (c *SlackConfig) Set() error

func (*SlackConfig) ValidateAndDefault

func (c *SlackConfig) ValidateAndDefault() error

type UIConfig

type UIConfig struct {
	Url            string `bson:"url" json:"url" yaml:"url"`
	HelpUrl        string `bson:"help_url" json:"help_url" yaml:"helpurl"`
	HttpListenAddr string `bson:"http_listen_addr" json:"http_listen_addr" yaml:"httplistenaddr"`
	// Secret to encrypt session storage
	Secret string `bson:"secret" json:"secret" yaml:"secret"`
	// Default project to assume when none specified, e.g. when using
	// the /waterfall route use this project, while /waterfall/other-project
	// then use `other-project`
	DefaultProject string `bson:"default_project" json:"default_project" yaml:"defaultproject"`
	// Cache results of template compilation, so you don't have to re-read files
	// on every request. Note that if this is true, changes to HTML templates
	// won't take effect until server restart.
	CacheTemplates bool `bson:"cache_templates" json:"cache_templates" yaml:"cachetemplates"`
	// SecureCookies sets the "secure" flag on user tokens. Evergreen
	// does not yet natively support SSL UI connections, but this option
	// is available, for example, for deployments behind HTTPS load balancers.
	SecureCookies bool `bson:"secure_cookies" json:"secure_cookies" yaml:"securecookies"`
	// CsrfKey is a 32-byte key used to generate tokens that validate UI requests
	CsrfKey string `bson:"csrf_key" json:"csrf_key" yaml:"csrfkey"`
}

UIConfig holds relevant settings for the UI server.

func (*UIConfig) Get

func (c *UIConfig) Get() error

func (*UIConfig) SectionId

func (c *UIConfig) SectionId() string

func (*UIConfig) Set

func (c *UIConfig) Set() error

func (*UIConfig) ValidateAndDefault

func (c *UIConfig) ValidateAndDefault() error

type VSphereConfig

type VSphereConfig struct {
	Host     string `bson:"host" json:"host" yaml:"host"`
	Username string `bson:"username" json:"username" yaml:"username"`
	Password string `bson:"password" json:"password" yaml:"password"`
}

VSphereConfig stores auth info for VMware vSphere. The config fields refer to your vCenter server, a centralized management tool for the vSphere suite.

type WriteConcern

type WriteConcern struct {
	W        int    `yaml:"w"`
	WMode    string `yaml:"wmode"`
	WTimeout int    `yaml:"wtimeout"`
	FSync    bool   `yaml:"fsync"`
	J        bool   `yaml:"j"`
}

Directories

Path Synopsis
The REST API V2 has a series of central types that are useful to understand when adding new endpoints or increasing its functionality.
The REST API V2 has a series of central types that are useful to understand when adding new endpoints or increasing its functionality.
data
Adding to the Connector The Connector is a very large interface that defines how to access the main state of the database and data central to Evergreen's function.
Adding to the Connector The Connector is a very large interface that defines how to access the main state of the database and data central to Evergreen's function.
model
Adding Models Each model is kept in the model package of the REST v2 API in its own file.
Adding Models Each model is kept in the model package of the REST v2 API in its own file.
route
Adding a Route Adding a new route to the REST v2 API requires creation of a few structs and implementation of a few new methods.
Adding a Route Adding a new route to the REST v2 API requires creation of a few structs and implementation of a few new methods.
Archive Provides a single "MakeTarball" function to create tar (tar.gz) archives.
Archive Provides a single "MakeTarball" function to create tar (tar.gz) archives.
Local Exec The "local exec" implementation of the command interface is roughly equvalent to the LocalCommand interface; however, it does not enforce the use of any shell and
Local Exec The "local exec" implementation of the command interface is roughly equvalent to the LocalCommand interface; however, it does not enforce the use of any shell and
Package units contains amboy.Job definiteness for Evergreen tasks.
Package units contains amboy.Job definiteness for Evergreen tasks.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL