build: Index | Files

package pool

import ""


Package Files

ec2.go gce.go kube.go ledger.go log.go pool.go reverse.go

func HandleReverse Uses

func HandleReverse(w http.ResponseWriter, r *http.Request)

HandleReverse handles reverse buildlet connections.

func HasScope Uses

func HasScope(want string) bool

HasScope returns true if the GCE metadata contains the default scopes.

func InitGCE Uses

func InitGCE(sc *secret.Client, vmDeleteTimeout time.Duration, tFiles map[string]string, basePin *atomic.Value, fn IsRemoteBuildletFunc, buildEnvName, mode string) error

InitGCE initializes the GCE buildlet pool.

func InitKube Uses

func InitKube(monitorGitMirror MonitorGitMirrorFunc) error

InitGCE must be called before initKube

func KubeErr Uses

func KubeErr() error

KubeErr retrieves the kube error value.

func KubeGoClient Uses

func KubeGoClient() *kubernetes.Client

KubeGoClient retrieves a kube client for the go cluster.

func KubePool Uses

func KubePool() *kubeBuildletPool

KubePool returns the kube buildlet pool.

func KubeSetErr Uses

func KubeSetErr(err error)

KubeSetErr sets the kube error to passed in value.

func ReadGCSFile Uses

func ReadGCSFile(name string) ([]byte, error)

ReadGCSFile reads the named file from the GCS bucket.

func SetBuilderMasterKey Uses

func SetBuilderMasterKey(masterKey []byte)

SetBuilderMasterKey sets the builder master key used to generate keys used by the builders.

type Buildlet Uses

type Buildlet interface {
    // GetBuildlet returns a new buildlet client.
    // The hostType is the key into the dashboard.Hosts
    // map (such as "host-linux-jessie"), NOT the buidler type
    // ("linux-386").
    // Users of GetBuildlet must both call Client.Close when done
    // with the client as well as cancel the provided Context.
    // The ctx may have context values of type buildletTimeoutOpt
    // and highPriorityOpt.
    GetBuildlet(ctx context.Context, hostType string, lg Logger) (*buildlet.Client, error)

    String() string // TODO(bradfitz): more status stuff

Buildlet defines an interface for a pool of buildlets.

type BuildletTimeoutOpt Uses

type BuildletTimeoutOpt struct{} // context Value key; value is time.Duration

BuildletTimeoutOpt is a context.Value key for BuildletPool.GetBuildlet.

type ByCreationTime Uses

type ByCreationTime []ResourceTime

ByCreationTime provides the functionality to sort resource times by the time of creation.

func (ByCreationTime) Len Uses

func (s ByCreationTime) Len() int

func (ByCreationTime) Less Uses

func (s ByCreationTime) Less(i, j int) bool

func (ByCreationTime) Swap Uses

func (s ByCreationTime) Swap(i, j int)

type EC2Buildlet Uses

type EC2Buildlet struct {
    // contains filtered or unexported fields

EC2Buildlet manages a pool of AWS EC2 buildlets.

func EC2BuildetPool Uses

func EC2BuildetPool() *EC2Buildlet

EC2BuildetPool retrieves the package level EC2Buildlet pool set by the constructor.

TODO( remove once a package level variable is no longer required by the main package.

func NewEC2Buildlet Uses

func NewEC2Buildlet(client *cloud.AWSClient, buildEnv *buildenv.Environment, hosts map[string]*dashboard.HostConfig, fn IsRemoteBuildletFunc, opts ...EC2Opt) (*EC2Buildlet, error)

NewEC2Buildlet creates a new EC2 buildlet pool used to create and manage the lifecycle of EC2 buildlets. Information about ARM64 instance types is retrieved before starting the pool. EC2 quota types are also retrieved before starting the pool. The pool will continuously poll for quotas which limit the resources that can be consumed by the pool. It will also periodically search for VMs which are no longer in use or are untracked by the pool in order to delete them.

func (*EC2Buildlet) Close Uses

func (eb *EC2Buildlet) Close()

Close stops the pollers used by the EC2Buildlet pool from running.

func (*EC2Buildlet) GetBuildlet Uses

func (eb *EC2Buildlet) GetBuildlet(ctx context.Context, hostType string, lg Logger) (*buildlet.Client, error)

GetBuildlet retrieves a buildlet client for a newly created buildlet.

func (*EC2Buildlet) String Uses

func (eb *EC2Buildlet) String() string

String gives a report of capacity usage for the EC2 buildlet pool.

func (*EC2Buildlet) WriteHTMLStatus Uses

func (eb *EC2Buildlet) WriteHTMLStatus(w io.Writer)

WriteHTMLStatus writes the status of the EC2 buildlet pool to an io.Writer.

type EC2Opt Uses

type EC2Opt func(*EC2Buildlet)

EC2Opt is optional configuration for the the buildlet.

func WithVMDeleteTimeout Uses

func WithVMDeleteTimeout(timeout time.Duration) EC2Opt

WithVMDeleteTimeout sets the VM deletion timeout for all EC2 VMs.

type EventTimeLogger Uses

type EventTimeLogger interface {
    LogEventTime(event string, optText ...string)

EventTimeLogger is the logging interface used to log an event at a point in time.

type GCEBuildlet Uses

type GCEBuildlet struct {
    // contains filtered or unexported fields

GCEBuildlet manages a pool of GCE buildlets.

func (*GCEBuildlet) CleanUpOldVMs Uses

func (p *GCEBuildlet) CleanUpOldVMs()

CleanUpOldVMs loops forever and periodically enumerates virtual machines and deletes those which have expired.

A VM is considered expired if it has a "delete-at" metadata attribute having a unix timestamp before the current time.

This is the safety mechanism to delete VMs which stray from the normal deleting process. VMs are created to run a single build and should be shut down by a controlling process. Due to various types of failures, they might get stranded. To prevent them from getting stranded and wasting resources forever, we instead set the "delete-at" metadata attribute on them when created to some time that's well beyond their expected lifetime.

func (*GCEBuildlet) GetBuildlet Uses

func (p *GCEBuildlet) GetBuildlet(ctx context.Context, hostType string, lg Logger) (bc *buildlet.Client, err error)

GetBuildlet retrieves a buildlet client for an available buildlet.

func (*GCEBuildlet) SetEnabled Uses

func (p *GCEBuildlet) SetEnabled(enabled bool)

SetEnabled marks the buildlet pool as enabled.

func (*GCEBuildlet) String Uses

func (p *GCEBuildlet) String() string

func (*GCEBuildlet) WriteHTMLStatus Uses

func (p *GCEBuildlet) WriteHTMLStatus(w io.Writer)

WriteHTMLStatus writes the status of the buildlet pool to an io.Writer.

type GCEConfiguration Uses

type GCEConfiguration struct{}

GCEConfiguration manages and contains all of the GCE configuration.

func NewGCEConfiguration Uses

func NewGCEConfiguration() *GCEConfiguration

NewGCEConfiguration creates a new GCEConfiguration.

func (*GCEConfiguration) BuildEnv Uses

func (c *GCEConfiguration) BuildEnv() *buildenv.Environment

BuildEnv retrieves the GCE build env.

func (*GCEConfiguration) BuildletPool Uses

func (c *GCEConfiguration) BuildletPool() *GCEBuildlet

BuildletPool retrieves the GCE buildlet pool.

func (*GCEConfiguration) DSClient Uses

func (c *GCEConfiguration) DSClient() *datastore.Client

DSClient retrieves the datastore client.

func (*GCEConfiguration) ErrorsClient Uses

func (c *GCEConfiguration) ErrorsClient() *errorreporting.Client

ErrorsClient retrieves the stackdriver errors client.

func (*GCEConfiguration) GCPCredentials Uses

func (c *GCEConfiguration) GCPCredentials() *google.Credentials

GCPCredentials retrieves the GCP credentials.

func (*GCEConfiguration) GKENodeIP Uses

func (c *GCEConfiguration) GKENodeIP() string

GKENodeIP retrieves the GKE node IP.

func (*GCEConfiguration) GerritClient Uses

func (c *GCEConfiguration) GerritClient() *gerrit.Client

GerritClient retrieves a gerrit client.

func (*GCEConfiguration) GoDSClient Uses

func (c *GCEConfiguration) GoDSClient() *datastore.Client

GoDSClient retrieves the datastore client for project.

func (*GCEConfiguration) InStaging Uses

func (c *GCEConfiguration) InStaging() bool

InStaging returns a boolean denoting if the enviornment is stageing.

func (*GCEConfiguration) MetricsClient Uses

func (c *GCEConfiguration) MetricsClient() *monapi.MetricClient

MetricsClient retrieves a metrics client.

func (*GCEConfiguration) OAuthHTTPClient Uses

func (c *GCEConfiguration) OAuthHTTPClient() *http.Client

OAuthHTTPClient retrieves an OAuth2 HTTP client used to make API calls to GCP.

func (*GCEConfiguration) SetBuildEnv Uses

func (c *GCEConfiguration) SetBuildEnv(b *buildenv.Environment)

SetBuildEnv sets the GCE build env. This is primarily reserved for testing purposes.

func (*GCEConfiguration) StorageClient Uses

func (c *GCEConfiguration) StorageClient() *storage.Client

StorageClient retrieves the GCE storage client.

func (*GCEConfiguration) TryDepsErr Uses

func (c *GCEConfiguration) TryDepsErr() error

TryDepsErr retrives any Trybot dependency error.

type IsRemoteBuildletFunc Uses

type IsRemoteBuildletFunc func(instanceName string) bool

IsRemoteBuildletFunc should report whether the buildlet instance name is is a remote buildlet. This is applicable to GCE and EC2 instances.

TODO( should be removed once remote buildlet management functions are moved into a package.

type Logger Uses

type Logger interface {
    EventTimeLogger // point in time
    spanlog.Logger  // action spanning time

Logger is the logging interface used within the coordinator. It can both log a message at a point in time, as well as log a span (something having a start and end time, as well as a final success status).

type MonitorGitMirrorFunc Uses

type MonitorGitMirrorFunc func()

MonitorGitMirrorFunc defines a function used to monitor gitmirror.

type ResourceTime Uses

type ResourceTime struct {
    Name     string
    Creation time.Time

ResourceTime is a GCE instance or Kube pod name and its creation time.

type ReverseBuildletPool Uses

type ReverseBuildletPool struct {
    // contains filtered or unexported fields

ReverseBuildletPool manages the pool of reverse buildlet pools.

func ReversePool Uses

func ReversePool() *ReverseBuildletPool

ReversePool retrieves the reverse buildlet pool.

func (*ReverseBuildletPool) BuildReverseStatusJSON Uses

func (p *ReverseBuildletPool) BuildReverseStatusJSON() *types.ReverseBuilderStatus

BuildReverseStatusJSON is an HTTP handler implementation which builds the reverse status reverse buildlets.

func (*ReverseBuildletPool) BuildletHostnames Uses

func (p *ReverseBuildletPool) BuildletHostnames() []string

BuildletHostnames returns a slice of reverse buildlet hostnames.

func (*ReverseBuildletPool) BuildletLastSeen Uses

func (p *ReverseBuildletPool) BuildletLastSeen(host string) (time.Time, bool)

BuildletLastSeen gives the last time a buildlet was connected to the pool. If the buildlet has not been seen a false is returned by the boolean.

func (*ReverseBuildletPool) CanBuild Uses

func (p *ReverseBuildletPool) CanBuild(hostType string) bool

CanBuild reports whether the pool has a machine capable of building mode, even if said machine isn't currently idle.

func (*ReverseBuildletPool) GetBuildlet Uses

func (p *ReverseBuildletPool) GetBuildlet(ctx context.Context, hostType string, lg Logger) (*buildlet.Client, error)

GetBuildlet builds a buildlet client for the passed in host.

func (*ReverseBuildletPool) HostTypeCount Uses

func (p *ReverseBuildletPool) HostTypeCount() map[string]int

HostTypeCount iterates through the running reverse buildlets, and constructs a count of running buildlets per hostType.

func (*ReverseBuildletPool) HostTypes Uses

func (p *ReverseBuildletPool) HostTypes() (types []string)

HostTypes returns the a deduplicated list of buildlet types curently supported by the pool.

func (*ReverseBuildletPool) ServeReverseStatusJSON Uses

func (p *ReverseBuildletPool) ServeReverseStatusJSON(w http.ResponseWriter, r *http.Request)

ServeReverseStatusJSON is an HTTP handler implementation which serves the status in JSON format.

func (*ReverseBuildletPool) SingleHostTypeCount Uses

func (p *ReverseBuildletPool) SingleHostTypeCount(hostType string) int

SingleHostTypeCount iterates through the running reverse buildlets, and constructs a count of the running buildlet hostType requested.

func (*ReverseBuildletPool) String Uses

func (p *ReverseBuildletPool) String() string

func (*ReverseBuildletPool) WriteHTMLStatus Uses

func (p *ReverseBuildletPool) WriteHTMLStatus(w io.Writer)

WriteHTMLStatus writes a status of the reverse buildlet pool, in HTML format,

to the passed in io.Writer.

Package pool imports 49 packages (graph) and is imported by 1 packages. Updated 2021-01-21. Refresh now. Tools for package owners.