build: golang.org/x/build/buildlet Index | Files

package buildlet

import "golang.org/x/build/buildlet"

Package buildlet contains client tools for working with a buildlet server.

Index

Package Files

buildletclient.go gce.go keypair.go kube.go remote.go

Variables

var (
    // TODO(evanbrown): resource requirements should be
    // defined per-builder in dashboard/builders.go
    BuildletCPU      = api.MustParse("2")         // 2 Cores
    BuildletCPULimit = api.MustParse("8")         // 8 Cores
    BuildletMemory   = api.MustParse("4000000Ki") // 4,000,000Ki RAM
)
var ErrClosed = errors.New("buildlet: Client closed")
var ErrTimeout = errors.New("buildlet: timeout waiting for command to complete")
var GCEGate func()

GCEGate optionally specifies a function to run before any GCE API call. It's intended to be used to bound QPS rate to GCE.

var NoKeyPair = KeyPair{}

NoKeyPair is used by the coordinator to speak http directly to buildlets, inside their firewall, without TLS.

func DestroyVM Uses

func DestroyVM(ts oauth2.TokenSource, proj, zone, instance string) error

DestroyVM sends a request to delete a VM. Actual VM description is currently (2015-01-19) very slow for no good reason. This function returns once it's been requested, not when it's done.

func ListVMs Uses

func ListVMs(ts oauth2.TokenSource, proj, zone string) ([]VM, error)

ListVMs lists all VMs.

func RegisterFlags Uses

func RegisterFlags()

RegisterFlags registers "user" and "staging" flags that control the behavior of NewCoordinatorClientFromFlags. These are used by remote client commands like gomote.

type Client Uses

type Client struct {
    // contains filtered or unexported fields
}

A Client interacts with a single buildlet.

func NewClient Uses

func NewClient(ipPort string, kp KeyPair) *Client

NewClient returns a *Client that will manipulate ipPort, authenticated using the provided keypair.

This constructor returns immediately without testing the host or auth.

func StartNewVM Uses

func StartNewVM(ts oauth2.TokenSource, instName, hostType string, opts VMOpts) (*Client, error)

StartNewVM boots a new VM on GCE and returns a buildlet client configured to speak to it.

func StartPod Uses

func StartPod(ctx context.Context, kubeClient *kubernetes.Client, podName, hostType string, opts PodOpts) (*Client, error)

StartPod creates a new pod on a Kubernetes cluster and returns a buildlet client configured to speak to it.

func (*Client) Close Uses

func (c *Client) Close() error

Closes destroys and closes down the buildlet, destroying all state immediately.

func (*Client) ConnectSSH Uses

func (c *Client) ConnectSSH(user, authorizedPubKey string) (net.Conn, error)

ConnectSSH opens an SSH connection to the buildlet for the given username. The authorizedPubKey must be a line from an ~/.ssh/authorized_keys file and correspond to the private key to be used to communicate over the net.Conn.

func (*Client) DestroyVM Uses

func (c *Client) DestroyVM(ts oauth2.TokenSource, proj, zone, instance string) error

DestroyVM shuts down the buildlet and destroys the VM instance.

func (*Client) Exec Uses

func (c *Client) Exec(cmd string, opts ExecOpts) (remoteErr, execErr error)

Exec runs cmd on the buildlet.

Two errors are returned: one is whether the command succeeded remotely (remoteErr), and the second (execErr) is whether there were system errors preventing the command from being started or seen to completition. If execErr is non-nil, the remoteErr is meaningless.

func (*Client) GetTar Uses

func (c *Client) GetTar(ctx context.Context, dir string) (io.ReadCloser, error)

GetTar returns a .tar.gz stream of the given directory, relative to the buildlet's work dir. The provided dir may be empty to get everything.

func (*Client) IPPort Uses

func (c *Client) IPPort() string

func (*Client) IsBroken Uses

func (c *Client) IsBroken() bool

IsBroken reports whether this client is broken in some way.

func (*Client) ListDir Uses

func (c *Client) ListDir(dir string, opts ListDirOpts, fn func(DirEntry)) error

ListDir lists the contents of a directory. The fn callback is run for each entry.

func (*Client) MarkBroken Uses

func (c *Client) MarkBroken()

MarkBroken marks this client as broken in some way.

func (*Client) Name Uses

func (c *Client) Name() string

func (*Client) ProxyRoundTripper Uses

func (c *Client) ProxyRoundTripper() http.RoundTripper

ProxyRoundTripper returns a RoundTripper that sends HTTP requests directly through to the underlying buildlet, adding auth and X-Buildlet-Proxy headers as necessary. This is really only intended for use by the coordinator.

func (*Client) Put Uses

func (c *Client) Put(r io.Reader, path string, mode os.FileMode) error

Put writes the provided file to path (relative to workdir) and sets mode.

func (*Client) PutTar Uses

func (c *Client) PutTar(r io.Reader, dir string) error

PutTar writes files to the remote buildlet, rooted at the relative directory dir. If dir is empty, they're placed at the root of the buildlet's work directory. The dir is created if necessary. The Reader must be of a tar.gz file.

func (*Client) PutTarFromURL Uses

func (c *Client) PutTarFromURL(tarURL, dir string) error

PutTarFromURL tells the buildlet to download the tar.gz file from tarURL and write it to dir, a relative directory from the workdir. If dir is empty, they're placed at the root of the buildlet's work directory. The dir is created if necessary. The url must be of a tar.gz file.

func (*Client) RemoteName Uses

func (c *Client) RemoteName() string

RemoteName returns the name of this client's buildlet on the coordinator. If this buildlet isn't a remote buildlet created via a buildlet, this returns the empty string.

func (*Client) RemoveAll Uses

func (c *Client) RemoveAll(paths ...string) error

RemoveAll deletes the provided paths, relative to the work directory.

func (*Client) SetDescription Uses

func (c *Client) SetDescription(v string)

SetDescription sets a short description of where the buildlet connection came from. This is used by the build coordinator status page, mostly for debugging.

func (*Client) SetDialer Uses

func (c *Client) SetDialer(dialer func() (net.Conn, error))

SetDialer sets the function that creates a new connection to the buildlet. By default, net.Dial is used.

func (*Client) SetHTTPClient Uses

func (c *Client) SetHTTPClient(httpClient *http.Client)

SetHTTPClient replaces the underlying HTTP client. It should only be called before the Client is used.

func (*Client) SetOnHeartbeatFailure Uses

func (c *Client) SetOnHeartbeatFailure(fn func())

SetOnHeartbeatFailure sets a function to be called when heartbeats against this builder fail, or when the client is destroyed with Close. The function fn is never called more than once. SetOnHeartbeatFailure must be set before any use of the buildlet.

func (*Client) SetReleaseMode Uses

func (c *Client) SetReleaseMode(v bool)

SetReleaseMode sets whether this client is being used in "release mode", to prepare the final binaries to be shipped to users. All this mode does for now is disable pargzip multi-stream gzip files. See golang.org/issue/19052.

SetReleaseMode must be set before using the client.

func (*Client) Status Uses

func (c *Client) Status() (Status, error)

Status returns an Status value describing this buildlet.

func (*Client) String Uses

func (c *Client) String() string

func (*Client) URL Uses

func (c *Client) URL() string

URL returns the buildlet's URL prefix, without a trailing slash.

func (*Client) WorkDir Uses

func (c *Client) WorkDir() (string, error)

WorkDir returns the absolute path to the buildlet work directory.

type CoordinatorClient Uses

type CoordinatorClient struct {
    // Auth specifies how to authenticate to the coordinator.
    Auth UserPass

    // Instance optionally specifies the build coordinator to connect
    // to. If zero, the production coordinator is used.
    Instance build.CoordinatorInstance
    // contains filtered or unexported fields
}

A CoordinatorClient makes calls to the build coordinator.

func NewCoordinatorClientFromFlags Uses

func NewCoordinatorClientFromFlags() (*CoordinatorClient, error)

NewCoordinatorClientFromFlags constructs a CoordinatorClient for the current user.

func (*CoordinatorClient) CreateBuildlet Uses

func (cc *CoordinatorClient) CreateBuildlet(builderType string) (*Client, error)

CreateBuildlet creates a new buildlet of the given builder type on cc.

This takes a builderType (instead of a hostType), but the returned buildlet can be used as any builder that has the same underlying buildlet type. For instance, a linux-amd64 buildlet can act as either linux-amd64 or linux-386-387.

It may expire at any time. To release it, call Client.Destroy.

func (*CoordinatorClient) NamedBuildlet Uses

func (cc *CoordinatorClient) NamedBuildlet(name string) (*Client, error)

NamedBuildlet returns a buildlet client for the named remote buildlet. Names are not validated. Use Client.Status to check whether the client works.

func (*CoordinatorClient) RemoteBuildlets Uses

func (cc *CoordinatorClient) RemoteBuildlets() ([]RemoteBuildlet, error)

type DirEntry Uses

type DirEntry struct {
    // contains filtered or unexported fields
}

DirEntry is the information about a file on a buildlet.

func (DirEntry) Digest Uses

func (de DirEntry) Digest() string

func (DirEntry) Name Uses

func (de DirEntry) Name() string

func (DirEntry) String Uses

func (de DirEntry) String() string

type ExecOpts Uses

type ExecOpts struct {
    // Output is the output of stdout and stderr.
    // If nil, the output is discarded.
    Output io.Writer

    // Dir is the directory from which to execute the command.
    // It is optional. If not specified, it defaults to the directory of
    // the command, or the work directory if SystemLevel is set.
    Dir string

    // Args are the arguments to pass to the cmd given to Client.Exec.
    Args []string

    // ExtraEnv are KEY=VALUE pairs to append to the buildlet
    // process's environment.
    ExtraEnv []string

    // Path, if non-nil, specifies the PATH variable of the executed
    // process's environment. A non-nil empty list clears the path.
    // The following expansions apply:
    //   - the string "$PATH" expands to any existing PATH element(s)
    //   - the substring "$WORKDIR" expands to buildlet's temp workdir
    // After expansions, the list is joined with an OS-specific list
    // separator and supplied to the executed process as its PATH
    // environment variable.
    Path []string

    // SystemLevel controls whether the command is run outside of
    // the buildlet's environment.
    SystemLevel bool

    // Debug, if true, instructs to the buildlet to print extra debug
    // info to the output before the command begins executing.
    Debug bool

    // OnStartExec is an optional hook that runs after the 200 OK
    // response from the buildlet, but before the output begins
    // writing to Output.
    OnStartExec func()

    // Timeout is an optional duration before ErrTimeout is returned.
    Timeout time.Duration
}

ExecOpts are options for a remote command invocation.

type KeyPair Uses

type KeyPair struct {
    CertPEM string
    KeyPEM  string
}

KeyPair is the TLS public certificate PEM file and its associated private key PEM file that a builder will use for its HTTPS server. The zero value means no HTTPs, which is used by the coordinator for machines running within a firewall.

func NewKeyPair Uses

func NewKeyPair() (KeyPair, error)

func (KeyPair) IsZero Uses

func (kp KeyPair) IsZero() bool

func (KeyPair) Password Uses

func (kp KeyPair) Password() string

Password returns the SHA1 of the KeyPEM. This is used as the HTTP Basic Auth password.

type ListDirOpts Uses

type ListDirOpts struct {
    // Recursive controls whether the directory is listed
    // recursively.
    Recursive bool

    // Skip are the directories to skip, relative to the directory
    // passed to ListDir. Each item should contain only forward
    // slashes and not start or end in slashes.
    Skip []string

    // Digest controls whether the SHA-1 digests of regular files
    // are returned.
    Digest bool
}

ListDirOpts are options for Client.ListDir.

type PodOpts Uses

type PodOpts struct {
    // ProjectID is the GCE project ID. Required.
    ProjectID string

    // ImageRegistry specifies the Docker registry Kubernetes
    // will use to create the pod. Required.
    ImageRegistry string

    // TLS optionally specifies the TLS keypair to use.
    // If zero, http without auth is used.
    TLS KeyPair

    // Description optionally describes the pod.
    Description string

    // Labels optionally specify key=value strings that Kubernetes
    // can use to filter and group pods.
    Labels map[string]string

    // DeleteIn optionally specifies a duration at which
    // to delete the pod.
    DeleteIn time.Duration

    // OnPodCreating optionally specifies a hook to run synchronously
    // after the pod create request has been made, but before the create
    // has succeeded.
    OnPodCreating func()

    // OnPodCreated optionally specifies a hook to run synchronously
    // after the pod create request succeeds.
    OnPodCreated func()

    // OnGotPodInfo optionally specifies a hook to run synchronously
    // after the pod Get call.
    OnGotPodInfo func()
}

PodOpts control how new pods are started.

type RemoteBuildlet Uses

type RemoteBuildlet struct {
    HostType    string // "host-linux-kubestd"
    BuilderType string // "linux-386-387"
    Name        string // "buildlet-adg-openbsd-386-2"
    Created     time.Time
    Expires     time.Time
}

type Status Uses

type Status struct {
    Version int // buildlet version, coordinator rejects any value less than 1.
}

Status provides status information about the buildlet.

A coordinator can use the provided information to decide what, if anything, to do with a buildlet.

type UserPass Uses

type UserPass struct {
    Username string // "user-$USER"
    Password string // buildlet key
}

type VM Uses

type VM struct {
    // Name is the name of the GCE VM instance.
    // For example, it's of the form "mote-bradfitz-plan9-386-foo",
    // and not "plan9-386-foo".
    Name   string
    IPPort string
    TLS    KeyPair
    Type   string // buildlet type
}

type VMOpts Uses

type VMOpts struct {
    // Zone is the GCE zone to create the VM in. Required.
    Zone string

    // ProjectID is the GCE project ID. Required.
    ProjectID string

    // TLS optionally specifies the TLS keypair to use.
    // If zero, http without auth is used.
    TLS KeyPair

    // Optional description of the VM.
    Description string

    // Optional metadata to put on the instance.
    Meta map[string]string

    // DeleteIn optionally specifies a duration at which
    // to delete the VM.
    DeleteIn time.Duration

    // OnInstanceRequested optionally specifies a hook to run synchronously
    // after the computeService.Instances.Insert call, but before
    // waiting for its operation to proceed.
    OnInstanceRequested func()

    // OnInstanceCreated optionally specifies a hook to run synchronously
    // after the instance operation succeeds.
    OnInstanceCreated func()

    // OnInstanceCreated optionally specifies a hook to run synchronously
    // after the computeService.Instances.Get call.
    OnGotInstanceInfo func()

    // OnBeginBuildletProbe optionally specifies a hook to run synchronously
    // before StartNewVM tries to hit buildletURL to see if it's up yet.
    OnBeginBuildletProbe func(buildletURL string)

    // OnEndBuildletProbe optionally specifies a hook to run synchronously
    // after StartNewVM tries to hit the buildlet's URL to see if it's up.
    // The hook parameters are the return values from http.Get.
    OnEndBuildletProbe func(*http.Response, error)

    // FallbackToFullPrice optionally specifies a hook to return a new
    // GCE instance name if the first one failed to launch
    // as preemptible. (If you use the same name, GCE complains about
    // resources already existing, even if it failed to be created)
    FallbackToFullPrice func() (newInstname string)
}

VMOpts control how new VMs are started.

Package buildlet imports 36 packages (graph) and is imported by 7 packages. Updated 2017-11-18. Refresh now. Tools for package owners.