provision: Index | Files

package api

import ""

Package api implements a client API for working with digitalrebar/provision.


Package Files

agent.go bootenv.go chroot_linux.go client.go cmdHelper.go content.go event.go event_stream.go jobs.go utils.go


const (
    AGENT_INIT = AgentState(iota)
const APIPATH = "/api/v3"

APIPATH is the base path for all API endpoints that digitalrebar provision provides.

func DecodeYaml Uses

func DecodeYaml(buf []byte, ref interface{}) error

DecodeYaml is a helper function for dealing with user input -- when accepting input from the user, we want to treat both YAML and JSON as first-class citizens. The YAML library we use makes that easier by using the json struct tags for all marshalling and unmarshalling purposes.

Note that the REST API does not use YAML as a wire protocol, so this function should never be used to decode data coming from the provision service.

func GenPatch Uses

func GenPatch(source, target interface{}, paranoid bool) (jsonpatch2.Patch, error)

GenPatch generates a JSON patch that will transform source into target. The generated patch will have all the applicable test clauses.

func Pretty Uses

func Pretty(f string, obj interface{}) ([]byte, error)

Pretty marshals object acciording to the the fmt, in whatever passed for "pretty" according to fmt.

type AgentState Uses

type AgentState int

type Client Uses

type Client struct {
    // contains filtered or unexported fields

Client wraps *http.Client to include our authentication routines and routines for handling some of the biolerplate CRUD operations against digitalrebar provision.

func TokenSession Uses

func TokenSession(endpoint, token string) (*Client, error)

TokenSession creates a new api.Client that will use the passed-in Token for authentication. It should be used whenever the API is not acting on behalf of a user.

func UserSession Uses

func UserSession(endpoint, username, password string) (*Client, error)

UserSession creates a new api.Client that can act on behalf of a user. It will perform a single request using basic authentication to get a token that expires 600 seconds from the time the session is crated, and every 300 seconds it will refresh that token.

UserSession does not currently attempt to cache tokens to persistent storage, although that may change in the future.

func (*Client) Agent Uses

func (c *Client) Agent(m *models.Machine, exitOnNotRunnable, exitOnFailure, actuallyPowerThings bool, logger io.Writer) error

Agent runs the machine Agent on the current machine. It assumes there is only one Agent, which is not actually a safe assumption. We should make it safe someday.

func (*Client) AllIndexes Uses

func (c *Client) AllIndexes() (map[string]map[string]models.Index, error)

AllIndexes returns all the static indexes available for all object types on the server.

func (*Client) Authorize Uses

func (c *Client) Authorize(req *http.Request) error

Authorize sets the Authorization header in the Request with the current bearer token. The rest of the helper methods call this, so you don't have to unless you are building your own http.Requests.

func (*Client) BundleContent Uses

func (c *Client) BundleContent(src string, dst store.Store, params map[string]string) error

func (*Client) Close Uses

func (c *Client) Close()

Close should be called whenever you no longer want to use this client connection. It will stop any token refresh routines running in the background, and force any API calls made to this client that would communicate with the server to return an error

func (*Client) CreateContent Uses

func (c *Client) CreateContent(content *models.Content) (*models.ContentSummary, error)

func (*Client) CreateModel Uses

func (c *Client) CreateModel(ref models.Model) error

CreateModel takes the passed-in model and creates an instance of it on the server. It will return an error if the passed-in model does not validate or if it already exists on the server.

func (*Client) DeleteBlob Uses

func (c *Client) DeleteBlob(at ...string) error

DeleteBlob deletes a blob on the server at the location indicated by 'at'

func (*Client) DeleteContent Uses

func (c *Client) DeleteContent(name string) error

func (*Client) DeleteModel Uses

func (c *Client) DeleteModel(prefix, key string) (models.Model, error)

DeleteModel deletes the model matching the passed-in prefix and key. It returns the object that was deleted.

func (*Client) Endpoint Uses

func (c *Client) Endpoint() string

func (*Client) Events Uses

func (c *Client) Events() (*EventStream, error)

Events creates a new EventStream from the client.

func (*Client) ExistsModel Uses

func (c *Client) ExistsModel(prefix, key string) (bool, error)

ExistsModel tests to see if an object exists on the server following the same rules as GetModel

func (*Client) File Uses

func (c *Client) File(pathParts ...string) (io.ReadCloser, error)

func (*Client) FillModel Uses

func (c *Client) FillModel(ref models.Model, key string) error

FillModel fills the passed-in model with new information retrieved from the server.

func (*Client) GetBlob Uses

func (c *Client) GetBlob(dest io.Writer, at ...string) error

GetBlob fetches a binary blob from the server, writing it to the passed io.Writer.

func (*Client) GetContentItem Uses

func (c *Client) GetContentItem(name string) (*models.Content, error)

func (*Client) GetContentSummary Uses

func (c *Client) GetContentSummary() ([]*models.ContentSummary, error)

func (*Client) GetModel Uses

func (c *Client) GetModel(prefix, key string, params ...string) (models.Model, error)

GetModel returns an object if type prefix with the unique identifier key, if such an object exists. Key can be either the unique key for an object, or any field on an object that has an index that enforces uniqueness.

func (*Client) GetModelForPatch Uses

func (c *Client) GetModelForPatch(prefix, key string, params ...string) (models.Model, models.Model, error)

func (*Client) Indexes Uses

func (c *Client) Indexes(prefix string) (map[string]models.Index, error)

Indexes returns all the static indexes available for a given type of object on the server.

func (*Client) Info Uses

func (c *Client) Info() (*models.Info, error)

Info returns some basic system information that was retrieved as part of the initial authentication.

func (*Client) InstallBootEnvFromFile Uses

func (c *Client) InstallBootEnvFromFile(src string) (*models.BootEnv, error)

func (*Client) InstallISOForBootenv Uses

func (c *Client) InstallISOForBootenv(env *models.BootEnv, src string, downloadOK bool) error

func (*Client) InstallRawTemplateFromFile Uses

func (c *Client) InstallRawTemplateFromFile(src string) (*models.Template, error)

func (*Client) InstallRawTemplateFromFileWithId Uses

func (c *Client) InstallRawTemplateFromFileWithId(src, tid string) (*models.Template, error)

func (*Client) JobActions Uses

func (c *Client) JobActions(j *models.Job, targetOS string) (models.JobActions, error)

JobActions returns the expanded list of templates that should be written or executed for a specific Job.

func (*Client) JobLog Uses

func (c *Client) JobLog(j *models.Job, dst io.Writer) error

JobLog gets the log for a specific Job and writes it to the passed io.Writer

func (*Client) ListBlobs Uses

func (c *Client) ListBlobs(at string, params ...string) ([]string, error)

ListBlobs lists the names of all the binary objects at 'at', using the indexing parameters suppied by params.

func (*Client) ListModel Uses

func (c *Client) ListModel(prefix string, params ...string) ([]models.Model, error)

func (*Client) Logs Uses

func (c *Client) Logs() ([]logger.Line, error)

Logs returns the currently buffered logs from the dr-provision server

func (*Client) NewAgent Uses

func (c *Client) NewAgent(m *models.Machine,
    exitOnNotRunnable, exitOnFailure, actuallyPowerThings bool,
    logger io.Writer) (*MachineAgent, error)

NewAgent creates a new FSM based Machine Agent that starts out in the AGENT_INIT state.

func (*Client) OneIndex Uses

func (c *Client) OneIndex(prefix, param string) (models.Index, error)

OneIndex tests to see if there is an index on the object type indicated by prefix for a specific parameter. If the returned Index is empty, there is no such Index.

func (*Client) PatchModel Uses

func (c *Client) PatchModel(prefix, key string, patch jsonpatch2.Patch) (models.Model, error)

PatchModel attempts to update the object matching the passed prefix and key on the server side with the passed-in JSON patch (as sepcified in To ensure that conflicting changes are rejected, your patch should contain the appropriate test stanzas, which will allow the server to detect and reject conflicting changes from different sources.

func (*Client) PatchTo Uses

func (c *Client) PatchTo(old models.Model, new models.Model) (models.Model, error)

func (*Client) PatchToFull Uses

func (c *Client) PatchToFull(old models.Model, new models.Model, paranoid bool) (models.Model, error)

func (*Client) PostBlob Uses

func (c *Client) PostBlob(blob io.Reader, at ...string) (models.BlobInfo, error)

PostBlob uploads the binary blob contained in the passed io.Reader to the location specified by at on the server. You are responsible for closing the passed io.Reader.

func (*Client) PostEvent Uses

func (c *Client) PostEvent(evt *models.Event) error

func (*Client) PutModel Uses

func (c *Client) PutModel(obj models.Model) error

PutModel replaces the server-side object matching the passed-in object with the passed-in object. Note that PutModel does not allow the server to detect and reject conflicting changes from multiple sources.

func (*Client) ReplaceContent Uses

func (c *Client) ReplaceContent(content *models.Content) (*models.ContentSummary, error)

func (*Client) Req Uses

func (c *Client) Req() *R

Req creates a new R for the current client. It defaults to using the GET method.

func (*Client) Token Uses

func (c *Client) Token() string

Token returns the current authentication token associated with the Client.

func (*Client) Trace Uses

func (c *Client) Trace(lvl string)

Trace sets the log level that incoming requests generated by a Client will be logged at, overriding the levels they would normally be logged at on the server side. Setting lvl to an empty string turns off tracing.

func (*Client) TraceToken Uses

func (c *Client) TraceToken(t string)

TraceToken is a unique token that the server-side logger will emit a log for at the Error log level. It can be used to tie logs generated on the server side to requests made by a specific Client.

func (*Client) UnbundleContent Uses

func (c *Client) UnbundleContent(content store.Store, dst string) error

func (*Client) UploadISOForBootEnv Uses

func (c *Client) UploadISOForBootEnv(env *models.BootEnv, src io.Reader, dest string) (models.BlobInfo, error)

func (*Client) UrlFor Uses

func (c *Client) UrlFor(args ...string) (*url.URL, error)

func (*Client) Username Uses

func (c *Client) Username() string

type Decoder Uses

type Decoder interface {
    Decode(interface{}) error

type Encoder Uses

type Encoder interface {
    Encode(interface{}) error

type EventStream Uses

type EventStream struct {
    // contains filtered or unexported fields

EventStream receives events from the digitalrebar provider. You can read received events by reading from its Events channel.

func (*EventStream) Close Uses

func (es *EventStream) Close() error

Close closes down the EventStream. You should drain the Events until you read a RecievedEvent that has an empty E and a non-nil Err

func (*EventStream) Deregister Uses

func (es *EventStream) Deregister(handle int64) error

Deregister directs the EventStream to unsubscribe from Events from the digitalrebar provisioner. It takes the same parameters as Register.

func (*EventStream) Register Uses

func (es *EventStream) Register(events ...string) (int64, <-chan RecievedEvent, error)

Register directs the EventStream to subscribe to Events from the digital rebar provisioner.

Event subscriptions consist of a string with the following format:


type is the object type that you want to listen for events about. * means to listen for events about all object types.

action is the action that caused the event to be created. * means to listen for all actions.

key is the unique identifier of the object to listen for. * means to listen for events from all objects

func (*EventStream) Subscribe Uses

func (es *EventStream) Subscribe(handle int64, events ...string) error

func (*EventStream) WaitFor Uses

func (es *EventStream) WaitFor(
    item models.Model,
    test TestFunc,
    timeout time.Duration) (string, error)

WaitFor waits for an item to match test. It subscribes to an EventStream that watches all update and save envents for the object in question, and returns a string indicating whether the match succeeded, failed, or timed out.

The API for this function is subject to refactoring and change, and should not be considered to be stable yet.

type MachineAgent Uses

type MachineAgent struct {
    // contains filtered or unexported fields

func (*MachineAgent) ChangeStage Uses

func (a *MachineAgent) ChangeStage()

ChangeStage handles determining what to do when the Agent runs out of tasks to run in the current Stage. It may transition to the following states:

* AGENT_WAIT_FOR_CHANGE_STAGE if there is no next stage for this

machine in the change stage map and it is not in an -install

* AGENT_EXIT if there is no next stage in the change stage map and

the machine is in an -install bootenv.  In this case, ChangeStage
will set the machine stage to `local`.

* AGENT_REBOOT if the next stage has the Reboot flag, the change

stage map has a Reboot specifier, or the next stage has a different
bootenv than the machine and the machine is not in an -install

* AGENT_EXIT if the machine is in an -install bootenv and the next

stage requires a different bootenv.

* AGENT_POWEROFF if the change stage map wants to power the system

off after changing the stage.

* AGENT_WAIT_FOR_RUNNABLE if no other condition applies

func (*MachineAgent) Init Uses

func (a *MachineAgent) Init()

Init resets the Machine Agent back to its initial state. This consists of marking any current running jobs as Failed and repoening the event stream from dr-provision.

func (*MachineAgent) Logf Uses

func (a *MachineAgent) Logf(f string, args ...interface{})

Logf is a helper function to make logging of Agent actions a bit easier.

func (*MachineAgent) Run Uses

func (a *MachineAgent) Run() error

Run kicks off the state machine for this agent.

func (*MachineAgent) RunTask Uses

func (a *MachineAgent) RunTask()

RunTask attempts to run the next task on the Machine. It may transition to the following states:

* AGENT_CHANGE_STAGE if there are no tasks to run.

* AGENT_REBOOT if the task signalled that the machine should reboot

* AGENT_POWEROFF if the task signalled that the machine should shut down

* AGENT_EXIT if the task signalled that the agent should stop.

* AGENT_WAIT_FOR_RUNNABLE if no other conditions were met.

func (*MachineAgent) Timeout Uses

func (a *MachineAgent) Timeout(t time.Duration) *MachineAgent

Timeout allows you to change how long the Agent will wait for an event from dr-provision from the default of 1 hour.

func (*MachineAgent) WaitChangeStage Uses

func (a *MachineAgent) WaitChangeStage()

WaitChangeStage has waitOn wait for any of the following on the machine to change:

* The CurrentTask index * The task list * The Runnable flag * The boot environment * The stage

func (*MachineAgent) WaitRunnable Uses

func (a *MachineAgent) WaitRunnable()

WaitRunnable has waitOn wait for the Machine to become runnable.

type R Uses

type R struct {
    Req  *http.Request
    Resp *http.Response
    // contains filtered or unexported fields

R encapsulates a single Request/Response round trip. It has a slew of helper methods that can be chained together to handle all common operations with this API. It handles capturing any errors that may occur in building and executing the request.

func (*R) Body Uses

func (r *R) Body(b interface{}) *R

Body arranges for b to be used as the body of the request. It also sets the Content-Type of the request depending on what the body is:

If b is an io.Reader or a raw byte array, Content-Type will be set to application/octet-stream, otherwise Content-Type will be set to application/json.

If b is something other than nil, an io.Reader, or a byte array, Body will attempt to marshal the object as a JSON byte array and use that.

func (*R) Del Uses

func (r *R) Del() *R

Del sets the R method to DELETE

func (*R) Delete Uses

func (r *R) Delete(m models.Model) error

Delete deletes a single object

func (*R) Do Uses

func (r *R) Do(val interface{}) error

Do attempts to execute the reqest built up by previous method calls on R. If any errors occurred while building up the request, they will be returned and no API interaction will actually take place. Otherwise, Do will generate am http.Request, perform it, and marshal the results to val. If any errors occur while processing the request, fibbonaci based backoff will be performed up to 6 times.

If val is an io.Writer, the body of the response will be copied verbatim into val using io.Copy

Otherwise, the response body will be unmarshalled into val as directed by the Content-Type header of the response.

func (*R) FailFast Uses

func (r *R) FailFast() *R

FailFast skips the usual fibbonaci backoff retry in the case of transient errors.

func (*R) Fill Uses

func (r *R) Fill(m models.Model) error

func (*R) Filter Uses

func (r *R) Filter(prefix string, filterArgs ...string) *R

Filter is a helper for using freeform index operations. The prefix arg is the type of object you want to filter, and filterArgs describes how you want the results filtered. Currently, filterArgs must be in the following format:

    to reverse the order of the results
"sort" "indexName"
    to sort the results according to indexName's native ordering
"limit" "number"
    to limit the number of results returned
"offset" "number"
    to skip <number> of results before returning
"indexName" "Eq/Lt/Lte/Gt/Gte/Ne" "value"
    to return results Equal, Less Than, Less Than Or Equal, Greater Than, Greater Than Or Equal, or Not Equal to value according to IndexName
"indexName" "Between/Except" "lowerBound" "upperBound"
    to return values Between(inclusive) lowerBound and Upperbound or its complement for Except.

If formatArgs does not contain some valid combination of the above, the request will fail.

func (*R) Get Uses

func (r *R) Get() *R

Get sets the R method to GET

func (*R) Head Uses

func (r *R) Head() *R

Head sets the R method to HEAD

func (*R) Headers Uses

func (r *R) Headers(args ...string) *R

Headers arranges for its arguments to be added as HTTP headers. You must pass an even number of arguments to Headers

func (*R) List Uses

func (r *R) List(prefix string) *R

func (*R) Meth Uses

func (r *R) Meth(v string) *R

Meth sets an arbitrary method for R

func (*R) Params Uses

func (r *R) Params(args ...string) *R

Params appends query parameters to the URL R will use. r.UrlFor() or r.UrlForM() must have already been called. You must pass an even number of parameters to Params

func (*R) ParanoidPatch Uses

func (r *R) ParanoidPatch() *R

Must be used before PatchXXX calls

func (*R) Patch Uses

func (r *R) Patch(b jsonpatch2.Patch) *R

Patch sets the R method to PATCH, and arranges for b (which must be a valid JSON patch) to be used as the body of the request by calling r.Body().

func (*R) PatchObj Uses

func (r *R) PatchObj(old, new interface{}) *R

func (*R) PatchTo Uses

func (r *R) PatchTo(old, new models.Model) *R

func (*R) PatchToFull Uses

func (r *R) PatchToFull(old models.Model, new models.Model, paranoid bool) (models.Model, error)

func (*R) Post Uses

func (r *R) Post(b interface{}) *R

Post sets the R method to POST, and arranged for b to be the body of the request by calling r.Body().

func (*R) Put Uses

func (r *R) Put(b interface{}) *R

Put sets the R method to PUT, and arranges for b to be used as the body of the request by calling r.Body(). If no body is desired, b can be nil

func (*R) Trace Uses

func (r *R) Trace(lvl string) *R

Trace will arrange for the server to log this specific request at the passed-in Level, overriding any client Trace requests or the levels things would usually be logged at by the server.

func (*R) TraceToken Uses

func (r *R) TraceToken(t string) *R

TraceToken is a unique token that the server-side logger will emit a log for at the Error log level. It can be used to tie logs generated on the server side to requests made by a specific Req.

func (*R) UrlFor Uses

func (r *R) UrlFor(args ...string) *R

UrlFor arranges for a sane request URL to be used for R. The generated URL will be in the form of:


func (*R) UrlForM Uses

func (r *R) UrlForM(m models.Model, rest ...string) *R

UrlForM is similar to UrlFor, but the prefix and key of the passed-in Model will be used as the first two path components in the URL after /api/v3. If m.Key() == "", it will be omitted.

type RecievedEvent Uses

type RecievedEvent struct {
    E   models.Event
    Err error

RecievedEvent contains an event received from the digitalrebar provision server along with any errors that occurred while receiving the event.

type TaskRunner Uses

type TaskRunner struct {
    // contains filtered or unexported fields

TaskRunner is responsible for expanding templates and running scripts for a single task.

func NewTaskRunner Uses

func NewTaskRunner(c *Client, m *models.Machine, agentDir, chrootDir string, logger io.Writer) (*TaskRunner, error)

NewTaskRunner creates a new TaskRunner for the passed-in machine. It creates the matching Job (or resumes the previous incomplete one), and handles making sure that all relevant output is written to the job log as well as local stderr

func (*TaskRunner) Close Uses

func (r *TaskRunner) Close()

Close() shuts down the writer side of the logging pipe. This will also flush any remaining data to stderr

func (*TaskRunner) Expand Uses

func (r *TaskRunner) Expand(action *models.JobAction, taskDir string) error

Expand a writes a file template to the appropriate location.

func (*TaskRunner) Log Uses

func (r *TaskRunner) Log(s string, items ...interface{})

Log writes the string (with a timestamp) to stderr and to the server-side log for the current job.

func (*TaskRunner) Perform Uses

func (r *TaskRunner) Perform(action *models.JobAction, taskDir string) error

Perform runs a single script action.

func (*TaskRunner) Run Uses

func (r *TaskRunner) Run() error

Run loops over all of the actions for a particular job, placing files and executing scripts as appropriate. It also arranges for all logging output for the actions to go to the right places.

type TestFunc Uses

type TestFunc func(interface{}) (bool, error)

func AndItems Uses

func AndItems(fs ...TestFunc) TestFunc

func EqualItem Uses

func EqualItem(field string, value interface{}) TestFunc

EqualItem creates a test function to see if a value in the passed interface is equal

func NotItem Uses

func NotItem(f TestFunc) TestFunc

func OrItems Uses

func OrItems(fs ...TestFunc) TestFunc

Package api imports 34 packages (graph) and is imported by 2 packages. Updated 2018-12-17. Refresh now. Tools for package owners.