Documentation ¶
Overview ¶
Package agent is the core Mistify Agent for managing a hypervisor. It runs local on the hypervisor and exposes an HTTP API for managing virtual machines.
General Architecture ¶
Mistify Agent provides a REST-ish API for manipulating a hypervisor and defines a general set of actions. It is primarilly an interface to the sub-agents, which do most of the work. Sub-agents are assumed to be running on the same host as the agent and communication is done via JSON-RPC over HTTP.
The data structures for the REST API are defined in: http://godoc.org/github.com/mistifyio/mistify-agent/client
The data structures for the RPC API are defined in: http://godoc.org/github.com/mistifyio/mistify-agent/rpc
Sub-agents do not have to be written in Go, but the agent does provide helpers for easily creating them. All of the official sub-agents are written in Go.
There are two types of sub-agents:
* images/storage - Mistify will provide an opinionated version that uses ZFS ZVOLs for guest disks.
* guest "actions" - create/delete/reboot/etc.
Only one storage sub-agent is used, but multiple guest sub-agents may be used.
Actions ¶
While there is a defined set of actions, the work performed is configurable. Each action has a pipeline, a series of one or more steps that need to be performed to complete the action, configurable in the config file. All steps must succeed, in order, for an action to be considered successful.
There are three action types:
* Info - Information retrieval actions, such as getting a list of guests, called synchronously at request time. A JSON result is returned to the requesting client when the pipeline is complete.
* Async - Modification actions, such as rebooting a guest, called asynchronously. One action per guest is performed at a time, while the rest are queued. A response containing the job id in the header X-Guest-Job-ID is returned after queueing the action, which can be used to check the status at a later time.
* Stream - Data retrieval, such as downloading a zfs snapshot, called synchronously at request time. Rather than a JSON response, data is streamed back in chunks.
Valid actions are defined in: http://godoc.org/github.com/mistifyio/mistify-agent/config
HTTP API Endpoints
/debug/pprof * GET /debug/pprof/{profileName} * GET /debug/pprof/cmdline * GET /debug/pprof/symbol * GET /debug/pprof/symbol * GET /metadata * GET - Retrieve the hypervisor's metadata * PATCH - Modify the hypervisor's metadata /images * GET - Retrieve a list of disk images * POST - Fetch a disk image /images/{imageID} * GET - Retrieve information about a disk image * DELETE - Delete a disk image /container_images * GET - Retrieve a list of container images * POST - Fetch a container image /container_images/{imageID} * GET - Retrieve information about a container image * DELETE - Delete a container image /guests * GET - Retrieve a list of guests * POST - Create a new guest /guests/{guestID} * GET - Retrieve information about a guest /guests/{guestID}/jobs * GET - Retrieve a list of recent action jobs for the guest /guests/{guestID}/jobs/{jobID} * GET - Retrieve information about a specific action job /guests/{guestID}/metadata * GET - Retrieve a guest's metadata * PATCH - Modify the guest's metadata /guests/{guestID}/metrics/cpu * GET - Retrieve guest CPU metrics /guests/{guestID}/metrics/disk * GET - Retrieve guest disk metrics /guests/{guestID}/metrics/nic * GET - Retrieve guest NIC metrics /guests/{guestID}/{actionName} Actions: shutdown, reboot, restart, poweroff, start, suspend, delete * GET - Perform the specified action for the guest /guests/{guestID}/snapshots /guests/{guestID}/disks/{diskID}/snapshots * GET - Retrieve a list of snapshots * POST - Create a new snapshot /guests/{guestID}/snapshots/{snapshotName} /guests/{guestID}/disks/{diskID}/snapshots/{snapshotName} * GET - Retrieve information about a snapshot * DELETE - Delete a snapshot /guests/{guestID}/snapshots/{snapshotName}/rollback /guests/{guestID}/disks/{diskID}/snapshots/{snapshotName}/rollback * POST - Roll back to the snapshot /guests/{guestID}/snapshots/{snapshotName}/download /guests/{guestID}/disks/{diskID}/snapshots/{snapshotName}/download * GET - Download the snapshot
Contributing ¶
See the guidelines: https://github.com/mistifyio/mistify-agent/blob/master/CONTRIBUTING.md
Index ¶
- Constants
- Variables
- func AttachProfiler(router *mux.Router)
- func LogRunnerError(guestID string, runnerName string, pipelineID string, logLine string)
- func LogRunnerInfo(guestID string, runnerName string, pipelineID string, logLine string)
- func Run(ctx *Context, address string) error
- type Action
- type Context
- func (ctx *Context) CreateJobLog() error
- func (ctx *Context) DeleteGuest(g *client.Guest) error
- func (context *Context) DeleteGuestRunner(guestID string)
- func (ctx *Context) GetAction(name string) (*Action, error)
- func (context *Context) GetAgentRunner() (*GuestRunner, error)
- func (ctx *Context) GetGuest(id string) (*client.Guest, error)
- func (context *Context) GetGuestRunner(guestID string) (*GuestRunner, error)
- func (ctx *Context) GetJobLog() (*JobLog, error)
- func (context *Context) NewGuestRunner(guestID string, maxInfo uint, maxStream uint) *GuestRunner
- func (ctx *Context) NewService(name string, port uint, path string, maxConcurrent uint) (*Service, error)
- func (ctx *Context) PersistGuest(g *client.Guest) error
- func (ctx *Context) RunGuests() error
- type Guest
- type GuestRunner
- type HTTPError
- type HTTPResponse
- type Job
- type JobLog
- func (jobLog *JobLog) AddJob(jobID, guestID, action string) error
- func (jobLog *JobLog) GetJob(jobID string) (*Job, error)
- func (jobLog *JobLog) GetLatestGuestJobs(guestID string, limit int) []*Job
- func (jobLog *JobLog) GetLatestJobs(limit int) []*Job
- func (jobLog *JobLog) Persist() error
- func (jobLog *JobLog) UpdateJob(jobID string, action string, status JobStatus, message string) error
- type JobStatus
- type Pipeline
- type PipelineQueue
- type Service
- type Stage
- type SyncThrottle
Constants ¶
const ( // MaxLoggedJobs configures how many jobs to prune the log to MaxLoggedJobs int = 1000 // Queued is the queued job status Queued JobStatus = "Queued" // Running is the running job status Running JobStatus = "Running" // Complete is the complete job status Complete JobStatus = "Complete" // Errored is the errored job status Errored JobStatus = "Error" )
Variables ¶
var ( // ErrNotFound is the error for a resouce not found ErrNotFound = errors.New("not found") )
Functions ¶
func AttachProfiler ¶
AttachProfiler enables debug profiling exposed on http api endpoints
func LogRunnerError ¶
LogRunnerError writes error logs
func LogRunnerInfo ¶
LogRunnerInfo writes informational logs
Types ¶
type Action ¶
type Action struct { Name string Type config.ActionType Stages []*Stage }
Action is a full set of stage templates required to complete an action
func (*Action) GeneratePipeline ¶
func (action *Action) GeneratePipeline(request interface{}, response interface{}, rw http.ResponseWriter, done chan error) *Pipeline
GeneratePipeline creates an instance of Pipeline based on an action's stages and supplied request & response. It is returned so that any additional modifications (such as adding stage args to requests) can be made before running if needed.
type Context ¶
type Context struct { Config *config.Config Actions map[string]*Action Services map[string]*Service GuestRunners map[string]*GuestRunner GuestRunnerMutex sync.Mutex JobLog *JobLog // contains filtered or unexported fields }
Context is the core of the Agent.
func NewContext ¶
NewContext creates a new context. In general, there should only be one.
func (*Context) CreateJobLog ¶
CreateJobLog creates a new job log
func (*Context) DeleteGuest ¶
DeleteGuest removes a guest from the data store
func (*Context) DeleteGuestRunner ¶
DeleteGuestRunner deletes a GuestRunner
func (*Context) GetAgentRunner ¶
func (context *Context) GetAgentRunner() (*GuestRunner, error)
GetAgentRunner retrieves the main agent runner
func (*Context) GetGuestRunner ¶
func (context *Context) GetGuestRunner(guestID string) (*GuestRunner, error)
GetGuestRunner retrieves a GuestRunner
func (*Context) NewGuestRunner ¶
func (context *Context) NewGuestRunner(guestID string, maxInfo uint, maxStream uint) *GuestRunner
NewGuestRunner creates a new GuestRunner
func (*Context) NewService ¶
func (ctx *Context) NewService(name string, port uint, path string, maxConcurrent uint) (*Service, error)
NewService creates a new Service
func (*Context) PersistGuest ¶
PersistGuest writes guest data to the data store
type GuestRunner ¶
type GuestRunner struct { Context *Context GuestID string Info *SyncThrottle Stream *SyncThrottle Async *PipelineQueue QuitChan chan struct{} }
GuestRunner manages actions being performed for a guest
func (*GuestRunner) Process ¶
func (gr *GuestRunner) Process(pipeline *Pipeline) error
Process directs actions into sync or async handling depending on the type
type HTTPError ¶
type HTTPError struct { Message string `json:"message"` Code int `json:"code"` Stack []string `json:"stack"` }
HTTPError is an enhanced error struct for http error responses
func NewHTTPError ¶
NewHTTPError prepares an HTTPError with a stack trace
type HTTPResponse ¶
type HTTPResponse struct {
http.ResponseWriter
}
HTTPResponse is a wrapper for http.ResponseWriter which provides access to several convenience methods
func (*HTTPResponse) JSON ¶
func (hr *HTTPResponse) JSON(code int, obj interface{})
JSON writes appropriate headers and JSON body to the http response
func (*HTTPResponse) JSONError ¶
func (hr *HTTPResponse) JSONError(code int, err error)
JSONError prepares an HTTPError with a stack trace and writes it with HTTPResponse.JSON
func (*HTTPResponse) JSONMsg ¶
func (hr *HTTPResponse) JSONMsg(code int, msg string)
JSONMsg is a convenience method to write a JSON response with just a message string
type Job ¶
type Job struct { ID string GuestID string Action string QueuedAt time.Time StartedAt time.Time UpdatedAt time.Time Status JobStatus Message string }
Job holds information about a job
type JobLog ¶
type JobLog struct { GuestID string Context *Context ModifyMutex sync.RWMutex Index map[string]int GuestIndex map[string][]int Jobs []*Job }
JobLog holds the most recent jobs for a guest
func (*JobLog) GetLatestGuestJobs ¶
GetLatestGuestJobs returns the latest X jobs in the log for a guest
func (*JobLog) GetLatestJobs ¶
GetLatestJobs returns the latest X jobs in the log
type Pipeline ¶
type Pipeline struct { ID string Action string Type config.ActionType Stages []*Stage PreStageFunc func(*Pipeline, *Stage) error PostStageFunc func(*Pipeline, *Stage) error DoneChan chan error // Signal async is done or errored, for post-hooks }
Pipeline is a full set of stage instances required to complete an action
type PipelineQueue ¶
type PipelineQueue struct { GuestID string Name string Context *Context PipelineChan chan *Pipeline QuitChan chan struct{} }
PipelineQueue holds asyncronous action pipelines
func NewPipelineQueue ¶
func NewPipelineQueue(name string, guestID string, context *Context) *PipelineQueue
NewPipelineQueue creates a new PipelineQueue
func (*PipelineQueue) Enqueue ¶
func (pq *PipelineQueue) Enqueue(pipeline *Pipeline)
Enqueue queues an async action
func (*PipelineQueue) Process ¶
func (pq *PipelineQueue) Process()
Process monitors the queue and kicks off async actions
func (*PipelineQueue) Quit ¶
func (pq *PipelineQueue) Quit()
Quit signals the pipeline queue to stop processing after the current action
type Stage ¶
type Stage struct { Service *Service Type config.ActionType Method string Args map[string]string Request interface{} Response interface{} RW http.ResponseWriter // For streaming responses }
Stage is a single step an action must take
type SyncThrottle ¶
type SyncThrottle struct { GuestID string Name string ConcurrentChan chan struct{} QuitChan chan struct{} }
SyncThrottle throttles synchronous actions
func NewSyncThrottle ¶
func NewSyncThrottle(name string, guestID string, maxConcurrency uint) *SyncThrottle
NewSyncThrottle creates a new SyncThrottle
func (*SyncThrottle) Process ¶
func (st *SyncThrottle) Process(pipeline *Pipeline) error
Process runs an action
func (*SyncThrottle) Release ¶
func (st *SyncThrottle) Release()
Release signals that the action is done
func (*SyncThrottle) Reserve ¶
func (st *SyncThrottle) Reserve()
Reserve blocks until an action is allowed to run based on throttling
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
Package client provides a simple client for the Mistify Agent.
|
Package client provides a simple client for the Mistify Agent. |
cmd
|
|
mistify-agent
mistify-agent runs the agent and HTTP API.
|
mistify-agent runs the agent and HTTP API. |
Package config handles agent configuration from a config file.
|
Package config handles agent configuration from a config file. |
examples
|
|
simple-subagent
simple-subagent is a simple example of a sub-agent for Mistify Agent.
|
simple-subagent is a simple example of a sub-agent for Mistify Agent. |
test-rpc-service
test-rpc-service is a test Mistify sub-agent that implements stubs of all the actions the agent may call.
|
test-rpc-service is a test Mistify sub-agent that implements stubs of all the actions the agent may call. |
webhook-post
webhook-post is an example sub-agent that posts to a webhook.
|
webhook-post is an example sub-agent that posts to a webhook. |
Package rpc defines the JSON-RPC over HTTP types the Agent uses to communicate with sub-agents.
|
Package rpc defines the JSON-RPC over HTTP types the Agent uses to communicate with sub-agents. |