agent

package
v0.0.0-...-900fa13 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 11, 2015 License: Apache-2.0 Imports: 27 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// DefaultCmdTimeout specifies the duration after which agent sends
	// an IdleTimeout signal if a task's command does not run to completion.
	DefaultCmdTimeout = 2 * time.Hour
	// DefaultIdleTimeout specifies the duration after which agent sends an
	// IdleTimeout signal if a task produces no logs.
	DefaultIdleTimeout = 15 * time.Minute
	// DefaultHeartbeatInterval is interval after which agent sends a heartbeat
	// to API server.
	DefaultHeartbeatInterval = 30 * time.Second
	// DefaultStatsInterval is the interval after which agent sends system stats
	// to API server
	DefaultStatsInterval = 60 * time.Second
)
View Source 
const APIVersion = 2

Variables

View Source 
var (
	// InitialSetupTimeout indicates the time allowed for the agent to collect
	// relevant information - for running a task - from the API server.
	InitialSetupTimeout = 5 * time.Minute
	// InitialSetupCommand is a placeholder command for the period during which
	// the agent requests information for running a task
	InitialSetupCommand = model.PluginCommandConf{
		DisplayName: "initial task setup",
		Type:        model.SystemCommandType,
	}
)
View Source 
var HTTPConflictError = errors.New("Conflict")
View Source 
var InterruptedCmdError = errors.New("Command interrupted")

InterruptedCmdError is returned by commands that were stopped before they could complete.

Functions

This section is empty.

Types

type APILogger 

type APILogger struct {

	// The number of log lines that the buffer must reach to trigger a flush
	SendAfterLines int

	// How long to wait without any flushes before triggering one automatically
	SendAfterDuration time.Duration

	// The mechanism for communicating with the remote endpoint.
	TaskCommunicator
	// contains filtered or unexported fields
}

APILogger is a slogger.Appender which makes a call to the remote service's log endpoint after SendAfterLines messages have been received (or if set, after SendAfterDuration time has passed with no flush).

func NewAPILogger 

func NewAPILogger(tc TaskCommunicator) *APILogger

NewAPILogger creates an initialized logger around the given TaskCommunicator.

func (*APILogger) Append 

func (apiLgr *APILogger) Append(log *slogger.Log) error

Append (to satisfy the Appender interface) adds a log message to the internal buffer, and translates the log message into a format that is used by the remote endpoint.

func (*APILogger) Flush 

func (apiLgr *APILogger) Flush()

func (*APILogger) FlushAndWait 

func (apiLgr *APILogger) FlushAndWait() int

type Agent 

type Agent struct {

	// TaskCommunicator handles all communication with the API server -
	// marking task started/ended, sending test results, logs, heartbeats, etc
	TaskCommunicator

	// ExecTracker keeps track of the agent's current stage of execution.
	ExecTracker

	// APILogger is a slogger.Appender which sends log messages
	// to the API server.
	APILogger *APILogger

	// Registry manages plugins available for the agent.
	Registry plugin.Registry
	// contains filtered or unexported fields
}

Agent controls the various components and background processes needed throughout the lifetime of the execution of the task.

func New 

func New(apiServerURL, taskId, taskSecret, logFile, cert string) (*Agent, error)

New creates a new agent to run a given task.

func (*Agent) CheckIn 

func (agt *Agent) CheckIn(command model.PluginCommandConf, duration time.Duration)

CheckIn updates the agent's execution stage and current timeout duration, and resets its timer back to zero.

func (*Agent) GetCurrentCommand 

func (agt *Agent) GetCurrentCommand() model.PluginCommandConf

GetCurrentCommand returns the current command being executed by the agent.

func (*Agent) GetTaskConfig 

func (agt *Agent) GetTaskConfig() (*model.TaskConfig, error)

GetTaskConfig fetches task configuration data required to run the task from the API server.

func (*Agent) RunCommands 

func (agt *Agent) RunCommands(commands []model.PluginCommandConf, returnOnError bool, stop chan bool) error

RunCommands takes a slice of commands and executes then sequentially. If returnOnError is set, it returns immediately if one of the commands fails. All plugins listen on the stop channel and must terminate immediately when a value is received.

func (*Agent) RunTask 

func (agt *Agent) RunTask() (*apimodels.TaskEndResponse, error)

RunTask manages the process of running a task. It returns a response indicating the end result of the task.

func (*Agent) RunTaskCommands 

func (agt *Agent) RunTaskCommands(completed chan FinalTaskFunc) (*apimodels.TaskEndResponse, error)

RunTaskCommands runs all commands for the task currently assigend to the agent.

func (*Agent) StartBackgroundActions 

func (agt *Agent) StartBackgroundActions(signalHandler TerminateHandler) chan FinalTaskFunc

StartBackgroundActions spawns goroutines that monitor various parts of the execution - heartbeats, timeouts, logging, etc.

type AgentCommand 

type AgentCommand struct {
	*StreamLogger
	ScriptLine string
	Expansions *command.Expansions
	KillChan   chan bool
}

AgentCommand encapsulates a running local command and streams logs back to the API server.

func (*AgentCommand) Run 

func (ac *AgentCommand) Run(workingDir string) error

Run will execute the command in workingDir, by applying the expansions to the script and then invoking it with sh -c, and logging all of the command's stdout/stderr using the Logger. It will block until the command either finishes, or is aborted prematurely via the kill channel.

type CommandLogger 

type CommandLogger struct {
	// contains filtered or unexported fields
}

Wraps an Logger, with additional context about which command is currently being run.

func (*CommandLogger) Flush 

func (cmdLgr *CommandLogger) Flush()

func (*CommandLogger) GetTaskLogWriter 

func (cmdLgr *CommandLogger) GetTaskLogWriter(level slogger.Level) io.Writer

func (*CommandLogger) LogExecution 

func (cmdLgr *CommandLogger) LogExecution(level slogger.Level, messageFmt string, args ...interface{})

func (*CommandLogger) LogLocal 

func (cmdLgr *CommandLogger) LogLocal(level slogger.Level, messageFmt string, args ...interface{})

func (*CommandLogger) LogSystem 

func (cmdLgr *CommandLogger) LogSystem(level slogger.Level, messageFmt string, args ...interface{})

func (*CommandLogger) LogTask 

func (cmdLgr *CommandLogger) LogTask(level slogger.Level, messageFmt string, args ...interface{})

type ExecTracker 

type ExecTracker interface {
	// Returns the current command being executed.
	CurrentCommand() *model.PluginCommandConf
	// Sets the current command being executed as well as a timeout for the command.
	CheckIn(command model.PluginCommandConf, timeout time.Duration)
}

ExecTracker exposes functions to update and get the current execution stage of the agent.

type FinalTaskFunc 

type FinalTaskFunc func() (*apimodels.TaskEndResponse, error)

The FinalTaskFunc describes the expected return values for a given task run by the agent. The finishAndAwaitCleanup listens on a channel for this function and runs returns its values once it receives the function to run. This will typically be an HTTP call to the API server (to end the task).

type HTTPCommunicator 

type HTTPCommunicator struct {
	ServerURLRoot string
	TaskId        string
	TaskSecret    string
	MaxAttempts   int
	RetrySleep    time.Duration
	SignalChan    chan Signal
	Logger        *slogger.Logger
	HttpsCert     string
	// contains filtered or unexported fields
}

HTTPCommunicator handles communication with the API server. An HTTPCommunicator is scoped to a single task, and all communication performed by it is only relevant to that running task.

func NewHTTPCommunicator 

func NewHTTPCommunicator(serverURL, taskId, taskSecret, cert string, sigChan chan Signal) (*HTTPCommunicator, error)

NewHTTPCommunicator returns an initialized HTTPCommunicator. The cert parameter may be blank if default system certificates are being used.

func (*HTTPCommunicator) End 

func (h *HTTPCommunicator) End(detail *apimodels.TaskEndDetail) (*apimodels.TaskEndResponse, error)

End marks the communicator's task as finished with the given status.

func (*HTTPCommunicator) FetchExpansionVars 

func (h *HTTPCommunicator) FetchExpansionVars() (*apimodels.ExpansionVars, error)

FetchExpansionVars loads expansions for a communicator's task from the API server.

func (*HTTPCommunicator) GetDistro 

func (h *HTTPCommunicator) GetDistro() (*distro.Distro, error)

GetDistro returns the distro for the communicator's task.

func (*HTTPCommunicator) GetPatch 

func (h *HTTPCommunicator) GetPatch() (*patch.Patch, error)

GetPatch loads the task's patch diff from the API server.

func (*HTTPCommunicator) GetProjectConfig 

func (h *HTTPCommunicator) GetProjectConfig() (*model.Project, error)

GetProjectConfig loads the communicator's task's project from the API server.

func (*HTTPCommunicator) GetProjectRef 

func (h *HTTPCommunicator) GetProjectRef() (*model.ProjectRef, error)

GetProjectConfig loads the communicator's task's project from the API server.

func (*HTTPCommunicator) GetTask 

func (h *HTTPCommunicator) GetTask() (*model.Task, error)

GetTask returns the communicator's task.

func (*HTTPCommunicator) Heartbeat 

func (h *HTTPCommunicator) Heartbeat() (bool, error)

Heartbeat sends a heartbeat to the API server. The server can respond with and "abort" response. This function returns true if the agent should abort.

func (*HTTPCommunicator) Log 

func (h *HTTPCommunicator) Log(messages []model.LogMessage) error

Log sends a batch of log messages for the task's logs to the API server.

func (*HTTPCommunicator) Start 

func (h *HTTPCommunicator) Start(pid string) error

Start marks the communicator's task as started.

type Heartbeat 

type Heartbeat interface {
	Heartbeat() (bool, error)
}

Heartbeat encapsulates heartbeat behavior (i.e., pinging the API server at regular intervals to ensure that communication hasn't broken down).

type HeartbeatTicker 

type HeartbeatTicker struct {
	// Number of consecutive failed heartbeats allowed before signaling a failure
	MaxFailedHeartbeats int

	// Period of time to wait between heartbeat attempts
	Interval time.Duration

	// Channel on which to notify of failed heartbeats or aborted task
	SignalChan chan Signal

	// Interface which handles sending the actual heartbeat over the network
	TaskCommunicator

	Logger *slogger.Logger
	// contains filtered or unexported fields
}

HeartbeatTicker manages heartbeat communication with the API server

func (*HeartbeatTicker) StartHeartbeating 

func (hbt *HeartbeatTicker) StartHeartbeating()

func (*HeartbeatTicker) Stop 

func (hbt *HeartbeatTicker) Stop()

type Signal 

type Signal int64

Signal describes the various conditions under which the agent will complete execution of a task. 

const (
	// HeartbeatMaxFailed indicates that repeated attempts to send heartbeat to
	// the API server fails.
	HeartbeatMaxFailed Signal = iota
	// IncorrectSecret indicates that the secret for the task the agent is running
	// does not match the task secret held by API server.
	IncorrectSecret
	// AbortedByUser indicates a user decided to prematurely end the task.
	AbortedByUser
	// IdleTimeout indicates the task appears to be idle - e.g. no logs produced
	// for the duration indicated by DefaultIdleTimeout.
	IdleTimeout
	// CompletedSuccess indicates task successfully ran to completion and passed.
	CompletedSuccess
	// CompletedFailure indicates task successfully ran to completion but failed.
	CompletedFailure
)

Recognized agent signals.

type SignalHandler 

type SignalHandler struct {
	// KillChan is a channel which once closed, causes any in-progress commands to abort.
	KillChan chan bool
	// Post is a set of commands to run after an agent completes a task execution.
	Post *model.YAMLCommandSet
	// Timeout is a set of commands to run if/when an IdleTimeout signal is received.
	Timeout *model.YAMLCommandSet
	// contains filtered or unexported fields
}

SignalHandler is an implementation of TerminateHandler which runs the post-run script when a task finishes, and reports its results back to the API server.

func (*SignalHandler) HandleSignals 

func (sh *SignalHandler) HandleSignals(agt *Agent, completed chan FinalTaskFunc)

HandleSignals listens on its signal channel and properly handles any signal received.

type StatsCollector 

type StatsCollector struct {
	Cmds []string
	// indicates the sampling frequency
	Interval time.Duration
	// contains filtered or unexported fields
}

StatsCollector samples machine statistics and logs them back to the API server at regular intervals.

func NewSimpleStatsCollector 

func NewSimpleStatsCollector(logger *slogger.Logger, interval time.Duration, cmds ...string) *StatsCollector

NewSimpleStatsCollector creates a StatsCollector that runs the given commands at the given interval and sends the results to the given logger.

func (*StatsCollector) LogStats 

func (sc *StatsCollector) LogStats(exp *command.Expansions)

func (*StatsCollector) Stop 

func (sc *StatsCollector) Stop()

type StreamLogger 

type StreamLogger struct {
	// Local is used for file system logging on the host the agent is running on.
	Local *slogger.Logger
	// System is used for logging system stats gotten from commands like df, ps, etc.
	System *slogger.Logger
	// Task is used for logging command input, output and errors of the task.
	Task *slogger.Logger
	// Execution is used for logging the agent's internal state.
	Execution *slogger.Logger
	// contains filtered or unexported fields
}

StreamLogger holds a set of stream-delineated loggers. Each logger is used to communicate different kinds of logs to the API Server or local file system. StreamLogger is used to distinguish logs of system statistics, shell output, and internal agent logs.

func NewStreamLogger 

func NewStreamLogger(timeoutWatcher *TimeoutWatcher, apiLgr *APILogger, logFile string) (*StreamLogger, error)

NewStreamLogger creates a StreamLogger wrapper for the apiLogger with a given timeoutWatcher. Any logged messages on the StreamLogger will reset the TimeoutWatcher.

func NewTestLogger 

func NewTestLogger(appender slogger.Appender) *StreamLogger

NewTestLogger creates a logger for testing. This Logger stores everything in memory.

func (*StreamLogger) Flush 

func (lgr *StreamLogger) Flush()

Flush flushes the logs to the server. Returns immediately.

func (*StreamLogger) FlushAndWait 

func (lgr *StreamLogger) FlushAndWait() int

FlushAndWait flushes and blocks until the HTTP request to send the logs has completed. This works in contrast with Flush, which triggers the flush asynchronously.

func (*StreamLogger) GetTaskLogWriter 

func (lgr *StreamLogger) GetTaskLogWriter(level slogger.Level) io.Writer

GetTaskLogWriter returns an io.Writer of the given level. Useful for working with other libraries seamlessly.

func (*StreamLogger) LogExecution 

func (lgr *StreamLogger) LogExecution(level slogger.Level, messageFmt string, args ...interface{})

LogExecution logs a message related to the agent's internal workings.

Internally this is used to log things like heartbeats and command internals that would pollute the regular task test output.

func (*StreamLogger) LogLocal 

func (lgr *StreamLogger) LogLocal(level slogger.Level, messageFmt string, args ...interface{})

LogLocal logs a message to the agent logs on the machine's local file system.

Anything logged by this method will not be sent to the server, so only use it to log information that would only be useful when debugging locally.

func (*StreamLogger) LogSystem 

func (lgr *StreamLogger) LogSystem(level slogger.Level, messageFmt string, args ...interface{})

LogSystem logs passive system messages.

Internally this is used for periodically logging process information and CPU usage.

func (*StreamLogger) LogTask 

func (lgr *StreamLogger) LogTask(level slogger.Level, messageFmt string, args ...interface{})

LogTask logs a message to the task's logs.

This log type is for main task input and output. LogTask should be used for logging first-class information like test results and shell script output.

type TaskCommunicator 

type TaskCommunicator interface {
	Start(pid string) error
	End(detail *apimodels.TaskEndDetail) (*apimodels.TaskEndResponse, error)
	GetTask() (*model.Task, error)
	GetProjectRef() (*model.ProjectRef, error)
	GetDistro() (*distro.Distro, error)
	GetProjectConfig() (*model.Project, error)
	GetPatch() (*patch.Patch, error)
	Log([]model.LogMessage) error
	Heartbeat() (bool, error)
	FetchExpansionVars() (*apimodels.ExpansionVars, error)
	// contains filtered or unexported methods
}

TaskCommunicator is an interface that handles the remote procedure calls between an agent and the remote server.

type TaskJSONCommunicator 

type TaskJSONCommunicator struct {
	PluginName string
	TaskCommunicator
}

TaskJSONCommunicator handles plugin-specific JSON-encoded communication with the API server.

func (*TaskJSONCommunicator) PostTaskFiles 

func (t *TaskJSONCommunicator) PostTaskFiles(task_files []*artifact.File) error

PostTaskFiles is used by the PluginCommunicator interface for attaching task files.

func (*TaskJSONCommunicator) TaskGetJSON 

func (t *TaskJSONCommunicator) TaskGetJSON(endpoint string) (*http.Response, error)

TaskGetJSON does an HTTP GET for the communicator's plugin + task.

func (*TaskJSONCommunicator) TaskPostJSON 

func (t *TaskJSONCommunicator) TaskPostJSON(endpoint string, data interface{}) (*http.Response, error)

TaskPostJSON does an HTTP POST for the communicator's plugin + task.

func (*TaskJSONCommunicator) TaskPostResults 

func (t *TaskJSONCommunicator) TaskPostResults(results *model.TestResults) error

TaskPostResults posts a set of test results for the communicator's task.

func (*TaskJSONCommunicator) TaskPostTestLog 

func (t *TaskJSONCommunicator) TaskPostTestLog(log *model.TestLog) (string, error)

TaskPostTestLog posts a test log for a communicator's task.

type TerminateHandler 

type TerminateHandler interface {
	HandleSignals(*Agent, chan FinalTaskFunc)
}

TerminateHandler is an interface which defines how the agent should respond to signals resulting in the end of the task (heartbeat fail, timeout, etc)

type TimeoutResetLogger 

type TimeoutResetLogger struct {
	*TimeoutWatcher
	slogger.Appender
}

TimeoutResetLogger wraps any slogger.Appender and resets a TimeoutWatcher each time any log message is appended to it.

func (*TimeoutResetLogger) Append 

func (trLgr *TimeoutResetLogger) Append(log *slogger.Log) error

Append passes the message to the underlying appender, and resets the timeout

type TimeoutWatcher 

type TimeoutWatcher struct {
	// contains filtered or unexported fields
}

TimeoutWatcher tracks and handles command timeout within the agent.

func (*TimeoutWatcher) CheckIn 

func (tw *TimeoutWatcher) CheckIn()

CheckIn resets the idle timer to zero.

func (*TimeoutWatcher) NotifyTimeouts 

func (tw *TimeoutWatcher) NotifyTimeouts(sigChan chan Signal)

NotifyTimeouts sends a signal on sigChan whenever the timeout threshold of the current execution stage is reached.

func (*TimeoutWatcher) SetDuration 

func (tw *TimeoutWatcher) SetDuration(duration time.Duration)

SetDuration sets the duration after which a timeout is triggered.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.