Documentation ¶
Overview ¶
Package api is responsible for communication between agent and Neptune.io service. This includes the data structures and logic related to agent registration, heartbeating, uploading runbook execution results, uploading agent errors, etc.
Package api is responsible for communication between agent and Neptune.io service. This includes the data structures and logic related to agent registration, heartbeating, uploading runbook execution results, uploading agent errors, etc.
Package util contains the utility code used in Neptune.io agent.
Package config is responsible for parsing the agent config file, command-line params and coming up with the final config.
Package api is responsible for communication between agent and Neptune.io service. This includes the data structures and logic related to agent registration, heartbeating, uploading runbook execution results, uploading agent errors, etc.
Package state contains logic to maintain an event store on Agent to remember the recently processed events. This helps in avoiding the processing of duplicate events, just in case an agent receives duplicate events (which shouldn't happen ideally).
Package executor is responsible for executing runbooks on agent machine. This will have logic to fetch Github runbooks if the agent is configured to use Github.
Package api is responsible for communication between agent and Neptune.io service. This includes the data structures and logic related to agent registration, heartbeating, uploading runbook execution results, uploading agent errors, etc.
Package metadata contains all the meta data used by the agent in registration protocol ¶
Package api is responsible for communication between agent and Neptune.io service. This includes the data structures and logic related to agent registration, heartbeating, uploading runbook execution results, uploading agent errors, etc.
Package worker is responsible for communicating with AWS SQS and handing over the events to executor for runbook execution if the message passes all the checks.
Package api is responsible for communication between agent and Neptune.io service. This includes the data structures and logic related to agent registration, heartbeating, uploading runbook execution results, uploading agent errors, etc.
Package security is responsible for verifying the integrity of received SQS messages before agent processes them. Currently, Neptune.io signs all the messages with a private key and Agent verifies the message signature with public key before processing the message.
Index ¶
- Constants
- Variables
- func Beat(configObj *NeptuneConfig, agentId string) error
- func DeleteMessage(regInfo *RegistrationInfo, receiptHandle *string) error
- func ExecuteAction(event *Event, regInfo *RegistrationInfo, ...) error
- func GetConfig(configFilePath string, cmdlineConfig NeptuneConfig, errorChannel chan error) (NeptuneConfig, AgentConfig, error)
- func HasProcessedEvent(eventId string) bool
- func InitializeEventsFile(dir string)
- func KillCommand(cmd *exec.Cmd)
- func PersistEvent(event *Event) error
- func ReportError(err string)
- func RunLoop(regInfo *RegistrationInfo, regInfoUpdatesCh <-chan string, ...)
- func SendActionOutput(configObj *NeptuneConfig, request *ActionOutputMessage) error
- func SetPGroup(cmd *exec.Cmd)
- func SetRegistrationInfo(reg *RegistrationInfo, metaData HostMetaData, nConfig NeptuneConfig)
- func UpdateStatus(newStatus Status)
- func UploadLogs(configObj *NeptuneConfig, filename string, agentId string) error
- func VerifyMessage(message, signature string) (bool, error)
- type ActionOutputMessage
- type AgentConfig
- type AgentError
- type ConcurrentMap
- func (m ConcurrentMap) Count() int
- func (m ConcurrentMap) Get(key string) (int64, bool)
- func (m ConcurrentMap) GetShard(key string) *ConcurrentMapShared
- func (m *ConcurrentMap) Has(key string) bool
- func (m *ConcurrentMap) IsEmpty() bool
- func (m ConcurrentMap) Iter() <-chan Tuple
- func (m ConcurrentMap) IterBuffered() <-chan Tuple
- func (m ConcurrentMap) MarshalJSON() ([]byte, error)
- func (m *ConcurrentMap) Remove(key string)
- func (m *ConcurrentMap) Set(key string, value int64)
- func (m *ConcurrentMap) UnmarshalJSON(b []byte) (err error)
- type ConcurrentMapShared
- type Config
- type Event
- type Heartbeat
- type HostMetaData
- type NeptuneConfig
- type RegistrationInfo
- type RegistrationRequest
- type Response
- type Status
- type Tuple
- type UploadLogsRequest
Constants ¶
const ( DefaultBaseURL = "www.neptune.io" DefaultConfigFileName = "neptune-agent.json" )
const ( // Agent's current version. // Note: We should improve the versioning logic to avoid hard-coding like this. AgentVersion = "1.1.4" )
Variables ¶
var ErrorsChannel = make(chan string, 10)
Channel to hold agent errors. All components of agents should push errors into this channel and a separate thread uploads them to Neptune.io service currently. In future, we can even log these to syslog so that customers will catch agent issues sooner.
var SHARD_COUNT = 32
Functions ¶
func Beat ¶
func Beat(configObj *NeptuneConfig, agentId string) error
Function to send a heartbeat to Neptune.io service.
func DeleteMessage ¶
func DeleteMessage(regInfo *RegistrationInfo, receiptHandle *string) error
Function to delete SQS message after the processing is done.
func ExecuteAction ¶
func ExecuteAction(event *Event, regInfo *RegistrationInfo, actionOutputs chan<- *ActionOutputMessage, githubKey string) error
Main function to execute runbook based on the given event.
This function does following checks before executing the runbook.
- Using persistent event store, it verifies that the newly received event is not a duplicate.
- Based on the timestamp on event, it checks if the event is not too old.
- If the agent is configured to execute only Github runbooks, it double checks that the event contains Github runbook link and agent configuration has the Github access key.
The event will be discarded and SQS message will be deleted if any of the above checks fail.
func GetConfig ¶
func GetConfig(configFilePath string, cmdlineConfig NeptuneConfig, errorChannel chan error) (NeptuneConfig, AgentConfig, error)
Function to return config objects based on the config file, command-line values, etc.
func HasProcessedEvent ¶
Function to check if the given event id was already processed by this agent or not.
func InitializeEventsFile ¶
func InitializeEventsFile(dir string)
func KillCommand ¶
func PersistEvent ¶
func ReportError ¶
func ReportError(err string)
Function to report an error happened on this agent. This function only pushes the error into a channel.
func RunLoop ¶
func RunLoop(regInfo *RegistrationInfo, regInfoUpdatesCh <-chan string, eventsChannel chan<- *Event, regChannel chan<- time.Time)
Main worker function which does the following things in an infinite loop.
- Poll for SQS messages using long polling technique.
- Check if the messages received are for this agent, by checking agent id. Release the messages not meant for this agent.
- Verify the signature of the message and delete the message immediately if signature isn't correct.
- Deserialize the event from SQS message.
- Re-verify the agent id (which is inside the payload) again just to double check that agent id attribute was not tampered. This guards against replaying old messages, etc.
- At this point, agent has decided to process the event. So, hide the SQS message for the action timeout and hand over the event to executor for runbook execution.
func SendActionOutput ¶
func SendActionOutput(configObj *NeptuneConfig, request *ActionOutputMessage) error
Function to upload runbook execution results to Neptune.io service.
func SetRegistrationInfo ¶
func SetRegistrationInfo(reg *RegistrationInfo, metaData HostMetaData, nConfig NeptuneConfig)
func UpdateStatus ¶
func UpdateStatus(newStatus Status)
Function to update the agent status with the given new status.
func UploadLogs ¶
func UploadLogs(configObj *NeptuneConfig, filename string, agentId string) error
Function to upload agent logs to Neptune.io service.
func VerifyMessage ¶
Function to verify the signature of given message and check if the received signature is same as computed one.
Types ¶
type ActionOutputMessage ¶
type ActionOutputMessage struct { RuleName string `json:"ruleName"` RuleId string `json:"ruleId"` AgentId string `json:"agentId"` EventId string `json:"eventId"` Status string `json:"status"` ActionOutput string `json:"actionOutput"` FailureReason string `json:"failureReason"` StatusCode int `json:"statusCode"` InflightActionId string `json:"inflightActionId"` IsTimeout bool `json:"isTimeout"` HostName string `json:"hostName"` ActionType string `json:"actionType"` }
Message sent by Agent to Neptune.io service to report runbook execution results.
type AgentConfig ¶
type AgentConfig struct { AssignedHostname string LogFile string DebugMode bool GithubApiKey string }
Agent (machine info) section of the config file.
type AgentError ¶
type AgentError struct { ErrorMessage string AgentId string FullLogs bool Hostname string Status string }
Data structure to hold an error that has happened on Agent. All agent errors are sent to Neptune.io service for quick identification of agent problems.
type ConcurrentMap ¶
type ConcurrentMap []*ConcurrentMapShared
A "thread" safe map of type string:int64. To avoid lock bottlenecks this map is dived to several (SHARD_COUNT) map shards.
func (ConcurrentMap) Count ¶
func (m ConcurrentMap) Count() int
Returns the number of elements within the map.
func (ConcurrentMap) Get ¶
func (m ConcurrentMap) Get(key string) (int64, bool)
Retrieves an element from map under given key.
func (ConcurrentMap) GetShard ¶
func (m ConcurrentMap) GetShard(key string) *ConcurrentMapShared
Returns shard under given key
func (*ConcurrentMap) Has ¶
func (m *ConcurrentMap) Has(key string) bool
Looks up an item under specified key
func (ConcurrentMap) Iter ¶
func (m ConcurrentMap) Iter() <-chan Tuple
Returns an iterator which could be used in a for range loop.
func (ConcurrentMap) IterBuffered ¶
func (m ConcurrentMap) IterBuffered() <-chan Tuple
Returns a buffered iterator which could be used in a for range loop.
func (ConcurrentMap) MarshalJSON ¶
func (m ConcurrentMap) MarshalJSON() ([]byte, error)
Reviles ConcurrentMap "private" variables to json marshal.
func (*ConcurrentMap) Remove ¶
func (m *ConcurrentMap) Remove(key string)
Removes an element from the map.
func (*ConcurrentMap) Set ¶
func (m *ConcurrentMap) Set(key string, value int64)
Sets the given value under the specified key.
func (*ConcurrentMap) UnmarshalJSON ¶
func (m *ConcurrentMap) UnmarshalJSON(b []byte) (err error)
type ConcurrentMapShared ¶
type ConcurrentMapShared struct { // contains filtered or unexported fields }
type Config ¶
type Config struct { Neptune NeptuneConfig Agent AgentConfig }
Combine config object for agent. This contains sections for Neptune.io, agent machine data, etc.
type Event ¶
type Event struct { Timestamp int64 `json:"timestamp"` Source string `json:"source"` Hostname string `json:"hostname"` ActionType string `json:"actionType"` EventId string `json:"eventId"` AgentId string `json:"agentId"` RuleId string `json:"ruleId"` RuleName string `json:"ruleName"` InflightActionId string `json:"inflightActionId"` RunbookName string `json:"runbookName"` RawCommand string `json:"rawCommand"` Signature string `json:"signature"` Timeout int32 `json:"timeout"` GithubFilePath string `json:"githubFilePath"` Environment map[string]string `json:"env"` SQSMessageId string ReceiptHandle string }
Event is a type holding the data sent from Neptune.io as a single SQS message. Each event corresponds to one execute runbook request to agent.
type Heartbeat ¶
type Heartbeat struct {
Status string
}
Message sent by Agent to Neptune.io service as a heartbeat.
type HostMetaData ¶
type HostMetaData struct { HostName string AssignedHostname string ProviderId string ProviderType string Platform string PrivateIpAddress string PrivateDnsName string PublicIpAddress string PublicDnsName string Region string }
Host meta data used by the agent in registration protocol.
func GetHostMetaData ¶
func GetHostMetaData(agentConfig *AgentConfig) (HostMetaData, error)
Function to get the complete meta data for the host on which agent is running. This method tries to get cloud specific meta data also, in case the machine is in a cloud.
type NeptuneConfig ¶
Neptune.io section of config file.
type RegistrationInfo ¶
type RegistrationInfo struct { AgentId string CreateTime int64 UpdateTime int64 ActionQueueEndpoint string AWSAccessKey string AWSSecretAccessKey string AWSSecurityToken string }
Message received from Neptune.io service when agent's registration has succeeded.
func RegisterAgent ¶
func RegisterAgent(data HostMetaData, configObj *NeptuneConfig) (*RegistrationInfo, error)
Function to register this agent with Neptune.io service.
type RegistrationRequest ¶
type RegistrationRequest struct { AgentVersion string Hostname string AssignedHostname string ProviderServerId string ProviderServerType string Platform string PrivateIpAddress string PrivateDnsName string PublicIpAddress string PublicDnsName string Region string StartTime int64 }
Message sent by Agent to Neptune.io service to register itself.
type Response ¶
type Response struct {
// contains filtered or unexported fields
}
Response sent by Neptune.io service when agent sends different requests.
type Status ¶
type Status int
Agent status type definition.
func CurrentStatus ¶
func CurrentStatus() Status
Function to get current status of the running agent.
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
Package cmd is responsible for bootstrapping the agent, starting GO routines for different tasks in Neptune.io agent.
|
Package cmd is responsible for bootstrapping the agent, starting GO routines for different tasks in Neptune.io agent. |
neptuneagent
Package cmd/main is the starting point for Neptune.io agent.
|
Package cmd/main is the starting point for Neptune.io agent. |
windows
Package cmd/main is the starting point for Neptune.io's windows agent, which runs as a service.
|
Package cmd/main is the starting point for Neptune.io's windows agent, which runs as a service. |
Package logging is the logging package used by Neptune.io agent.
|
Package logging is the logging package used by Neptune.io agent. |