orca

package module
v0.0.0-...-ac339c7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 21, 2016 License: MIT Imports: 21 Imported by: 0

README

Orca

Build Status Coverage Status GoDoc Reference Go Report Card Stories in Ready

*Echolocation of device with static nodes and network latency.

Orca

Orca is a reflector/ping generator service that is intended to measure network latency of gRPC requests from mobile devices (laptops) to storage locations in home and work networks. Orca was built to get some baseline metrics for latencies in distributed storage systems and is purely experimental, not intended for large scale use.

Orca specifies two primary commands: reflect and generate. The reflector is a Protocol Buffers service that accepts echo requests and returns replies. The generator is a service that sends a request to all known reflectors on a routine interval and logs the latency in a SQLite database.

Installation

In order to install Orca and get it running on your system, you have two options: build from source or download a prebuilt binary that may or may not exist for your system.

To download the prebuilt binary, visit the Current Release on GitHub and download the executable for your system. Extract the archive to reveal the executable, and put it on your path, making sure it has executable permissions.

Note: currently only Linux x64 and OS X (darwin) x64 binaries are prebuilt. My preference for path location is $HOME/bin but make sure that location is in your path. Otherwise, /usr/local/bin is a good choice for moving the binary to.

To build from source, make sure you have your Go environment setup and run:

$ go get github.com/bbengfort/orca/...

You can then use go install to install the binary to $GOBIN as follows:

$ go install cmd/orca.go

To contribute or modify the orca command, you'll want to go the source route; otherwise if you are using Linux or OS X, I recommend going the binary route. If there is an architecture that you'd like prebuilt, please let me know in a GitHub issue.

Getting Started

Once installed, some configuration is necessary. This is the recommended configuration, though other configuration options are available and can be inspected via the orca help command.

  1. Create the directory where configuration and data will reside:

     $ mkdir ~/.orca
    
  2. Copy the example configuration file to the orca directory

     $ cp fixtures/orca-config-example.yml ~/.orca/config.yml
    

    Note that the first path refers to the fixtures directory inside of the GitHub repository. If you downloaded the binary package from the releases page, then the example configuration will be in the archive you downloaded.

  3. Edit the configuration as needed, noting the defaults and comments in the example configuration file.

  4. Create the orca SQLite database used to track meta information about reflection and generators.

     $ orca createdb
    

At this point orca is configured to begin reflecting, however generators require an extra configuration step.

Relectors

Running orca as a reflector runs a gRPC service that listens for echo requests on the current IP address and port, increments the request sequence (to detect out of order messages), and responds with an echo reply. This process means that there is a small database transaction before reply, so latency will not be exactly the same as using the ping utility. If debug: true in the configuration, then the server will also log all incoming echo requests to stdout, which can be redirected to a log file if required.

Run the reflector as follows:

$ orca reflect

To run in the background on a constant server:

$ nohup orca reflect &

Upstart and LaunchAgent scripts for managing the background process are forthcoming, though I'd be happy to accept a pull request for them!

Generators

Generators require a bit more configuration, since you'll have to add all of the reflectors that you want the generator to ping to the database. Do this via the orca devices --add command:

$ orca devices --add
Enter device name []: rogue
Enter device IP address []: 1.2.3.4:3265
Enter device domain []:

Do this for as many devices as you'd like to ping on each interval. To see the devices already added use orca devices --list. Run the generator as follows:

$ orca generate

Similar to the reflector, you'll have to nohup and background this in order to ensure it always runs. LaunchAgent and Upstart scripts are coming soon. The generator waits until the interval has passed, loads up the list of devices to ping, and sends an echo request to them, recording the request (in the case of non-connectivity) and sequence number in the database. On receipt of the reply, it measures latency and stores the information in the database.

Location Servicesd

Orca can provide location services for mobile devices via the MaxMind GeoIP2 Precision City Service. In order to enable location services, you need to register for a MaxMind developer account and include your API user id and license key in the YAML configuration file. Because MaxMind is a paid service, location lookups are only made when the current IP address of the machine changes.

Note also that the granularity for this service is limited; for example, a GeoIP2 lookup from my office in the A.V. Williams Building of the University of Maryland yielded the following location via latitude and longitude:

Map Granularity

This location is not centered on my office, the building, or even in the center of the university. The level of granularity probably differs based on the type of network you're connected to. If a higher level of granularity is required, then the use of GPS is recommended. Additional location inaccuracy can come about when tethering to a mobile phone. Use specified locations with care!

Acknowledgements

Orca is an open source project built to obtain metrics about mobile distributed systems and various latencies. If you'd like to contribute, I'd love some help, but no current plans are underway for future development.

Attribution

The photo used in this README, “Orca (Design Exploration)” by Alberto Cerriteño is used under a CC BY-NC-ND 2.0 creative commons license.

Documentation

Overview

Package orca provides the library for a systems experiment that measures latency and uptime of mobile nodes against fixed responder nodes.

Index

Constants

View Source
const DefaultPort = 3265

DefaultPort is used to compute the TCP address in the absense of one.

View Source
const Timeout = time.Duration(30) * time.Second

Timeout is the amount of time sonar will wait for a reply

View Source
const Version = "0.1"

Version specifies the current version of the Orca library.

Variables

View Source
var ConfigPath = []string{
	"/etc/orca.yml",
	filepath.Join(getUserDir(), ".orca", "config.yml"),
	filepath.Join(getCwd(), "orca.yml"),
}

ConfigPath specifies the locations to look up configurations

Functions

func Asset

func Asset(name string) ([]byte, error)

Asset loads and returns the asset for the given name. It returns an error if the asset could not be found or could not be loaded.

func AssetDir

func AssetDir(name string) ([]string, error)

AssetDir returns the file names below a certain directory embedded in the file by go-bindata. For example if you run go-bindata on data/... and data contains the following hierarchy:

data/
  foo.txt
  img/
    a.png
    b.png

then AssetDir("data") would return []string{"foo.txt", "img"} AssetDir("data/img") would return []string{"a.png", "b.png"} AssetDir("foo.txt") and AssetDir("notexist") would return an error AssetDir("") will return []string{"data"}.

func AssetInfo

func AssetInfo(name string) (os.FileInfo, error)

AssetInfo loads and returns the asset info for the given name. It returns an error if the asset could not be found or could not be loaded.

func AssetNames

func AssetNames() []string

AssetNames returns the names of the assets.

func ExternalIP

func ExternalIP() (string, error)

ExternalIP looks up an the first available external IP address.

func MustAsset

func MustAsset(name string) []byte

MustAsset is like Asset but panics when Asset would return an error. It simplifies safe initialization of global variables.

func ResolveAddr

func ResolveAddr(addr string) (string, error)

ResolveAddr accepts an address as a string and if the IP address is missing it replaces it with the result from ExternalIP then returns the addr string. Likewise if the Port is missing, it returns an address with the DefaultPort appended to the address string. This function hopefully will make the network configuration easy for new scio apps.

func RestoreAsset

func RestoreAsset(dir, name string) error

RestoreAsset restores an asset under the given directory

func RestoreAssets

func RestoreAssets(dir, name string) error

RestoreAssets restores an asset under the given directory recursively

Types

type App

type App struct {
	Config     *Config        // The configuration loaded from the YAML file
	GeoIP      *MaxMindClient // GeoIP Lookup API client
	Device     *Device        // Descriptor for the device in the database
	Location   *Location      // Current location of the application
	ExternalIP string         // Current external IP address of the machine
	// contains filtered or unexported fields
}

App is the primary application that maintains references to the config and device details as well as initializes the environment and runs the reflect and generate commands.

func Init

func Init() (*App, error)

Init the orca application NOTE: SyncLocation should not be called in init!

func (*App) ConnectDB

func (app *App) ConnectDB() error

ConnectDB establishes a connection to the Sqlite3 database

func (*App) CreateDB

func (app *App) CreateDB() error

CreateDB executes the create table statements in the schema.sql stored as binary data in the application (as well as any alter table statements).

func (*App) Echo

func (app *App) Echo(ctx context.Context, in *echo.Request) (*echo.Reply, error)

Echo implements the echo.EchoServer interface on the App

func (*App) FetchDevices

func (app *App) FetchDevices() (Devices, error)

FetchDevices returns a collection of devices, ordered by the created timestamp. This function expects you to limit the size of the collection by specifying the maximum number of nodes to return in the Devices list.

func (*App) FetchDevicesExcept

func (app *App) FetchDevicesExcept(device *Device) (Devices, error)

FetchDevicesExcept the specified device by excluding the device ID from the SQL query. Allows the creation of a device list except for the local device.

func (*App) Generate

func (app *App) Generate() error

Generate is long running function that initializes pings then sleeps.

func (*App) GetDB

func (app *App) GetDB() *sql.DB

GetDB returns the database connectio object from the app. Use with care, I didn't want to expose this outside the app ...

func (*App) GetDevice

func (app *App) GetDevice() *Device

GetDevice returns the device on the app, if it is nil, it performs a database query for the device based on the Config name; if there is nothing in the database, it inserts the record from the configuration.

func (*App) GetListenAddr

func (app *App) GetListenAddr() (string, error)

GetListenAddr looks up the IP address on the config or gets an external IP This is meant to be used by reflect mode to respond to echo requests.

func (*App) Ping

func (app *App) Ping(device *Device) error

Ping sends an echo request to a device and handles the response

func (*App) Reflect

func (app *App) Reflect() error

Reflect listens for EchoRequests and Replies to them.

func (*App) SendPing

func (app *App) SendPing(device *Device) (*echo.Reply, error)

SendPing sends an echo request to another device

func (*App) SetLocation

func (app *App) SetLocation(loc *Location, save bool) error

SetLocation is a wrapper method that sets the location on the app struct, but also does a check about whether or not to save it to the database.

func (*App) SyncLocation

func (app *App) SyncLocation() error

SyncLocation checks the external IP address against the current IP address, if they're different then it performs another location lookup to track mobility in the generator application, but does not perform GeoIP lookups if they're not necessary (to save bandwidth and cost).

type Config

type Config struct {
	Debug    bool   `yaml:"debug"`    // Print out log messages or not
	Name     string `yaml:"name"`     // The name of hte local device
	Addr     string `yaml:"addr"`     // The listen address of the local device
	Domain   string `yaml:"domain"`   // The domain name of the local device
	Interval int64  `yaml:"interval"` // The wait in seconds between pings to reflectors
	DBPath   string `yaml:"dbpath"`   // The path to the SQLite3 database
	MaxMind  *MaxMindConfig
}

Config is read from a YAML file and defines the current configuration of the project and can be exported as such.

func LoadConfig

func LoadConfig() *Config

LoadConfig the configuration from the ConfigPath

func (*Config) Parse

func (conf *Config) Parse(data []byte) error

Parse configuration from data

func (*Config) Read

func (conf *Config) Read(path string) error

Read a configuration from a path

func (Config) String

func (conf Config) String() string

String returns a string representation of the configuration

type Device

type Device struct {
	Name     string // Hostname of the device
	IPAddr   string // IP Address of the device
	Domain   string // Domain name of the device
	Sequence int64  // The response/reply counter for a device

	ModelMeta
	// contains filtered or unexported fields
}

Device is an entity that represents nodes in the network that can be pinged. Device objects are stored in the devices table.

func (*Device) Delete

func (d *Device) Delete(db *sql.DB) (bool, error)

Delete a device from the database. Returns true if the number of rows affected is 1 or false otherwise.

func (*Device) Echo

func (d *Device) Echo() *echo.Device

Echo converts a device to an echo.Device protocol buffer message.

func (*Device) Exists

func (d *Device) Exists(id int64, db *sql.DB) (bool, error)

Exists checks if the specified device is in the database.

func (*Device) Get

func (d *Device) Get(id int64, db *sql.DB) error

Get a device from the database by ID and populate the struct fields.

func (*Device) GetByName

func (d *Device) GetByName(name string, db *sql.DB) error

GetByName a device from the database and populate the struct fields.

func (*Device) Save

func (d *Device) Save(db *sql.DB) (bool, error)

Save a device struct to the database. This function checks if the device has an ID or not. If it does, it will execute a SQL UPDATE, otherwise it will execute a SQL INSERT. Returns a boolean if the device was inserted. This method handles setting the created and updated timestamps as well.

func (*Device) String

func (d *Device) String() string

String returns the textual string representation of the device

type Devices

type Devices []*Device

Devices is a collection of Device objects loaded from the database.

type Location

type Location struct {
	IPAddr       string  // IP Address associated with the location
	Latitude     float64 // Decimal based latitude
	Longitude    float64 // Decimal based logitude
	City         string  // City returned by MaxMind for the IP address
	PostCode     string  // Postal code returned by MaxMind for the IP address
	Country      string  // Country returned by MaxMind for the IP address
	Organization string  // Organization associated with the given domain (ISP)
	Domain       string  // Domain associated with the IP address (ISP)
	Note         string  // Any additional annotations by the user
	ModelMeta
}

Location is a geographic record that is usually associated with an IP address via the geoip lookup service but could also come from GPS.

func (*Location) Delete

func (loc *Location) Delete(db *sql.DB) (bool, error)

Delete a location from the database. Returns true if the number of rows affected is 1 or false otherwise.

func (*Location) Exists

func (loc *Location) Exists(id int64, db *sql.DB) (bool, error)

Exists checks if the specified location is in the database.

func (*Location) Get

func (loc *Location) Get(id int64, db *sql.DB) error

Get a location from the database by ID and populate the struct fields.

func (*Location) IPExists

func (loc *Location) IPExists(db *sql.DB) error

IPExists sets the ID on the location if the location's IP address is already in the database, otherwise it sets it to zero.

func (*Location) Save

func (loc *Location) Save(db *sql.DB) (bool, error)

Save a location struct to the database. This function checks if the location has an ID or not. If it does, it will execute a SQL UPDATE, otherwise it will execute a SQL INSERT. Returns a boolean if the location was inserted. This method handles setting the meta timestamps as well.

func (*Location) String

func (loc *Location) String() string

String returns a pretty representation of the location

type MaxMindClient

type MaxMindClient struct {
	// contains filtered or unexported fields
}

MaxMindClient is a utility for making geoip requests to MaxMind services.

func NewMaxMindClient

func NewMaxMindClient(userid string, license string) *MaxMindClient

NewMaxMindClient initializes the client struct with required info and also initializes the http client to make requests with HTTP Authentication.

func (*MaxMindClient) GeoIPLookup

func (mm *MaxMindClient) GeoIPLookup(ipaddr string) (map[string]interface{}, error)

GeoIPLookup fetches the raw JSON data from the MaxMind API for the given IP Address. Note that 'me' is a special input to autolookup the IP address from the request, and empty strings are converted to 'me'. This function returns the parsed JSON data as a map object.

func (*MaxMindClient) GetCurrentLocation

func (mm *MaxMindClient) GetCurrentLocation() (*Location, error)

GetCurrentLocation returns the current location using the special 'me' and returns a Location struct ready to be saved to the database.

func (*MaxMindClient) NewRequest

func (mm *MaxMindClient) NewRequest(ipaddr string) (*http.Request, error)

NewRequest creates an authenticated request to look up the given IP address NOTE that you can also use the string 'me' which performs auto lookup, if an empty string is passed to this function, then 'me' is used.

type MaxMindConfig

type MaxMindConfig struct {
	Username string // MaxMind User ID
	License  string // MaxMind License Key
}

MaxMindConfig specifies the MaxMind API credentials

type Model

type Model interface {
	Get(id int64, db *sql.DB) error            // Populate model fields from the database
	Save(db *sql.DB) (bool, error)             // Insert or update the model
	Delete(db *sql.DB) (bool, error)           // Delete the model from the database
	Exists(id int64, db *sql.DB) (bool, error) // Determine if the model exists by ID
}

Model specifies types that can interact with the database.

type ModelMeta

type ModelMeta struct {
	ID      int64     // Unique ID of the model
	Created time.Time // Datetime the model was added to the database
	Updated time.Time // Datetime the model was updated in the database
}

ModelMeta specifies the fields that all models should have via embedding.

type Ping

type Ping struct {
	ID       int64           //  Unique ID of the record
	Source   *Device         // Source device of the ping (always the local node)
	Target   *Device         // Target device that the ping was sent to
	Location *Location       // The location of the source at the time of the ping
	Request  int64           // Request sequence number for the source/target pair
	Response int64           // Response sequence number for the target/source pair
	Sent     time.Time       // The time that the ping was sent
	Recv     time.Time       // The time that the ping was received
	Latency  sql.NullFloat64 // The latency in milliseconds of the ping
}

Ping is a timeseries record of latency requests reflected from echo servers.

func (*Ping) Delete

func (p *Ping) Delete(db *sql.DB) (bool, error)

Delete a ping from the database. Returns true if the number of rows affected is 1 or false otherwise.

func (*Ping) Get

func (p *Ping) Get(id int64, db *sql.DB) error

Get a ping from the database by ID and populate the struct fields.

func (*Ping) Save

func (p *Ping) Save(db *sql.DB) (bool, error)

Save a ping struct to the database. This function checks if the ping has an ID or not. If it does, it will execute a SQL UPDATE, otherwise it will execute a SQL INSERT. Returns a boolean if the device was inserted. This method handles setting the created and updated timestamps as well.

func (*Ping) String

func (p *Ping) String() string

String returns a pretty representation of the ping

Directories

Path Synopsis
This command implements the Mora console utility that can run one of two background processes: reflectors and generators.
This command implements the Mora console utility that can run one of two background processes: reflectors and generators.
Package echo is a generated protocol buffer package.
Package echo is a generated protocol buffer package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL