README
¶
Conjure Liveness Module
The liveness module is designed to keep track of all the live hosts in the
network by scanning the internet on a regular basis. The module provides cached
liveness detection and uncached liveness detection. Cached liveness detection
stores live hosts from previous PhantomIsLive calls to improve performance,
non-responsive hosts and stale cache entries are re-scanned each time. Uncached
liveness detection directly visits an IP address to check its liveness status on
each call.
The validity of cached liveness detection was tested in a week-long survey of the internet where we measured the change in liveness across the network. We observed a clear trend in the stability of discovered hosts over time and as such chose a 2 hour default cache period.
Usage
To check if an IP address is live in the network, call PhantomIsLive(addr string, port uint16) which return a bool and an error message if applicable.
Network Survey Result
 |
|---|
| Survey data for 3 weeks |
 |
| Survey data for first 48 hours |
Percentages of the number of current live hosts divided by the number of cached
live hosts(marked green) went down drastically in the first 24 hours of scanning
the network which indicates that caching every discoverable live hosts is not an
effective way to represent the current liveness status of the addresses in the
network over time. Instead, we decided to cache individual IP addresses that are passed to
PhantomIsLive for checking its liveness status. Every cached address gets a
timestamp when its liveness status is checked, cached addresses with expired
timestamp will no longer be considered as live hosts by the module. Expiration
duration can be set in the config file.
Caching Flow Diagram
Documentation
¶
Index ¶
- Variables
- type CachedLivenessTester
- func (blt *CachedLivenessTester) ClearExpiredCache()
- func (blt *CachedLivenessTester) Init(conf *Config) error
- func (blt *CachedLivenessTester) PhantomIsLive(addr string, port uint16) (bool, error)
- func (blt *CachedLivenessTester) PrintAndReset(logger *log.Logger)
- func (blt *CachedLivenessTester) PrintStats(logger *log.Logger)
- func (s CachedLivenessTester) Reset()
- func (blt *CachedLivenessTester) Stop()
- type Config
- type Stats
- type Tester
- type UncachedLivenessTester
Constants ¶
This section is empty.
Variables ¶
var ( // ErrCachedPhantom provides a constant expected error returned for cached // liveness hits ErrCachedPhantom = errors.New("cached live host") // ErrNotImplemented indicates that a feature will be implemented at some point // but is not yet completed. ErrNotImplemented = errors.New("not supported yet") // NotLive is an error returned when a liveness test reaches timeout, giving // confidence that the host in question is not live NotLive = errors.New("reached statistical timeout") // ErrLiveHost indicates that an error occurred or a successful connection // was formed with the host in question indicating that the host is live. ErrLiveHost = errors.New("phantom picked up the connection") )
Functions ¶
This section is empty.
Types ¶
type CachedLivenessTester ¶
type CachedLivenessTester struct {
// contains filtered or unexported fields
}
CachedLivenessTester implements LivenessTester interface with caching, PhantomIsLive will check historical results first before using the network to determine phantom liveness.
func (*CachedLivenessTester) ClearExpiredCache ¶
func (blt *CachedLivenessTester) ClearExpiredCache()
ClearExpiredCache cleans out stale entries in the cache.
func (*CachedLivenessTester) Init ¶
func (blt *CachedLivenessTester) Init(conf *Config) error
Init parses cache expiry duration and initializes the Cache.
func (*CachedLivenessTester) PhantomIsLive ¶
func (blt *CachedLivenessTester) PhantomIsLive(addr string, port uint16) (bool, error)
PhantomIsLive first checks the cached set of addresses for a fresh entry. If one is available and this is returned immediately and no network probes are sent. If the host was not recently measured, the entry is stale, or there is no entry then network probes are sent and the result is then added to the cache.
Lock on mutex is taken for lookup, then for cache update. Do NOT hold mutex while scanning for liveness as this will make cache extremely slow.
func (*CachedLivenessTester) PrintAndReset ¶
func (blt *CachedLivenessTester) PrintAndReset(logger *log.Logger)
PrintAndReset implements the Stats interface extending from the stats struct to add logging for the cache capacity
func (*CachedLivenessTester) PrintStats ¶
func (blt *CachedLivenessTester) PrintStats(logger *log.Logger)
PrintStats implements the Stats interface extending from the stats struct to add logging for the cache capacity
func (*CachedLivenessTester) Stop ¶
func (blt *CachedLivenessTester) Stop()
Stop end periodic scanning using running in separate goroutine. If periodic scanning is not running this will do nothing.
type Config ¶
type Config struct {
// CacheDuration specifies the duration that a phantom IP identified as
// "LIVE" using a liveness test is cached, preventing further lookups to the
// address. Empty string disables caching for live phantom hosts.
CacheDuration string `toml:"cache_expiration_time"`
// CacheCapacity specifies the cache capacity to use for phantom IPs
// identified as "LIVE". CacheDuration must be set otherwise no caching
// occurs for live hosts.
//
// If unset or 0 no capacity is set and a map is used for the cache
// otherwise cache will have finite capacity and implement LRU eviction.
CacheCapacity int `toml:"cache_capacity"`
// CacheDurationNonLive specifies the duration that a phantom IP identified
// as "NOT LIVE" using a liveness test is cached, preventing further lookups
// to the address. This should generally be shorter to be responsive to
// remain responsive to hosts that become live. Empty string disables
// caching for non-live phantom hosts.
CacheDurationNonLive string `toml:"cache_expiration_nonlive"`
// CacheCapacityNonLive specifies the cache capacity to use for phantom IPs
// identified as "NOT LIVE". CacheDurationNonLive must be set otherwise no
// caching occurs for non-live hosts.
//
// If unset or 0 no capacity is set and a map is used for the cache
// otherwise cache will have finite capacity and implement LRU eviction.
CacheCapacityNonLive int `toml:"cache_capacity_nonlive"`
}
Config provides all params relating to liveness testing construction
func (*Config) LivenessConfig ¶
LivenessConfig identity function for reflection in composed Config type
type Stats ¶
Stats provides an interface to write out the collected metrics about liveness tester usage
type Tester ¶
Tester provides a generic interface for testing hosts in phantom subnets for liveness. This prevents potential interference in connection creation.
type UncachedLivenessTester ¶
type UncachedLivenessTester struct {
// contains filtered or unexported fields
}
UncachedLivenessTester implements LivenessTester interface without caching, PhantomIsLive will always use the network to determine phantom liveness.
func (*UncachedLivenessTester) PhantomIsLive ¶
func (blt *UncachedLivenessTester) PhantomIsLive(addr string, port uint16) (bool, error)
PhantomIsLive sends 4 TCP syn packets to determine if the host will respond to traffic and potentially interfere with a connection if used as a phantom address. Measurement results are uncached, meaning endpoints are re-scanned every time.
Source Files