memcachey

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 6, 2019 License: Apache-2.0 Imports: 12 Imported by: 0

README

go-memcachey

GoDoc CircleCI

go-memcachey is a Golang client library for the Memcached in-memory database. See godoc for documentation.

Example

Basic Example

import "github.com/janberktold/go-memcachey"

func main() {
    client, _ := memcachey.NewClient([]string{"127.0.0.1:11211"})

    // Set a value to Memcached
    client.Set("some_key", []byte{1, 2, 3})

    // Read the value back
    value, _ := client.Get("some_key")

    fmt.Printf("Read %v from Memcached!", value)
}

Customized client

func main() {
    client, _ := memcachey.NewClient([]string{"127.0.0.1:11211"},
                    memcachey.WithConsistentHashing(),
                    memcachey.WithPoolLimitsPerHost(20, 100),
                    memcachey.WithTimeouts(10 * time.Second))

    // Read a value from Memcached
    value, _ := client.Get("some_key")

    fmt.Printf("Read %v from Memcached!", value)
}

Contributing

Testing

This client includes a testing suite which runs commands against a Memcached server running at 127.0.0.1:11211. It is run on every opened pull request, but it is recommended to test locally beforehand. The easiest way to spin up the required Memcached server is to use Docker:

docker run --publish 127.0.0.1:11211:11211 memcached:1.5.17-alpine -v -v -v

After which you can execute the tests any number of times:

go test -v

Documentation

Overview

Package memcachey provides a modern, scalable client for the Memcached in-memory database.

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrNotStored means that a conditional write operation failed because the condition was not satisfied.
	ErrNotStored = errors.New("item not stored")

	// ErrNotFound means that the operation expected a key+value to be present, but they were not.
	ErrNotFound = errors.New("key not found")

	// ErrNotAnInteger means that we expected an integer but did not find one.
	ErrNotAnInteger = errors.New("value is not an integer")

	// ErrKeyTooLong means that a passed in key contains more than 250 characters.
	ErrKeyTooLong = errors.New("key is longer than 250 characters")

	// ErrNoSuchAddress means that the requested address is not known.
	ErrNoSuchAddress = errors.New("requested address not found")

	// ErrCommandNotSupported means that the command is not supported on the Memcached server.
	ErrCommandNotSupported = errors.New("command not supported on memcached host")
)

Functions

This section is empty.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is our interface over the logical connection to several Memcached hosts.

func NewClient

func NewClient(addresses []string, options ...ClientOptionsSetter) (*Client, error)

NewClient creates a new Memcached client. The addresses are used round-robin by default, make sure to pass in the correct options.

func (*Client) Add

func (c *Client) Add(key string, value []byte) error

Add sets a key on the Memcached server only if the key did not have a value previously. Returns ErrNotStored if the key was assigned a value.

func (*Client) AddWithExpiry

func (c *Client) AddWithExpiry(key string, value []byte, expiry time.Duration) error

AddWithExpiry sets a key on the Memcached server which expires after the specified time, only if the key did not have a value previously. Returns ErrNotStored if the key was assigned a value.

func (*Client) ConnectionStatisticsForAddress

func (c *Client) ConnectionStatisticsForAddress(address string) (map[string]*ConnectionStatistics, error)

ConnectionStatisticsForAddress returns per-connection statistics.

func (*Client) Decrement

func (c *Client) Decrement(key string, delta uint64) (uint64, error)

Decrement decrements the value saved for the specified key, if it exists. Returns a ErrKeyNotFound if the key does not yet exist.

func (*Client) Delete

func (c *Client) Delete(key string) (existed bool, err error)

Delete marks a key as deleted in memcached. Returns true if the key existed.

func (*Client) FlushAllForAddress

func (c *Client) FlushAllForAddress(address string) error

FlushAllForAddress allows flushing an entire Memcached host.

func (*Client) FlushAllWithDelayForAddress

func (c *Client) FlushAllWithDelayForAddress(address string, delay time.Duration) error

FlushAllWithDelayForAddress allows flushing an entire Memcached host with a delay before the deletions take effect.

func (*Client) Get

func (c *Client) Get(key string) ([]byte, error)

Get queries memcached for a single key and returns the value.

func (*Client) GetUInt64

func (c *Client) GetUInt64(key string) (uint64, error)

GetUInt64 queries memcached for a single key and returns the value as an uint64. Returns an ErrNotAnInteger if we did not find an integer.

func (*Client) Increment

func (c *Client) Increment(key string, delta uint64) (uint64, error)

Increment increments the value saved for the specified key, if it exists. Returns a ErrKeyNotFound if the key does not yet exist.

func (*Client) ItemStatisticsForAddress

func (c *Client) ItemStatisticsForAddress(address string) (map[string]*ItemStatistics, error)

ItemStatisticsForAddress returns information about item storage per slab class.

func (*Client) MultiGet

func (c *Client) MultiGet(keys []string) (map[string][]byte, error)

MultiGet queries memcached for a collection of keys.

func (*Client) Replace

func (c *Client) Replace(key string, value []byte) error

Replace sets a key on the Memcached server only if the key already exists. Returns ErrNotStored if the key did not have a value.

func (*Client) ReplaceWithExpiry

func (c *Client) ReplaceWithExpiry(key string, value []byte, expiry time.Duration) error

ReplaceWithExpiry sets a key on the Memcached server which expires after the specified time, only if the key already exists. Returns ErrNotStored if the key did not have a value.

func (*Client) Set

func (c *Client) Set(key string, value []byte) error

Set sets a key on the Memcached server, regardless of the previous state.

func (*Client) SetSlabsAutomoveModeForAddress

func (c *Client) SetSlabsAutomoveModeForAddress(address string, mode SlabsAutomoveMode) error

SetSlabsAutomoveModeForAddress allows setting the SlabsAutomoveMode on the specified host.

func (*Client) SetUInt64

func (c *Client) SetUInt64(key string, value uint64) error

SetUInt64 sets a key on the Memcached server to the specified integer, regardless of the previous state.

func (*Client) SetWithExpiry

func (c *Client) SetWithExpiry(key string, value []byte, expiry time.Duration) error

SetWithExpiry sets a key on the Memcached server which expires after a duration, regardless of the previous state.

func (*Client) SettingsStatisticsForAddress

func (c *Client) SettingsStatisticsForAddress(address string) (*SettingsStatistics, error)

SettingsStatisticsForAddress returns details of the settings of the running memcached. This is primarily made up of the results of processing commandline options.

func (*Client) SlabsStatisticsForAddress

func (c *Client) SlabsStatisticsForAddress(address string) (*SlabStatistics, error)

SlabsStatisticsForAddress returns per-slab statistics for the specified memcached server.

func (*Client) StatisticsForAddress

func (c *Client) StatisticsForAddress(address string) (*Statistics, error)

StatisticsForAddress returns general-purpose statistics about the specified host.

func (*Client) Touch

func (c *Client) Touch(key string, expirationTime time.Duration) (existed bool, err error)

Touch updates the expiration time of an existing item without fetching it.

func (*Client) VersionForAddress

func (c *Client) VersionForAddress(address string) (string, error)

VersionForAddress returns the reported version for the specified Memcached host.

type ClientOptionsSetter

type ClientOptionsSetter func(client *Client) error

ClientOptionsSetter provides the type for all methods which are able to alter the Memcached client's behaviour.

func WithConsistentHashing

func WithConsistentHashing() ClientOptionsSetter

WithConsistentHashing enables consistent hashing when passed into NewClient() over the default which is a round-robin strategy.

func WithPoolLimitsPerHost

func WithPoolLimitsPerHost(min, max int) ClientOptionsSetter

WithPoolLimitsPerHost allows specifying custom min and max limits for the connection pool on a per-host basis. Defaults to min=1 and max=20 if not set.

func WithTimeouts

func WithTimeouts(connectionTimeout time.Duration) ClientOptionsSetter

WithTimeouts allows specifying custom timeouts for the client.

type ConnectionStatistics

type ConnectionStatistics struct {
	// Address is the the address of the remote side. For listening
	// sockets this is the listen address. Note that some socket types
	// (such as UNIX-domain) don't have meaningful remote addresses.
	Address string `proto:"addr"`

	// The address of the server. This field is absent for listening sockets.
	ListenAddress string `proto:"listen_addr"`

	// The current state of the connection.
	State string `proto:"state"`

	// The number of seconds since the most recently issued command on the connection.
	// This measures the time since the start of the command, so if "state" indicates a
	// command is currently executing, this will be the number of seconds the current
	// command has been running.
	TimeSinceLastCommand time.Duration `proto:"secs_since_last_cmd"`
}

ConnectionStatistics contains information about a specific connection.

type ItemStatistics

type ItemStatistics struct {
	// Number of items presently stored in this class. Expired items are not excluded.
	Number uint64 `proto:"number"`

	//Number of times an entry was stored using memory from an expired entry.
	ReclaimedTimes uint64 `proto:"reclaimed"`

	// NumberOfHotItems is the number of items presently stored in the HOT LRU.
	NumberOfHotItems uint64 `proto:"number_hot"`

	// NumberOfWarmItems is the number of items presently stored in the WARM LRU.
	NumberOfWarmItems uint64 `proto:"number_warm"`

	// NumberOfColdItems is the number of items presently stored in the COLD LRU.
	NumberOfColdItems uint64 `proto:"number_cold"`

	// NumberOfTemporaryItems is the number of items presently stored in the TEMPORARY LRU.
	NumberOfTemporaryItems uint64 `proto:"number_temp"`

	// OldestHotItem is the age of the oldest item in the hot lru.
	OldestHotItem time.Duration `proto:"age_hot"`

	// OldestWarmItem is the age of the oldest item in the warm lru.
	OldestWarmItem time.Duration `proto:"age_warm"`

	// OldestItem is the age of the oldest item in the lru.
	OldestItem time.Duration `proto:"age"`

	// Number of bytes requested to be stored in this LRU[*]
	MemoryRequested uint64 `proto:"mem_requested"`

	// Number of times an item had to be evicted from the LRU before it expired.
	EvictedItems uint64 `proto:"evicted"`

	// Number of times an item which had an explicit expire time set had
	// to be evicted from the LRU before it expired.
	EvictedItemsWithExpireTime uint64 `proto:"evicted_nonzero"`

	// Seconds since the last access for the most recent item
	// evicted from this class. Use this to judge how
	// recently active your evicted data is.
	EvictedTime time.Duration `proto:"evicted_time"`

	// Number of times the underlying slab class was unable to
	// store a new item. This means you are running with -M or
	// an eviction failed.
	ItemsDeniedDueToOutOfMemory uint64 `proto:"outofmemory"`

	//  Number of times we self-healed a slab with a refcount
	// leak. If this counter is increasing a lot, please
	// report your situation to the developers.
	TailRepairs uint64 `proto:"tailrepairs"`

	// Number of expired items reclaimed from the LRU which
	// were never touched after being set.
	ExpiredUnfetchedItems uint64 `proto:"expired_unfetched"`

	// Number of valid items evicted from the LRU which were
	// never touched after being set.
	EvictedUnfetchedItems uint64 `proto:"evicted_unfetched"`

	// Number of valid items evicted from the LRU which were
	// recently touched but were evicted before being moved to
	// the top of the LRU again.
	EvictedActiveItems uint64 `proto:"evicted_active"`

	// Number of items freed by the LRU Crawler.
	CrawlerReclaimedItems uint64 `proto:"crawler_reclaimed"`

	// Number of items found to be refcount locked in the LRU tail.
	LRUTailReflockedItems uint64 `proto:"lrutail_reflocked"`

	// Number of items moved from HOT or WARM into COLD.
	ItemsMovedToCold uint64 `proto:"moves_to_cold"`

	// Number of items moved from COLD to WARM.
	ItemsMovedToWarm uint64 `proto:"moves_to_warm"`

	// Number of times active items were bumped within HOT or WARM.
	ItemsMovedWithinLRU uint64 `proto:"moves_within_lru"`

	// Number of times worker threads had to directly pull LRU
	// tails to find memory for a new item.
	DirectReclaims uint64 `proto:"direct_reclaims"`

	HotHits  uint64 `proto:"hits_to_hot"`
	WarmHits uint64 `proto:"hits_to_warm"`
	ColdHits uint64 `proto:"hits_to_cold"`

	// Number of get_hits to each sub-LRU.
	TemporaryHits uint64 `proto:"hits_to_temp"`
}

ItemStatistics contains information about item storage per slab class.

type PerSlabStatistics

type PerSlabStatistics struct {
	// The amount of space each chunk uses. One item will use one
	// chunk of the appropriate size
	ChunkSize uint64 `proto:"chunk_size"`

	// How many chunks exist within one page. A page by default
	// is less than or equal to one megabyte in size
	ChunksPerPage uint64 `proto:"chunks_per_page"`

	// Total number of pages allocated to the slab class
	TotalPages uint64 `proto:"total_pages"`

	// Total number of chunks allocated to the slab class
	TotalChunks uint64 `proto:"total_chunks"`

	// Total number of get requests serviced by this class
	GetHits uint64 `proto:"get_hits"`

	// Total number of set requests storing data in this class
	SetCommands uint64 `proto:"cmd_set"`

	// Total number of successful deletes from this class
	DeleteHits uint64 `proto:"delete_hits"`

	// Total number of incrs modifying this class
	IncrementHits uint64 `proto:"incr_hits"`

	// Total number of decrs modifying this class
	DecrementHits uint64 `proto:"decr_hits"`

	// Total number of CAS commands modifying this class
	CheckAndSetHits uint64 `proto:"cas_hits"`

	// Total number of CAS commands that failed to modify a value due to a bad CAS id.
	CheckAndSetBadValue uint64 `proto:"cas_badval"`

	// Total number of touches serviced by this class
	TouchHits uint64 `proto:"touch_hits"`

	// How many chunks have been allocated to items
	UsedChunks uint64 `proto:"used_chunks"`

	// Chunks not yet allocated to items, or freed via delete
	FreeChunks uint64 `proto:"free_chunks"`

	// Number of free chunks at the end of the last allocated page
	FreeChecksEnd uint64 `proto:"free_chunks_end"`
}

PerSlabStatistics contains about statistics about a single slab.

type SettingsStatistics

type SettingsStatistics struct {
	// MaxBytes represents the maximum number of bytes allowed in the cache.
	MaxBytes uint64 `proto:"maxbytes"`

	// MaxConnections is the maximum number of clients allowed
	MaxConnections uint32 `proto:"maxconns"`

	// TCPPort is the TCP port to listen on.
	TCPPort uint32 `proto:"tcpport"`

	// TCPPort is the UDP port to listen on.
	UDPPort uint32 `proto:"udpport"`

	// The Interface to listen on.
	Interface string `proto:"inter"`

	// Verbosity level
	Verbosity uint32 `proto:"verbosity"`

	// Oldest is the age of the oldest honored object
	Oldest time.Duration `proto:"oldest"`

	// When off, LRU evictions are disabled
	LRUEvictionsEnabled bool `proto:"evictions"`

	// Path to the domain socket (if any).
	DomainSocket string `proto:"domain_socket"`

	// umask for the creation of the domain socket
	Umask uint32 `proto:"umask"`

	// Chunk size growth factor
	ChunkSizeGrowthFactor float64 `proto:"growth_factor"`

	// Minimum space allocated for key+value+flags
	InitialChunkSize uint32 `proto:"chunk_size"`

	// Number of threads (including dispatch)
	NumberOfThreads uint32 `proto:"num_threads"`

	// Stats prefix separator character
	StatsPrefixSeperatorCharacter string `proto:"stat_key_prefix"`

	// If yes, stats detail is enabled
	StatisticsDetailEnabled bool `proto:"detail_enabled"`

	// Max num IO ops processed within an event
	RequestsPerEvent uint32 `proto:"reqs_per_event"`

	// When no, CAS is not enabled for this server
	CheckAndSetEnabled bool `proto:"cas_enabled"`

	// TCP listen
	TCPBacklog uint32 `proto:"tcp_backlog"`

	// SASL auth requested and enabled
	SASLAuthenticationEnabled bool `proto:"auth_enabled_sasl"`

	// maximum item size
	MaximumItemSize uint32 `proto:"item_size_max"`

	// If fast disconnects are enabled
	FastMaximumConnectionsEnabled bool `proto:"maxconns_fast"`

	// Starting size multiplier for hash table
	InitialHashPower uint32 `proto:"hashpower_init"`

	// SlabReassignAllowed represents whether slab page reassignment is allowed
	SlabReassignAllowed bool `proto:"slab_reassign"`

	// SlabAutomoverMode represents the current mode.
	SlabAutomoverMode SlabsAutomoveMode `proto:"slab_automove"`

	// SlabAutomoverRatio is the ratio limit between young/old slab classes
	SlabAutomoverRatio float64 `proto:"slab_automove_ratio"`

	// SlabAutomoverWindow is an internal algo tunable for automove
	SlabAutomoverWindow uint32 `proto:"slab_automove_window"`

	// MaximumSlabChunkSize is the maximum slab class size (avoid unless necessary)
	MaximumSlabChunkSize uint32 `proto:"slab_chunk_max"`

	// HashAlgorithm is the hash algorithm used for the hash table.
	HashAlgorithm string `proto:"hash_algorithm"`

	// LRUCrawlerEnabled represents whether the background thread running the LRU crawler is running.
	LRUCrawlerEnabled bool `proto:"lru_crawler"`

	// Microseconds to sleep between LRU crawls
	LRUCrawlerSleep uint32 `proto:"lru_crawler_sleep"`

	// Max items to crawl per slab per run
	LRUCrawlerMaximumItems uint32 `proto:"lru_crawler_tocrawl"`

	// Split LRU mode and background threads
	LRUMaintainerThread bool `proto:"lru_maintainer_thread"`

	// Pct of slab memory reserved for HOT LRU
	HotLRUPct uint32 `proto:"hot_lru_pct"`

	// Pct of slab memory reserved for WARM LRU
	WarmLRUPct uint32 `proto:"warm_lru_pct"`

	// Set idle age of HOT LRU to COLD age * this
	MaximumHotFactor float64 `proto:"hot_max_factor"`

	// Set idle age of WARM LRU to COLD age * this
	MaximumWarmFactor float64 `proto:"warm_max_factor"`

	// If yes, items < temporary_ttl use TEMP_LRU
	TemporaryLRUEnabled bool `proto:"temp_lru"`

	// Items with TTL < this are marked temporary
	TemporaryTTL uint32 `proto:"temporary_ttl"`

	// Drop connections that are idle this many seconds (0 disables)
	ConnectionMaximumIdleTime time.Duration `proto:"idle_time"`

	// Size of internal (not socket) write buffer per active watcher connected.
	WatcherWriteBufferSize uint32 `proto:"watcher_logbuf_size"`

	// Size of internal per-worker-thread buffer which the background thread reads from.
	WorkerWriteBufferSize uint32 `proto:"worker_logbuf_size"`

	// If yes, a "stats sizes" histogram is being dynamically tracked.
	TrackingSizesEnabled bool `proto:"track_sizes"`

	// If yes, and available, drop unused syscalls (see seccomp on Linux, pledge on OpenBSD)
	DropPriviliges bool `proto:"drop_privileges"`
}

SettingsStatistics contains details of the settings of the running memcached

type SlabStatistics

type SlabStatistics struct {
	// Total number of slab classes allocated
	ActiveSlabs uint64 `proto:"active_slabs"`

	// Total amount of memory allocated to slab pages
	TotalMalloced uint64 `proto:"total_malloced"`

	// PerSlabStatistics contains detailed statistics on a per-chunk basis.
	PerSlabStatistics map[uint]*PerSlabStatistics
}

SlabStatistics contains statistics about the server's slabs.

type SlabsAutomoveMode

type SlabsAutomoveMode uint8

SlabsAutomoveMode represents the mode for the background thread which can decide on it's own when to move memory between slab classes.

const (
	// SlabsAutomoveModeStandby sets the thread on standby.
	SlabsAutomoveModeStandby SlabsAutomoveMode = iota
	// SlabsAutomoveModeStandard means to return pages to a global pool when there are more than 2 pages
	// worth of free chunks in a slab class. Pages are then re-assigned back into other classes as-needed.
	SlabsAutomoveModeStandard
	// SlabsAutomoveModeAggressive is a highly aggressive mode which causes pages to be moved every time
	// there is an eviction. It is not recommended to run for very long in this
	// mode unless your access patterns are very well understood.
	SlabsAutomoveModeAggressive
)

type Statistics

type Statistics struct {
	// ProcessID is the process id of the server process
	ProcessID uint32 `proto:"pid"`

	// Uptime is the time since the server started
	Uptime time.Duration `proto:"uptime"`

	// Time is the current UNIX time according to the server
	Time uint64 `proto:"time"`

	// Version is the version string reported by the server
	Version string `proto:"version"`

	// PointerSize on the host OS, generally either 32 or 64.
	PointerSize uint32 `proto:"pointer_size"`

	// CurrentItems is the current number of items stored
	CurrentItems uint64 `proto:"curr_items"`

	// TotalItems is the total number of items stored since the server started
	TotalItems uint64 `proto:"total_items"`

	// Bytes is the current number of bytes used to store items
	Bytes uint64 `proto:"bytes"`

	// Max number of  simultaneous connections
	MaxConnections uint32 `proto:"max_connections"`

	// Total number of connections opened since the server started running
	TotalConnections uint32 `proto:"total_connections"`

	// CurrentConenctions is the current number of open connections
	CurrentConnnections uint32 `proto:"curr_connections"`

	// RejectedConnections is the connections rejected in maxconns_fast mode
	RejectedConnections uint64 `proto:"rejected_connections"`

	// ConnectionStructures is the number of connection structures allocated by the server
	ConnectionStructures uint32 `proto:"connection_structures"`

	// Number of misc fds used internally
	ReservedFileDescriptors uint32 `proto:"reserved_fds"`

	// Cumulative number of retrieval reqs
	TotalGetCommands uint64 `proto:"cmd_get"`

	// Cumulative number of storage reqs
	TotalSetCommands uint64 `proto:"cmd_set"`

	// Cumulative number of flush reqs
	TotalFlushCommands uint64 `proto:"cmd_flush"`

	// Cumulative number of touch reqs
	TotalTotalCommands uint64 `proto:"cmd_touch"`

	// Number of keys that have been requested and found present
	GetHits uint64 `proto:"get_hits"`

	// Number of items that have been requested and not found
	GetMisses uint64 `proto:"get_misses"`

	// Number of items that have been requested but had already expired
	GetExpired uint64 `proto:"get_expired"`

	// Number of items that have been requested but have been flushed via flush_all
	GetFlushed uint64 `proto:"get_flushed"`

	// Number of deletion reqs resulting in an item being removed
	DeleteHits uint64 `proto:"delete_hits"`

	// Number of deletions reqs for missing keys
	DeleteMisses uint64 `proto:"delete_misses"`

	// Number of incr reqs against missing keys
	IncrementMisses uint64 `proto:"incr_misses"`

	// Number of successful incr reqs
	IncrementHits uint64 `proto:"incr_hits"`

	// Number of decr reqs against missing keys
	DecrementMisses uint64 `proto:"decr_misses"`

	// Number of successful decr reqs
	DecrementHits uint64 `proto:"decr_hits"`

	// Number of CAS reqs against missing keys
	CheckAndSetMisses uint64 `proto:"cas_misses"`

	// Number of successful CAS reqs
	CheckAndSetHits uint64 `proto:"cas_hits"`

	// Number of CAS reqs for which a key was found, but the CAS value did not match
	CheckAndSetBadValue uint64 `proto:"cas_badval"`

	// Number of keys that have been touched with a new expiration time
	TouchHits uint64 `proto:"touch_hits"`

	// Number of items that have been touched and not found
	TouchMisses uint64 `proto:"touch_misses"`

	// Number of authentication commands handled, success or failure
	AuthenticationCommands uint64 `proto:"auth_cmds"`

	// Number of failed authentications
	AuthenticationErrors uint64 `proto:"auth_errors"`
}

Statistics contains general-purpose statistics about a Memcached server.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL