ybc: github.com/valyala/ybc/libs/go/memcache Index | Files

package memcache

import "github.com/valyala/ybc/libs/go/memcache"

Package memcache provides fast client and server implementations for memcache.

Index

Package Files

auxiliary.go caching_client.go client.go consistent_hash.go distributed_client.go interfaces.go package.go server.go

Variables

var (
    ErrCacheMiss            = errors.New("memcache.Client: cache miss")
    ErrCasidMismatch        = errors.New("memcache.Client: casid mismatch")
    ErrClientNotRunning     = errors.New("memcache.Client: the client isn't running")
    ErrCommunicationFailure = errors.New("memcache.Client: communication failure")
    ErrMalformedKey         = errors.New("memcache.Client: malformed key")
    ErrNilValue             = errors.New("memcache.Client: nil value")
    ErrNotModified          = errors.New("memcache.Client: item not modified")
    ErrAlreadyExists        = errors.New("memcache.Client: the item already exists")
)
var (
    ErrNoServers = errors.New("memcache.DistributedClient: there are no registered servers")
)

type Cacher

type Cacher interface {
    Ccacher

    Start()
    Stop()
}

Client and DistributedClient implement this interface.

type CachingClient

type CachingClient struct {
    // The underlying memcache client.
    //
    // The client must be initialized before passing it here.
    //
    // Currently Client and DistributedClient may be passed here.
    Client Ccacher

    // The underlying local cache.
    //
    // The cache should be initialized before passing it here.
    //
    // Currently ybc.Cache and ybc.Cluster may be passed here.
    Cache ybc.Cacher
}

Memcache client with in-process data caching.

It can save network bandwidth between the client and memcache servers.

The client uses approach similar to HTTP cache validation with entity tags - see http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.11 .

Usage:

cache := openCache()
defer cache.Close()

client.Start()
defer client.Stop()

c := memcache.CachingClient{
    Client: client,
    Cache:  cache,
}

item := memcache.Item{
    Key:   []byte("key"),
    Value: []byte("value"),
}
if err := c.Set(&item); err != nil {
    log.Fatalf("Error in c.Set(): %s", err)
}
if err := c.Get(&item); err != nil {
    log.Fatalf("Error in c.Get(): %s", err)
}

func (*CachingClient) Add

func (c *CachingClient) Add(item *Item) error

See Client.Add()

func (*CachingClient) AddWithValidateTtl

func (c *CachingClient) AddWithValidateTtl(item *Item, validateTtl time.Duration) error

The same as CachingClient.Add(), but sets interval for periodic item revalidation on the server.

This means that outdated, locally cached version of the item may be returned during validateTtl interval. Use CachingClient.Add() if you don't unserstand how this works.

Setting validateTtl to 0 leads to item re-validation on every get() request. This is equivalent ot CachingClient.Add() call. Even in this scenario network bandwidth between the client and memcache servers is saved if the average item size exceeds ~100 bytes.

func (*CachingClient) Cas

func (c *CachingClient) Cas(item *Item) error

See Client.Cas()

func (*CachingClient) CasWithValidateTtl

func (c *CachingClient) CasWithValidateTtl(item *Item, validateTtl time.Duration) error

The same as CachingClient.Cas(), but sets interval for periodic item revalidation on the server.

This means that outdated, locally cached version of the item may be returned during validateTtl interval. Use CachingClient.Cas() if you don't unserstand how this works.

Setting validateTtl to 0 leads to item re-validation on every get() request. This is equivalent ot CachingClient.Cas() call. Even in this scenario network bandwidth between the client and memcache servers is saved if the average item size exceeds ~100 bytes.

func (*CachingClient) Delete

func (c *CachingClient) Delete(key []byte) error

See Client.Delete()

func (*CachingClient) DeleteNowait

func (c *CachingClient) DeleteNowait(key []byte)

See Client.DeleteNowait()

func (*CachingClient) FlushAll

func (c *CachingClient) FlushAll() error

See Client.FlushAll()

func (*CachingClient) FlushAllDelayed

func (c *CachingClient) FlushAllDelayed(expiration time.Duration) error

See Client.FlushAllDelayed()

func (*CachingClient) FlushAllDelayedNowait

func (c *CachingClient) FlushAllDelayedNowait(expiration time.Duration)

See Client.FlushAllDelayedNowait()

func (*CachingClient) FlushAllNowait

func (c *CachingClient) FlushAllNowait()

See Client.FlushAllNowait()

func (*CachingClient) Get

func (c *CachingClient) Get(item *Item) error

See Client.Get()

func (*CachingClient) GetDe

func (c *CachingClient) GetDe(item *Item, graceDuration time.Duration) error

See Client.GetDe()

func (*CachingClient) GetMulti

func (c *CachingClient) GetMulti(items []Item) error

See Client.GetMulti()

func (*CachingClient) Set

func (c *CachingClient) Set(item *Item) error

See Client.Set()

func (*CachingClient) SetNowait

func (c *CachingClient) SetNowait(item *Item)

See Client.SetNowait()

func (*CachingClient) SetWithValidateTtl

func (c *CachingClient) SetWithValidateTtl(item *Item, validateTtl time.Duration) error

The same as CachingClient.Set(), but sets interval for periodic item revalidation on the server.

This means that outdated, locally cached version of the item may be returned during validateTtl interval. Use CachingClient.Set() if you don't understand how this works.

Setting validateTtl to 0 leads to item re-validation on every get() request. This is equivalent to CachingClient.Set() call. Even in this scenario network bandiwdth between the client and memcache servers is saved if the average item size exceeds ~100 bytes.

func (*CachingClient) SetWithValidateTtlNowait

func (c *CachingClient) SetWithValidateTtlNowait(item *Item, validateTtl time.Duration)

The same as CachingClient.SetWithValidateTtl(), but doesn't wait for completion of the operation.

type Ccacher

type Ccacher interface {
    MemcacherDe

    Cget(item *Item) error
    CgetDe(item *Item, graceDuration time.Duration) error
}

Client and DistributedClient implement this interface.

type Client

type Client struct {
    ClientConfig

    // TCP address of memcached server to connect to.
    // Required parameter.
    //
    // The address should be in the form addr:port.
    ServerAddr string
    // contains filtered or unexported fields
}

Fast memcache client.

The client is goroutine-safe. It is designed to work fast when hundreds concurrent goroutines are calling simultaneously its' methods.

The client works with a single memcached server. Use DistributedClient if you want working with multiple servers.

Usage:

c := Client{
    ServerAddr: ":11211",
}
c.Start()
defer c.Stop()

item := Item{
    Key:   []byte("key"),
    Value: []byte("value"),
}
if err := c.Set(&item); err != nil {
    handleError(err)
}
if err := c.Get(&item); err != nil {
    handleError(err)
}

func (*Client) Add

func (c *Client) Add(item *Item) error

Stores the given item only if the server doesn't already hold data under the item.Key.

Returns ErrAlreadyExists error if the server already holds data under the item.Key.

func (*Client) Cas

func (c *Client) Cas(item *Item) error

Stores the given item only if item.Casid matches casid for the given item on the server.

Returns ErrCacheMiss if the server has no item with such a key. Returns ErrCasidMismatch if item on the server has other casid value.

func (*Client) Cget

func (c *Client) Cget(item *Item) error

Performs conditional get request for the given item.Key and item.Casid.

This is an extension to memcache protocol, so it isn't supported by the original memcache server.

Fills item.Value, item.Flags and item.Casid only on cache hit and only if the given casid doesn't match the casid on the server, i.e. if the server contains new value for the given key.

Returns ErrCacheMiss on cache miss. Returns ErrNotModified if the corresponding item on the server has the same casid (i.e. the item wasn't modified).

Client.Cget() is intended for reducing network bandwidth consumption in multi-level caches. It is modelled after HTTP cache validation approach with entity tags - see http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.11 .

func (*Client) CgetDe

func (c *Client) CgetDe(item *Item, graceDuration time.Duration) error

Combines functionality of Client.Cget() and Client.GetDe().

func (*Client) Delete

func (c *Client) Delete(key []byte) error

Deletes an item with the given key from memcache server.

Returns ErrCacheMiss if there were no item with such key on the server.

func (*Client) DeleteNowait

func (c *Client) DeleteNowait(key []byte)

The same as Client.Delete(), but doesn't wait for operation completion.

Do not modify slice pointed by key after passing to this function - it actually becomes an owner of this slice.

func (*Client) FlushAll

func (c *Client) FlushAll() error

Flushes all the items on the server.

func (*Client) FlushAllDelayed

func (c *Client) FlushAllDelayed(expiration time.Duration) error

Flushes all the items on the server after the given expiration delay.

func (*Client) FlushAllDelayedNowait

func (c *Client) FlushAllDelayedNowait(expiration time.Duration)

The same as Client.FlushAllDelayed(), but doesn't wait for operation completion.

func (*Client) FlushAllNowait

func (c *Client) FlushAllNowait()

The same as Client.FlushAll(), but doesn't wait for operation completion.

func (*Client) Get

func (c *Client) Get(item *Item) error

Obtains item.Value, item.Flags and item.Casid for the given item.Key.

Returns ErrCacheMiss on cache miss.

func (*Client) GetDe

func (c *Client) GetDe(item *Item, graceDuration time.Duration) error

Performs dogpile effect-aware get for the given item.Key.

This is an extension to memcache protocol, so it isn't supported by the original memcache server.

Returns ErrCacheMiss on cache miss. It is expected that the caller will create and store in the cache an item on cache miss during the given graceDuration interval.

func (*Client) GetMulti

func (c *Client) GetMulti(items []Item) error

Obtains multiple items associated with the the corresponding keys.

Sets Item.Value, Item.Flags and Item.Casid for each returned item. Doesn't modify Item.Value and Item.Flags for items missing on the server.

func (*Client) Set

func (c *Client) Set(item *Item) error

Stores the given item in the memcache server.

func (*Client) SetNowait

func (c *Client) SetNowait(item *Item)

The same as Client.Set(), but doesn't wait for operation completion.

Do not modify slices pointed by item.Key and item.Value after passing to this function - it actually becomes an owner of these slices.

func (*Client) Start

func (c *Client) Start()

Starts the given client.

No longer needed clients must be stopped via Client.Stop() call.

func (*Client) Stop

func (c *Client) Stop()

Stops the given client, which has been started via Client.Start() call.

type ClientConfig

type ClientConfig struct {
    // The number of simultaneous TCP connections to establish
    // to memcached server.
    // Optional parameter.
    //
    // The Client is able to squeeze out impossible from a single
    // connection by pipelining a ton of requests on it.
    // Multiple simultaneous connections may be required in the following
    // cases:
    //   * If memcached server delays incoming requests' execution.
    //     Since memcached protocol doesn't allow out-of-order requests'
    //     execution, a single slow request may delay execution of all
    //     the requests pipelined on the connection after it.
    //     Multiple concurrent connections may help in such a situation.
    //   * If memcached server runs on multi-CPU system, but uses a single
    //     CPU (thread) per connection.
    ConnectionsCount int

    // The maximum number of pending requests awaiting to be processed
    // by memcached server.
    // Optional parameter.
    MaxPendingRequestsCount int

    // The size in bytes of buffer used by the Client for reading responses
    // received from memcached per connection.
    // Optional parameter.
    ReadBufferSize int

    // The size in bytes of buffer used by the Client for writing requests
    // to be sent to memcached per connection.
    // Optional parameter.
    WriteBufferSize int

    // The size in bytes of OS-supplied read buffer per TCP connection.
    // Optional parameter.
    OSReadBufferSize int

    // The size in bytes of OS-supplied write buffer per TCP connection.
    // Optional parameter.
    OSWriteBufferSize int
}

Memcache client configuration. Can be passed to Client and DistributedClient.

type DistributedClient

type DistributedClient struct {
    ClientConfig
    // contains filtered or unexported fields
}

Memcache client, which can shard requests to multiple servers using consistent hashing.

Servers may be dynamically added and deleted at any time via AddServer() and DeleteServer() functions if the client is started via Start() call.

The client is goroutine-safe.

Usage:

c := DistributedClient{}
c.StartStatic([]string{"host1:11211", "host2:11211", "host3:11211"})
defer c.Stop()

item := Item{
    Key:   []byte("key"),
    Value: []byte("value"),
}
if err := c.Set(&item); err != nil {
    handleError(err)
}
if err := c.Get(&item); err != nil {
    handleError(err)
}

func (*DistributedClient) Add

func (c *DistributedClient) Add(item *Item) (err error)

See Client.Add().

func (*DistributedClient) AddServer

func (c *DistributedClient) AddServer(serverAddr string)

Dynamically adds the given server to the client.

serverAddr must be in the form 'host:port'.

This function may be called only if the client has been started via DistributedClient.Start() call, not via DistributedClient.StartStatic() call!

Added servers may be removed at any time via DistributedClient.DeleteServer() call.

func (*DistributedClient) Cas

func (c *DistributedClient) Cas(item *Item) (err error)

See Client.Cas()

func (*DistributedClient) Cget

func (c *DistributedClient) Cget(item *Item) (err error)

See Client.Cget().

func (*DistributedClient) CgetDe

func (c *DistributedClient) CgetDe(item *Item, graceDuration time.Duration) (err error)

See Client.CgetDe()

func (*DistributedClient) Delete

func (c *DistributedClient) Delete(key []byte) (err error)

See Client.Delete().

func (*DistributedClient) DeleteNowait

func (c *DistributedClient) DeleteNowait(key []byte)

See Client.DeleteNowait().

func (*DistributedClient) DeleteServer

func (c *DistributedClient) DeleteServer(serverAddr string)

Dynamically removes the given server from the client.

serverAddr must be in the form 'host:port'

This function may be called only if the client has been started via DistributedClient.Start() call, not via DistributedClient.StartStatic() call!

func (*DistributedClient) FlushAll

func (c *DistributedClient) FlushAll() (err error)

See Client.FlushAll().

func (*DistributedClient) FlushAllDelayed

func (c *DistributedClient) FlushAllDelayed(expiration time.Duration) (err error)

See Client.FlushAllDelayed().

func (*DistributedClient) FlushAllDelayedNowait

func (c *DistributedClient) FlushAllDelayedNowait(expiration time.Duration)

See Client.FlushAllDelayedNowait().

func (*DistributedClient) FlushAllNowait

func (c *DistributedClient) FlushAllNowait()

See Client.FlushAllNowait().

func (*DistributedClient) Get

func (c *DistributedClient) Get(item *Item) (err error)

See Client.Get().

func (*DistributedClient) GetDe

func (c *DistributedClient) GetDe(item *Item, graceDuration time.Duration) (err error)

See Client.GetDe().

func (*DistributedClient) GetMulti

func (c *DistributedClient) GetMulti(items []Item) (err error)

See Client.GetMulti().

func (*DistributedClient) Set

func (c *DistributedClient) Set(item *Item) (err error)

See Client.Set().

func (*DistributedClient) SetNowait

func (c *DistributedClient) SetNowait(item *Item)

See Client.SetNowait().

func (*DistributedClient) Start

func (c *DistributedClient) Start()

Starts distributed client with the ability to dynamically add/remove servers via DistributedClient.AddServer() and DistributedClient.DeleteServer().

Started client must be stopped via c.Stop() call when no longer needed!

Use DistributedClient.StartStatic() if you don't plan dynamically adding/removing servers to/from the client. The resulting static client may work faster than the dynamic client.

func (*DistributedClient) StartStatic

func (c *DistributedClient) StartStatic(serverAddrs []string)

Starts distributed client connected to the given memcache servers.

Each serverAddr must be in the form 'host:port'.

Started client must be stopped via DistributedClient.Stop() call when no longer needed.

Use DistributedClient.Start() if you plan dynamically adding/removing servers to/from the client. Note that the resuling dynamic client may work a bit slower than the static client.

func (*DistributedClient) Stop

func (c *DistributedClient) Stop()

Stops distributed client.

type Item

type Item struct {
    // Item's key.
    // Required parameter.
    Key []byte

    // Item's value.
    //
    // The Value is required in set()-type requests and isn't required in
    // get()-type requests.
    Value []byte

    // Expiration time for the item.
    // Zero means the item has no expiration time.
    //
    // The Expiration is used only in set()-type requests.
    Expiration time.Duration

    // An opaque value, which is passed to/from memcache.
    // Optional parameter.
    Flags uint32

    // This field is filled by get()-type requests and should be passed
    // to Cas() and Cget*() requests.
    Casid uint64
}

Memcache item.

type Memcacher

type Memcacher interface {
    Get(item *Item) error
    GetMulti(items []Item) error
    Set(item *Item) error
    SetNowait(item *Item)
    Delete(key []byte) error
    DeleteNowait(key []byte)
    Add(item *Item) error
    Cas(item *Item) error
    FlushAll() error
    FlushAllNowait()
    FlushAllDelayed(expiration time.Duration) error
    FlushAllDelayedNowait(expiration time.Duration)
}

Client, DistributedClient and CachingClient implement this interface.

type MemcacherDe

type MemcacherDe interface {
    Memcacher

    GetDe(item *Item, graceDuration time.Duration) error
}

Client, DistributedClient and CachingClient implement this interface.

type Server

type Server struct {
    // The underlying cache storage.
    // Required parameter.
    //
    // The cache must be initialized before passing it here.
    //
    // Currently ybc.Cache and ybc.Cluster may be passed here.
    Cache ybc.Cacher

    // TCP address to listen to. Must be in the form addr:port.
    // Required parameter.
    ListenAddr string

    // The size of buffer used for reading requests from clients
    // per each connection.
    // Optional parameter.
    ReadBufferSize int

    // The size of buffer used for writing responses to clients
    // per each connection.
    // Optional parameter.
    WriteBufferSize int

    // The size in bytes of OS-supplied read buffer per TCP connection.
    // Optional parameter.
    OSReadBufferSize int

    // The size in bytes of OS-supplied write buffer per TCP connection.
    // Optional parameter.
    OSWriteBufferSize int
    // contains filtered or unexported fields
}

Memcache server.

Usage:

cache := openCache()
defer cache.Close()

s := Server{
    Cache: cache,
    ListenAddr: ":11211",
}
if err := s.Serve(); err != nil {
    handleError(err)
}

func (*Server) Serve

func (s *Server) Serve() error

Start the server and waits until it is stopped via Server.Stop() call.

func (*Server) Start

func (s *Server) Start()

Starts the given server.

No longer needed servers must be stopped via Server.Stop() call.

func (*Server) Stop

func (s *Server) Stop()

Stops the server, which has been started via either Server.Start() or Server.Serve() calls.

Don't forget closing the Server.Cache, since the server doesn't close it automatically.

func (*Server) Wait

func (s *Server) Wait() error

Waits until the server is stopped.

Package memcache imports 14 packages (graph) and is imported by 2 packages. Updated 2014-11-14. Refresh now. Tools for package owners.