README
¶
fastcache - fast thread-safe inmemory cache for big number of entries in Go
Features
- Fast. Performance scales on multi-core CPUs. See benchmark results below.
- Thread-safe. Concurrent goroutines may read and write into a single cache instance.
- The fastcache is designed for storing big number of entries without GC overhead.
- Fastcache automatically evicts old entries when reaching the maximum cache size set on its creation.
- Simple API.
- Simple source code.
- Cache may be saved to file and loaded from file.
- Works on Google AppEngine.
Benchmarks
Fastcache performance is compared with BigCache, standard Go map
and sync.Map.
GOMAXPROCS=4 go test github.com/VictoriaMetrics/fastcache -bench='Set|Get' -benchtime=10s
goos: linux
goarch: amd64
pkg: github.com/VictoriaMetrics/fastcache
BenchmarkBigCacheSet-4 2000 10566656 ns/op 6.20 MB/s 4660369 B/op 6 allocs/op
BenchmarkBigCacheGet-4 2000 6902694 ns/op 9.49 MB/s 684169 B/op 131076 allocs/op
BenchmarkBigCacheSetGet-4 1000 17579118 ns/op 7.46 MB/s 5046744 B/op 131083 allocs/op
BenchmarkCacheSet-4 5000 3808874 ns/op 17.21 MB/s 1142 B/op 2 allocs/op
BenchmarkCacheGet-4 5000 3293849 ns/op 19.90 MB/s 1140 B/op 2 allocs/op
BenchmarkCacheSetGet-4 2000 8456061 ns/op 15.50 MB/s 2857 B/op 5 allocs/op
BenchmarkStdMapSet-4 2000 10559382 ns/op 6.21 MB/s 268413 B/op 65537 allocs/op
BenchmarkStdMapGet-4 5000 2687404 ns/op 24.39 MB/s 2558 B/op 13 allocs/op
BenchmarkStdMapSetGet-4 100 154641257 ns/op 0.85 MB/s 387405 B/op 65558 allocs/op
BenchmarkSyncMapSet-4 500 24703219 ns/op 2.65 MB/s 3426543 B/op 262411 allocs/op
BenchmarkSyncMapGet-4 5000 2265892 ns/op 28.92 MB/s 2545 B/op 79 allocs/op
BenchmarkSyncMapSetGet-4 1000 14595535 ns/op 8.98 MB/s 3417190 B/op 262277 allocs/op
MB/s column here actually means millions of operations per second.
As you can see, fastcache is faster than the BigCache in all the cases.
fastcache is faster than the standard Go map and sync.Map on workloads
with inserts.
Limitations
- Keys and values must be byte slices. Other types must be marshaled before storing them in the cache.
- Big entries with sizes exceeding 64KB must be stored via distinct API.
- There is no cache expiration. Entries are evicted from the cache only on cache size overflow. Entry deadline may be stored inside the value in order to implement cache expiration.
Architecture details
The cache uses ideas from BigCache:
- The cache consists of many buckets, each with its own lock. This helps scaling the performance on multi-core CPUs, since multiple CPUs may concurrently access distinct buckets.
- Each bucket consists of a
hash(key) -> (key, value) positionmap and 64KB-sized byte slices (chunks) holding encoded(key, value)entries. Each bucket contains onlyO(chunksCount)pointers. For instance, 64GB cache would contain ~1M pointers, while similarly-sizedmap[string][]bytewould contain ~1B pointers for short keys and values. This would lead to huge GC overhead.
64KB-sized chunks reduce memory fragmentation and the total memory usage comparing
to a single big chunk per bucket.
Chunks are allocated off-heap if possible. This reduces total memory usage because
GC collects unused memory more frequently without the need in GOGC tweaking.
Users
Fastcachehas been extracted from VictoriaMetrics sources. See this article for more info aboutVictoriaMetrics.
FAQ
What is the difference between fastcache and other similar caches like BigCache or FreeCache?
Fastcacheis faster. See benchmark results above.Fastcacheuses less memory due to lower heap fragmentation. This allows saving many GBs of memory on multi-GB caches.FastcacheAPI is simpler. The API is designed to be used in zero-allocation mode.
Why fastcache doesn't support cache expiration?
Because we don't need cache expiration in VictoriaMetrics.
Cached entries inside VictoriaMetrics never expire. They are automatically evicted on cache size overflow.
It is easy to implement cache expiration on top of fastcache by caching values
with marshaled deadlines and verifying deadlines after reading these values
from the cache.
Why fastcache doesn't support advanced features such as thundering herd protection or callbacks on entries' eviction?
Because these features would complicate the code and would make it slower.
Fastcache source code is simple - just copy-paste it and implement the feature you want
on top of it.
Documentation
¶
Overview ¶
Package fastcache implements fast in-memory cache.
The package has been extracted from https://victoriametrics.com/
Index ¶
- type BigStats
- type Cache
- func (c *Cache) Del(k []byte)
- func (c *Cache) Get(dst, k []byte) []byte
- func (c *Cache) GetBig(dst, k []byte) (r []byte)
- func (c *Cache) Has(k []byte) bool
- func (c *Cache) HasGet(dst, k []byte) ([]byte, bool)
- func (c *Cache) Reset()
- func (c *Cache) SaveToFile(filePath string) error
- func (c *Cache) SaveToFileConcurrent(filePath string, concurrency int) error
- func (c *Cache) Set(k, v []byte)
- func (c *Cache) SetBig(k, v []byte)
- func (c *Cache) UpdateStats(s *Stats)
- type Stats
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type BigStats ¶ added in v1.4.0
type BigStats struct {
// GetBigCalls is the number of GetBig calls.
GetBigCalls uint64
// SetBigCalls is the number of SetBig calls.
SetBigCalls uint64
// TooBigKeyErrors is the number of calls to SetBig with too big key.
TooBigKeyErrors uint64
// InvalidMetavalueErrors is the number of calls to GetBig resulting
// to invalid metavalue.
InvalidMetavalueErrors uint64
// InvalidValueLenErrors is the number of calls to GetBig resulting
// to a chunk with invalid length.
InvalidValueLenErrors uint64
// InvalidValueHashErrors is the number of calls to GetBig resulting
// to a chunk with invalid hash value.
InvalidValueHashErrors uint64
}
BigStats contains stats for GetBig/SetBig methods.
type Cache ¶
type Cache struct {
// contains filtered or unexported fields
}
Cache is a fast thread-safe inmemory cache optimized for big number of entries.
It has much lower impact on GC comparing to a simple `map[string][]byte`.
Use New or LoadFromFile* for creating new cache instance. Concurrent goroutines may call any Cache methods on the same cache instance.
Call Reset when the cache is no longer needed. This reclaims the allocated memory.
func LoadFromFile ¶ added in v1.1.0
LoadFromFile loads cache data from the given filePath.
See SaveToFile* for saving cache data to file.
func LoadFromFileOrNew ¶ added in v1.1.0
LoadFromFileOrNew tries loading cache data from the given filePath.
The function falls back to creating new cache with the given maxBytes capacity if error occurs during loading the cache from file.
func New ¶
New returns new cache with the given maxBytes capacity in bytes.
maxBytes must be smaller than the available RAM size for the app, since the cache holds data in memory.
If maxBytes is less than 32MB, then the minimum cache capacity is 32MB.
func (*Cache) Del ¶
Del deletes value for the given k from the cache.
k contents may be modified after returning from Del.
func (*Cache) Get ¶
Get appends value by the key k to dst and returns the result.
Get allocates new byte slice for the returned value if dst is nil.
Get returns only values stored in c via Set.
k contents may be modified after returning from Get.
func (*Cache) GetBig ¶ added in v1.4.0
GetBig searches for the value for the given k, appends it to dst and returns the result.
GetBig returns only values stored via SetBig. It doesn't work with values stored via other methods.
k contents may be modified after returning from GetBig.
func (*Cache) Has ¶ added in v1.5.0
Has returns true if entry for the given key k exists in the cache.
func (*Cache) HasGet ¶ added in v1.5.3
HasGet works identically to Get, but also returns whether the given key exists in the cache. This method makes it possible to differentiate between a stored nil/empty value versus and non-existing value.
func (*Cache) SaveToFile ¶ added in v1.1.0
SaveToFile atomically saves cache data to the given filePath using a single CPU core.
SaveToFile may be called concurrently with other operations on the cache.
The saved data may be loaded with LoadFromFile*.
See also SaveToFileConcurrent for faster saving to file.
func (*Cache) SaveToFileConcurrent ¶ added in v1.3.0
SaveToFileConcurrent saves cache data to the given filePath using concurrency CPU cores.
SaveToFileConcurrent may be called concurrently with other operations on the cache.
The saved data may be loaded with LoadFromFile*.
See also SaveToFile.
func (*Cache) Set ¶
Set stores (k, v) in the cache.
Get must be used for reading the stored entry.
The stored entry may be evicted at any time either due to cache overflow or due to unlikely hash collision. Pass higher maxBytes value to New if the added items disappear frequently.
(k, v) entries with summary size exceeding 64KB aren't stored in the cache. SetBig can be used for storing entries exceeding 64KB.
k and v contents may be modified after returning from Set.
func (*Cache) SetBig ¶ added in v1.4.0
SetBig sets (k, v) to c where len(v) may exceed 64KB.
GetBig must be used for reading stored values.
The stored entry may be evicted at any time either due to cache overflow or due to unlikely hash collision. Pass higher maxBytes value to New if the added items disappear frequently.
It is safe to store entries smaller than 64KB with SetBig.
k and v contents may be modified after returning from SetBig.
func (*Cache) UpdateStats ¶
UpdateStats adds cache stats to s.
Call s.Reset before calling UpdateStats if s is re-used.
type Stats ¶
type Stats struct {
// GetCalls is the number of Get calls.
GetCalls uint64
// SetCalls is the number of Set calls.
SetCalls uint64
// Misses is the number of cache misses.
Misses uint64
// Collisions is the number of cache collisions.
//
// Usually the number of collisions must be close to zero.
// High number of collisions suggest something wrong with cache.
Collisions uint64
// Corruptions is the number of detected corruptions of the cache.
//
// Corruptions may occur when corrupted cache is loaded from file.
Corruptions uint64
// EntriesCount is the current number of entries in the cache.
EntriesCount uint64
// BytesSize is the current size of the cache in bytes.
BytesSize uint64
// MaxBytesSize is the maximum allowed size of the cache in bytes (aka capacity).
MaxBytesSize uint64
// BigStats contains stats for GetBig/SetBig methods.
BigStats
}
Stats represents cache stats.
Use Cache.UpdateStats for obtaining fresh stats from the cache.
Source Files