cmap

package module
v1.1.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2019 License: MIT Imports: 5 Imported by: 0

README

concurrent map Build Status

As explained here and here, the map type in Go doesn't support concurrent reads and writes. concurrent-map provides a high-performance solution to this by sharding the map with minimal time spent waiting for locks.

Prior to Go 1.9, there was no concurrent map implementation in the stdlib. In Go 1.9, sync.Map was introduced. The new sync.Map has a few key differences from this map. The stdlib sync.Map is designed for append-only scenarios. So if you want to use the map for something more like in-memory db, you might benefit from using our version. You can read more about it in the golang repo, for example here and here

usage

Import the package:

import (
	"github.com/orcaman/concurrent-map"
)

go get "github.com/orcaman/concurrent-map"

The package is now imported under the "cmap" namespace.

example


	// Create a new map.
	m := cmap.New()

	// Sets item within map, sets "bar" under key "foo"
	m.Set("foo", "bar")

	// Retrieve item from map.
	if tmp, ok := m.Get("foo"); ok {
		bar := tmp.(string)
	}

	// Removes item under key "foo"
	m.Remove("foo")

For more examples have a look at concurrent_map_test.go.

Running tests:

go test "github.com/orcaman/concurrent-map"

guidelines for contributing

Contributions are highly welcome. In order for a contribution to be merged, please follow these guidelines:

  • Open an issue and describe what you are after (fixing a bug, adding an enhancement, etc.).
  • According to the core team's feedback on the above mentioned issue, submit a pull request, describing the changes and linking to the issue.
  • New code must have test coverage.
  • If the code is about performance issues, you must include benchmarks in the process (either in the issue or in the PR).
  • In general, we would like to keep concurrent-map as simple as possible and as similar to the native map. Please keep this in mind when opening issues.

license

MIT (see LICENSE file)

Documentation

Index

Constants

This section is empty.

Variables

View Source
var SHARD_COUNT = 32

Functions

This section is empty.

Types

type ConcurrentMap

type ConcurrentMap []*ConcurrentMapShared

ConcurrentMap is a "thread" safe map of type string:Anything. To avoid lock bottlenecks this map is dived to several (SHARD_COUNT) map shards.

func New

func New() ConcurrentMap

New creates a new concurrent map.

func (ConcurrentMap) Count

func (m ConcurrentMap) Count() int

Count returns the number of elements within the map.

func (ConcurrentMap) Get

func (m ConcurrentMap) Get(key string) (interface{}, bool)

Get retrieves an element from map under given key.

func (ConcurrentMap) GetShard

func (m ConcurrentMap) GetShard(key string) *ConcurrentMapShared

GetShard returns shard under given key

func (ConcurrentMap) Has

func (m ConcurrentMap) Has(key string) bool

Has Looks up an item under specified key

func (ConcurrentMap) IsEmpty

func (m ConcurrentMap) IsEmpty() bool

IsEmpty checks if map is empty.

func (ConcurrentMap) Items

func (m ConcurrentMap) Items() map[string]interface{}

Items returns all items as map[string]interface{}

func (ConcurrentMap) Iter deprecated

func (m ConcurrentMap) Iter() <-chan Tuple

Iter returns an iterator which could be used in a for range loop.

Deprecated: using IterBuffered() will get a better performence

func (ConcurrentMap) IterBuffered

func (m ConcurrentMap) IterBuffered() <-chan Tuple

IterBuffered returns a buffered iterator which could be used in a for range loop.

func (ConcurrentMap) IterCb

func (m ConcurrentMap) IterCb(fn IterCb)

IterCb Callback based iterator, cheapest way to read all elements in a map.

func (ConcurrentMap) Keys

func (m ConcurrentMap) Keys() []string

Keys returns all keys as []string

func (ConcurrentMap) MSet

func (m ConcurrentMap) MSet(data map[string]interface{})

MSet sets the given map to current maps.

func (ConcurrentMap) MarshalJSON

func (m ConcurrentMap) MarshalJSON() ([]byte, error)

MarshalJSON Reviles ConcurrentMap "private" variables to json marshal.

func (ConcurrentMap) Pop

func (m ConcurrentMap) Pop(key string) (v interface{}, exists bool)

Pop removes an element from the map and returns it

func (ConcurrentMap) Remove

func (m ConcurrentMap) Remove(key string)

Remove removes an element from the map.

func (ConcurrentMap) RemoveCb

func (m ConcurrentMap) RemoveCb(key string, cb RemoveCb) bool

RemoveCb locks the shard containing the key, retrieves its current value and calls the callback with those params If callback returns true and element exists, it will remove it from the map Returns the value returned by the callback (even if element was not present in the map)

func (ConcurrentMap) RemoveNoLock

func (m ConcurrentMap) RemoveNoLock(key string)

RemoveNoLock removes an element from the map without locking shard. If already lock for outer shard, locking inside will waste kernel resource

func (ConcurrentMap) Set

func (m ConcurrentMap) Set(key string, value interface{})

Set sets the given value under the specified key.

func (ConcurrentMap) SetIfAbsent

func (m ConcurrentMap) SetIfAbsent(key string, value interface{}) bool

SetIfAbsent Sets the given value under the specified key if no value was associated with it.

func (ConcurrentMap) SetNoLock

func (m ConcurrentMap) SetNoLock(key string, value interface{})

SetNoLock sets the given value under the specified key without locking shard. If already lock for outer shard, locking inside will waste kernel resource

func (ConcurrentMap) Upsert

func (m ConcurrentMap) Upsert(key string, value interface{}, cb UpsertCb) (res interface{})

Upsert is Insert or Update - updates existing element or inserts a new one using UpsertCb

type ConcurrentMapShared

type ConcurrentMapShared struct {
	sync.RWMutex // Read Write mutex, guards access to internal map.
	// contains filtered or unexported fields
}

ConcurrentMapShared is a "thread" safe string to anything map.

type IterCb

type IterCb func(key string, v interface{})

IterCb Iterator callback,called for every key,value found in maps. RLock is held for all calls for a given shard therefore callback sess consistent view of a shard, but not across the shards

type NestedCMap added in v1.0.1

type NestedCMap struct {
	// contains filtered or unexported fields
}

NestedCMap Cmap with NestedGSet as values CMap<NestedGSet>

func NewNestedCMap added in v1.0.1

func NewNestedCMap() *NestedCMap

NewNestedCMap return NestedCMap

func (*NestedCMap) Count added in v1.1.1

func (m *NestedCMap) Count() int

Count returns number of elements

func (*NestedCMap) DeleteInnerKey added in v1.1.4

func (m *NestedCMap) DeleteInnerKey(key, innerKey string)

DeleteInnerKey delete one innerkey in inner NestedGSet for key lock shard for outer key.

Clear empty NestedGSet

func (*NestedCMap) DeleteInnerKeyNoLock added in v1.1.4

func (m *NestedCMap) DeleteInnerKeyNoLock(key, innerKey string)

DeleteInnerKeyNoLock delete one innerkey in inner NestedGSet for key

Clear empty NestedGSet

func (*NestedCMap) DeleteInnerKeyVal added in v1.1.4

func (m *NestedCMap) DeleteInnerKeyVal(key, innerKey, innerVal string)

DeleteInnerKeyVal delete innerKey in inner NestedGSet for key CMap<key, NestedGSet<InnerKey[vals,...]>>

func (*NestedCMap) DeleteInnerKeyValNoLock added in v1.1.4

func (m *NestedCMap) DeleteInnerKeyValNoLock(key, innerKey, innerVal string)

DeleteInnerKeyValNoLock delete innerKey in inner NestedGSet for key lock shard for outer key CMap<key, NestedGSet<InnerKey[vals,...]>>

func (*NestedCMap) DeleteInnerKeys added in v1.1.4

func (m *NestedCMap) DeleteInnerKeys(key string, innerKeys []string)

DeleteInnerKeys delete list of innerkeys in inner NestedGSet for key lock shard for outer key

func (*NestedCMap) GGetInnerKeysNoLock added in v1.1.5

func (m *NestedCMap) GGetInnerKeysNoLock(key string) ([]string, bool)

GGetInnerKeysNoLock Get list of innerKeys in inner NestedGset

func (*NestedCMap) GetInnerKeys added in v1.1.5

func (m *NestedCMap) GetInnerKeys(key string) ([]string, bool)

GetInnerKeys Get list of inner keys in inner NestedGset <Keys, <[k1]val1, [k2]val2>> get k1,k2, ...

func (*NestedCMap) GetInnerValues added in v1.1.5

func (m *NestedCMap) GetInnerValues(key, innerKey string) ([]string, bool)

GetInnerValues Get list of inner values in inner NestedGset CMap<key, NestedGSet<InnerKey[vals,...]>> get [vals,...] ...

func (*NestedCMap) GetInnerValuesNoLock added in v1.1.5

func (m *NestedCMap) GetInnerValuesNoLock(key, innerKey string) ([]string, bool)

GetInnerValuesNoLock Get list of inner values in inner NestedGset CMap<key, NestedGSet<InnerKey[vals,...]>> get [vals,...] ...

func (*NestedCMap) Has added in v1.1.1

func (m *NestedCMap) Has(key string) bool

Has return true if cmap has key

func (*NestedCMap) IsEmpty added in v1.1.1

func (m *NestedCMap) IsEmpty() bool

IsEmpty return true if cmap empty

func (*NestedCMap) Keys added in v1.1.1

func (m *NestedCMap) Keys() []string

Keys returns all keys in cmap

func (*NestedCMap) Pop added in v1.1.3

func (m *NestedCMap) Pop(key string) (interface{}, bool)

Pop key in cmap and return (value, isexist bool)

func (*NestedCMap) Remove added in v1.1.1

func (m *NestedCMap) Remove(key string)

Remove key in cmap

func (*NestedCMap) Set added in v1.1.3

func (m *NestedCMap) Set(key string, value interface{})

Set key in cmap

func (*NestedCMap) SetInnerKey added in v1.1.4

func (m *NestedCMap) SetInnerKey(key, innerKey string)

SetInnerKey set inner key into inner CMap <key, ListofKeys> lock shard for outer key CMap<key, NestedGSet<InnerKey[vals,...]>>

func (*NestedCMap) SetInnerKeyVal added in v1.1.4

func (m *NestedCMap) SetInnerKeyVal(key, innerKey, innerVal string)

SetInnerKeyVal set inner key into inner NestedGSet (CMapOuter,cmap inner, gset (values)) lock shard for outer key CMap<key, NestedGSet<InnerKey[vals,...]>>

func (*NestedCMap) SetInnerKeyValNoLock added in v1.1.5

func (m *NestedCMap) SetInnerKeyValNoLock(key, innerKey, innerVal string)

SetInnerKeyValNoLock set inner key into inner NestedGSet (CMapOuter,cmap inner, gset (values)) CMap<key, NestedGSet<InnerKey[vals,...]>>

func (*NestedCMap) SetMultiInnerKeys added in v1.1.5

func (m *NestedCMap) SetMultiInnerKeys(key string, innerKeys []string)

SetMultiInnerKeys set lists of inner keys into inner NestedGSet lock shard for outer key CMap<key, NestedGSet<InnerKey[vals,...]>>

type NestedGSet added in v1.0.1

type NestedGSet struct {
	// contains filtered or unexported fields
}

NestedGSet CMap(<Gset>) ... key<set1>,key<set2>

func NewNestedGSet added in v1.0.1

func NewNestedGSet() *NestedGSet

NewNestedGSet CMap(<Gset>) key<set1>,key<set2>

func (*NestedGSet) Count added in v1.1.1

func (m *NestedGSet) Count() int

Count returns number of elements

func (*NestedGSet) DeleteMultipleStrValues added in v1.1.4

func (m *NestedGSet) DeleteMultipleStrValues(key string, values []string)

DeleteMultipleStrValues (outer shard lock) delete list of values in inner Gset for key in cmap lock shard for outer key

func (*NestedGSet) DeleteMultipleValues added in v1.1.4

func (m *NestedGSet) DeleteMultipleValues(key string, values []interface{})

DeleteMultipleValues (outer shard lock) delete list of values in inner Gset for key in cmap lock shard for outer key

func (*NestedGSet) DeleteMultipleValuesNoLock added in v1.1.4

func (m *NestedGSet) DeleteMultipleValuesNoLock(key string, values []interface{})

DeleteMultipleValuesNoLock delete list of values in inner Gset for key in cmap lock shard for outer key

func (*NestedGSet) DeleteValue added in v1.1.4

func (m *NestedGSet) DeleteValue(key string, value interface{})

DeleteValue (locked) delete one value in inner gset for key Clear empty cmap CMap<key, GSet[val1,val2]>

func (*NestedGSet) DeleteValueNoLock added in v1.1.4

func (m *NestedGSet) DeleteValueNoLock(key string, value interface{})

DeleteValueNoLock delete one value in inner GSet of outer key

Clear empty gset

CMap<key, GSet[val1,val2]>

func (*NestedGSet) GetGSet added in v1.0.1

func (m *NestedGSet) GetGSet(key string) (*mapset.Set, bool)

GetGSet returns inner gset in cmap for key

func (*NestedGSet) GetGSetNoLock added in v1.0.1

func (m *NestedGSet) GetGSetNoLock(key string) (*mapset.Set, bool)

GetGSetNoLock returns inner gset in cmap for key

func (*NestedGSet) GetStrValues added in v1.1.4

func (m *NestedGSet) GetStrValues(key string) ([]string, bool)

GetStrValues Get list of values in inner gset for key CMap<[<key, GSet1[val1,val2]>, <key2, GSet2[val1,val2]>,... ] > , get val1, val2 for key

func (*NestedGSet) GetStrValuesNoLock added in v1.1.4

func (m *NestedGSet) GetStrValuesNoLock(key string) ([]string, bool)

GetStrValuesNoLock Get list of values in inner gset for key CMap<[<key, GSet1[val1,val2]>, <key2, GSet2[val1,val2]>,... ] > , get val1, val2 for key

func (*NestedGSet) GetValues added in v1.1.4

func (m *NestedGSet) GetValues(key string) ([]interface{}, bool)

GetValues Get list of values in inner gset for key CMap<[<key, GSet1[val1,val2]>, <key2, GSet2[val1,val2]>,... ] > , get val1, val2 for key

func (*NestedGSet) GetValuesNoLock added in v1.1.4

func (m *NestedGSet) GetValuesNoLock(key string) ([]interface{}, bool)

GetValuesNoLock Get list of values in inner gset for key CMap<[<key, GSet1[val1,val2]>, <key2, GSet2[val1,val2]>,... ] > , get val1, val2 for key

func (*NestedGSet) Has added in v1.1.1

func (m *NestedGSet) Has(key string) bool

Has return true if cmap has key

func (*NestedGSet) HasValue added in v1.1.4

func (m *NestedGSet) HasValue(key string, value interface{}) bool

HasValue (locked) check if inner gset has value CMap<key, GSet[val1,val2]>

func (*NestedGSet) HasValueNoLock added in v1.1.4

func (m *NestedGSet) HasValueNoLock(key string, value interface{}) bool

HasValueNoLock check if inner gset has value CMap<key, GSet[val1,val2]>

func (*NestedGSet) IsEmpty added in v1.1.1

func (m *NestedGSet) IsEmpty() bool

IsEmpty return true if cmap empty

func (*NestedGSet) Keys added in v1.1.1

func (m *NestedGSet) Keys() []string

Keys returns all keys in cmap

func (*NestedGSet) Pop added in v1.1.3

func (m *NestedGSet) Pop(key string) (interface{}, bool)

Pop key in cmap and return (value, isexist bool)

func (*NestedGSet) PopGSet added in v1.1.4

func (m *NestedGSet) PopGSet(key string) (*mapset.Set, bool)

PopGSet deletes key and returns inner gset in cmap for key

func (*NestedGSet) PopGSetNoLock added in v1.1.4

func (m *NestedGSet) PopGSetNoLock(key string) (*mapset.Set, bool)

PopGSetNoLock deletes key and returns inner gset in cmap for key

func (*NestedGSet) PopStrValues added in v1.1.5

func (m *NestedGSet) PopStrValues(key string) ([]string, bool)

PopStrValues deletes key and returns []string values of inner gset in cmap for key

func (*NestedGSet) PopStrValuesNoLock added in v1.1.5

func (m *NestedGSet) PopStrValuesNoLock(key string) ([]string, bool)

PopStrValuesNoLock deletes key and returns []string values of inner gset in cmap for key

func (*NestedGSet) Remove added in v1.1.1

func (m *NestedGSet) Remove(key string)

Remove key in cmap

func (*NestedGSet) Set added in v1.1.3

func (m *NestedGSet) Set(key string, value interface{})

Set key in cmap

func (*NestedGSet) SetGSet added in v1.1.5

func (m *NestedGSet) SetGSet(key string, value *mapset.Set)

SetGSet key in cmap

func (*NestedGSet) SetMultiStrValues added in v1.1.4

func (m *NestedGSet) SetMultiStrValues(key string, values []string)

SetMultiStrValues set list of values into inner gset lock shard for outer key CMap<key, GSet[val1,val2]>

func (*NestedGSet) SetMultiStrValuesNoLock added in v1.1.4

func (m *NestedGSet) SetMultiStrValuesNoLock(key string, values []string)

SetMultiStrValuesNoLock set list of values into inner gset lock shard for outer key CMap<key, GSet[val1,val2]>

func (*NestedGSet) SetMultiValues added in v1.1.4

func (m *NestedGSet) SetMultiValues(key string, values []interface{})

SetMultiValues set list of values into inner gset lock shard for outer key CMap<key, GSet[val1,val2]>

func (*NestedGSet) SetMultiValuesNoLock added in v1.1.4

func (m *NestedGSet) SetMultiValuesNoLock(key string, values []interface{})

SetMultiValuesNoLock set list of values into inner gset lock shard for outer key CMap<key, GSet[val1,val2]>

func (*NestedGSet) SetValue added in v1.1.4

func (m *NestedGSet) SetValue(key string, value interface{}) bool

SetValue set value into inner set <key(Cmap), interkey(gset)> lock shard for outer key false if gset already has key CMap<key, GSet[val1,val2]>

func (*NestedGSet) SetValueNoLock added in v1.1.4

func (m *NestedGSet) SetValueNoLock(key string, value interface{}) bool

SetValueNoLock set inner key into inner set <key(Cmap), interkey(gset)> false if gset already has key CMap<key, GSet[val1,val2]>

type NestedQueue added in v1.0.1

type NestedQueue struct {
	// contains filtered or unexported fields
}

NestedQueue Cmap(key,<queue>)

func NewNestedQueue added in v1.0.1

func NewNestedQueue() *NestedQueue

NewNestedQueue returns Cmap(key,<queue>)

func (*NestedQueue) Count added in v1.1.1

func (m *NestedQueue) Count() int

Count returns number of elements

func (*NestedQueue) Has added in v1.1.1

func (m *NestedQueue) Has(key string) bool

Has return true if cmap has key

func (*NestedQueue) InsertpQueue added in v1.0.1

func (m *NestedQueue) InsertpQueue(key string, value interface{}, priority float64) bool

InsertpQueue key, ({topic, payloads}, ...), priority

func (*NestedQueue) IsEmpty added in v1.1.1

func (m *NestedQueue) IsEmpty() bool

IsEmpty return true if cmap empty

func (*NestedQueue) Keys added in v1.1.1

func (m *NestedQueue) Keys() []string

Keys returns all keys in cmap

func (*NestedQueue) Pop added in v1.1.3

func (m *NestedQueue) Pop(key string) (interface{}, bool)

Pop key in cmap and return (value, isexist bool)

func (*NestedQueue) PopQueue added in v1.0.1

func (m *NestedQueue) PopQueue(key string) (interface{}, bool)

PopQueue pop queue for key

func (*NestedQueue) PopQueueBytes added in v1.0.1

func (m *NestedQueue) PopQueueBytes(key string) ([]byte, bool)

PopQueueBytes pop queue for key

func (*NestedQueue) PopQueueStr added in v1.0.1

func (m *NestedQueue) PopQueueStr(key string) (string, bool)

PopQueueStr pop queue for key

func (*NestedQueue) Remove added in v1.1.1

func (m *NestedQueue) Remove(key string)

Remove key in cmap

func (*NestedQueue) Set added in v1.1.3

func (m *NestedQueue) Set(key string, value interface{})

Set key in cmap

type RemoveCb

type RemoveCb func(key string, v interface{}, exists bool) bool

RemoveCb is a callback executed in a map.RemoveCb() call, while Lock is held If returns true, the element will be removed from the map

type Tuple

type Tuple struct {
	Key string
	Val interface{}
}

Tuple is used by the Iter & IterBuffered functions to wrap two variables together over a channel,

type Uint64Map added in v1.1.4

type Uint64Map struct {
	// contains filtered or unexported fields
}

Uint64Map uses atomic uint64 as value for key CMap(key string, value uint64)

func NewUint64Map added in v1.1.4

func NewUint64Map() *Uint64Map

NewUint64Map CMap(key string, value uint64)

func (*Uint64Map) Count added in v1.1.5

func (m *Uint64Map) Count() int

Count returns number of elements

func (*Uint64Map) DecrementOrDeleteKey added in v1.1.4

func (m *Uint64Map) DecrementOrDeleteKey(key string) bool

DecrementOrDeleteKey decrement value of key by one or delete one key in CMap if key count is zero

func (*Uint64Map) DecrementOrDeleteKeyNoLock added in v1.1.4

func (m *Uint64Map) DecrementOrDeleteKeyNoLock(key string) bool

DecrementOrDeleteKeyNoLock decrement value of key by one or delete one key in CMap if key count is zero

func (*Uint64Map) DecrementOrDeleteMultiKeys added in v1.1.4

func (m *Uint64Map) DecrementOrDeleteMultiKeys(keys []string) []string

DecrementOrDeleteMultiKeys decrement value of keys by one or delete one key in CMap if key count is zero

func (*Uint64Map) DecrementOrDeleteMultiKeysNoLock added in v1.1.4

func (m *Uint64Map) DecrementOrDeleteMultiKeysNoLock(keys []string) []string

DecrementOrDeleteMultiKeysNoLock decrement value of keys by one or delete one key in CMap if key count is zero

func (*Uint64Map) Has added in v1.1.5

func (m *Uint64Map) Has(key string) bool

Has return true if cmap has key

func (*Uint64Map) InsertOrIncrementKey added in v1.1.4

func (m *Uint64Map) InsertOrIncrementKey(key string) uint64

InsertOrIncrementKey set key into CMap or increment value of key if it exists lock shard for outer key

func (*Uint64Map) InsertOrIncrementKeyNoLock added in v1.1.4

func (m *Uint64Map) InsertOrIncrementKeyNoLock(key string) uint64

InsertOrIncrementKeyNoLock set key into CMap or increment value of key if it exists

func (*Uint64Map) InsertOrIncrementMultiKeys added in v1.1.4

func (m *Uint64Map) InsertOrIncrementMultiKeys(keys []string) []uint64

InsertOrIncrementMultiKeys list keys []string into CMap or increment value of key if it exists lock shard for outer key

func (*Uint64Map) InsertOrIncrementMultiKeysNoLock added in v1.1.4

func (m *Uint64Map) InsertOrIncrementMultiKeysNoLock(keys []string) []uint64

InsertOrIncrementMultiKeysNoLock set list of keys <ListofKeys> or increment value of innerkey if it exists

func (*Uint64Map) IsEmpty added in v1.1.5

func (m *Uint64Map) IsEmpty() bool

IsEmpty return true if cmap empty

func (*Uint64Map) Keys added in v1.1.5

func (m *Uint64Map) Keys() []string

Keys returns all keys in cmap

func (*Uint64Map) Pop added in v1.1.5

func (m *Uint64Map) Pop(key string) (interface{}, bool)

Pop key in cmap and return (value, isexist bool)

func (*Uint64Map) Remove added in v1.1.5

func (m *Uint64Map) Remove(key string)

Remove key in cmap

func (*Uint64Map) Set added in v1.1.5

func (m *Uint64Map) Set(key string, value interface{})

Set key in cmap

type UpsertCb

type UpsertCb func(exist bool, valueInMap interface{}, newValue interface{}) interface{}

UpsertCb Callback to return new element to be inserted into the map It is called while lock is held, therefore it MUST NOT try to access other keys in same map, as it can lead to deadlock since Go sync.RWLock is not reentrant

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL