godis

package module
v0.0.0-...-5990362 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 29, 2022 License: GPL-3.0 Imports: 23 Imported by: 1

README

Godis

license Build Status Coverage Status Go Report Card Go Reference
Mentioned in Awesome Go

中文版

Godis is a golang implementation of Redis Server, which intents to provide an example of writing a high concurrent middleware using golang.

Key Features:

  • Support string, list, hash, set, sorted set
  • TTL
  • Publish/Subscribe
  • GEO
  • AOF and AOF Rewrite
  • MULTI Commands Transaction is Atomic and Isolated. If any errors are encountered during execution, godis will rollback the executed commands
  • Server-side Cluster which is transparent to client. You can connect to any node in the cluster to access all data in the cluster.
    • MSET, DEL command is supported and atomically executed in cluster mode
    • Rename, RenameNX command is supported within slot in cluster mode
    • MULTI Commands Transaction is supported within slot in cluster mode
  • Concurrent Core, so you don't have to worry about your commands blocking the server too much.

If you could read Chinese, you can find more details in My Blog.

Get Started

You can get runnable program in the releases of this repository, which supports Linux and Darwin system.

./godis-darwin
./godis-linux

You could use redis-cli or other redis client to connect godis server, which listens on 0.0.0.0:6399 on default mode.

The program will try to read config file path from environment variable CONFIG.

If environment variable is not set, then the program try to read redis.conf in the working directory.

If there is no such file, then the program will run with default config.

cluster mode

Godis can work in cluster mode, please append following lines to redis.conf file

peers localhost:7379,localhost:7389 // other node in cluster
self  localhost:6399 // self address

We provide node1.conf and node2.conf for demonstration. use following command line to start a two-node-cluster:

CONFIG=node1.conf ./godis-darwin &
CONFIG=node2.conf ./godis-darwin &

Connect to a node in the cluster to access all data in the cluster:

redis-cli -p 6399

Supported Commands

See: commands.md

Benchmark

Environment:

Go version：1.16

System: macOS Catalina 10.15.7

CPU: 2.6GHz 6-Core Intel Core i7

Memory: 16 GB 2667 MHz DDR4

Performance report by redis-benchmark:

PING_INLINE: 87260.03 requests per second
PING_BULK: 89206.06 requests per second
SET: 85034.02 requests per second
GET: 87565.68 requests per second
INCR: 91157.70 requests per second
LPUSH: 90334.23 requests per second
RPUSH: 90334.23 requests per second
LPOP: 90334.23 requests per second
RPOP: 90415.91 requests per second
SADD: 90909.09 requests per second
HSET: 84104.29 requests per second
SPOP: 82918.74 requests per second
LPUSH (needed to benchmark LRANGE): 78247.26 requests per second
LRANGE_100 (first 100 elements): 26406.13 requests per second
LRANGE_300 (first 300 elements): 11307.10 requests per second
LRANGE_500 (first 450 elements): 7968.13 requests per second
LRANGE_600 (first 600 elements): 6092.73 requests per second
MSET (10 keys): 65487.89 requests per second

Todo List

  • Multi Command
  • Watch Command and CAS support
  • Stream support
  • RDB file loader
  • Master-Slave mode
  • Sentinel

Read My Code

If you want to read my code in this repository, here is a simple guidance.

  • github.com/tigbox/godis/cmd: only the entry point
  • github.com/tigbox/godis/config: config parser
  • github.com/tigbox/godis/interface: some interface definitions
  • github.com/tigbox/godis/lib: some utils, such as logger, sync utils and wildcard

I suggest focusing on the following directories:

  • github.com/tigbox/godis/tcp: the tcp server
  • github.com/tigbox/godis/redis: the redis protocol parser
  • github.com/tigbox/godis/datastruct: the implements of data structures
    • dict: a concurrent hash map
    • list: a linked list
    • lock: it is used to lock keys to ensure thread safety
    • set: a hash set based on map
    • sortedset: a sorted set implements based on skiplist
  • github.com/tigbox/godis: the core of storage engine
    • db.go: the basement of database
    • exec.go: the gateway of database
    • router.go: the command table
    • keys.go: handlers for keys commands
    • string.go: handlers for string commands
    • list.go: handlers for list commands
    • hash.go: handlers for hash commands
    • set.go: handlers for set commands
    • sortedset.go: handlers for sorted set commands
    • pubsub.go: implements of publish / subscribe
    • aof.go: implements of AOF persistence and rewrite
    • geo.go: implements of geography features

License

This project is licensed under the GPL license.

Documentation

Overview

Package godis is a memory database with redis compatible interface

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Auth 

func Auth(c redis.Connection, args [][]byte) redis.Reply

Auth validate client's password

func BGRewriteAOF 

func BGRewriteAOF(db *MultiDB, args [][]byte) redis.Reply

BGRewriteAOF asynchronously rewrites Append-Only-File

func DiscardMulti 

func DiscardMulti(conn redis.Connection) redis.Reply

DiscardMulti drops MULTI pending commands

func EnqueueCmd 

func EnqueueCmd(conn redis.Connection, cmdLine [][]byte) redis.Reply

EnqueueCmd puts command line into `multi` pending queue

func GetRelatedKeys 

func GetRelatedKeys(cmdLine [][]byte) ([]string, []string)

GetRelatedKeys analysis related keys

func Ping 

func Ping(db *DB, args [][]byte) redis.Reply

Ping the server

func RegisterCommand 

func RegisterCommand(name string, executor ExecFunc, prepare PreFunc, rollback UndoFunc, arity int)

RegisterCommand registers a new command arity means allowed number of cmdArgs, arity < 0 means len(args) >= -arity. for example: the arity of `get` is 2, `mget` is -2

func RewriteAOF 

func RewriteAOF(db *MultiDB, args [][]byte) redis.Reply

func StartMulti 

func StartMulti(conn redis.Connection) redis.Reply

StartMulti starts multi-command-transaction

func Watch 

func Watch(db *DB, conn redis.Connection, args [][]byte) redis.Reply

Watch set watching keys

Types

type CmdLine 

type CmdLine = [][]byte

CmdLine is alias for [][]byte, represents a command line

type DB 

type DB struct {
	// contains filtered or unexported fields
}

DB stores data and execute user's commands

func MakeDB 

func MakeDB() *DB

MakeDB create DB instance

func (*DB) Exec 

func (db *DB) Exec(c redis.Connection, cmdLine [][]byte) redis.Reply

Exec executes command within one database

func (*DB) ExecMulti 

func (db *DB) ExecMulti(conn redis.Connection, watching map[string]uint32, cmdLines []CmdLine) redis.Reply

ExecMulti executes multi commands transaction Atomically and Isolated

func (*DB) Expire 

func (db *DB) Expire(key string, expireTime time.Time)

Expire sets ttlCmd of key

func (*DB) Flush 

func (db *DB) Flush()

Flush clean database

func (*DB) ForEach 

func (db *DB) ForEach(cb func(key string, data *database.DataEntity, expiration *time.Time) bool)

func (*DB) GetEntity 

func (db *DB) GetEntity(key string) (*database.DataEntity, bool)

GetEntity returns DataEntity bind to given key

func (*DB) GetUndoLogs 

func (db *DB) GetUndoLogs(cmdLine [][]byte) []CmdLine

GetUndoLogs return rollback commands

func (*DB) GetVersion 

func (db *DB) GetVersion(key string) uint32

func (*DB) IsExpired 

func (db *DB) IsExpired(key string) bool

IsExpired check whether a key is expired

func (*DB) Persist 

func (db *DB) Persist(key string)

Persist cancel ttlCmd of key

func (*DB) PutEntity 

func (db *DB) PutEntity(key string, entity *database.DataEntity) int

PutEntity a DataEntity into DB

func (*DB) PutIfAbsent 

func (db *DB) PutIfAbsent(key string, entity *database.DataEntity) int

PutIfAbsent insert an DataEntity only if the key not exists

func (*DB) PutIfExists 

func (db *DB) PutIfExists(key string, entity *database.DataEntity) int

PutIfExists edit an existing DataEntity

func (*DB) RWLocks 

func (db *DB) RWLocks(writeKeys []string, readKeys []string)

RWLocks lock keys for writing and reading

func (*DB) RWUnLocks 

func (db *DB) RWUnLocks(writeKeys []string, readKeys []string)

RWUnLocks unlock keys for writing and reading

func (*DB) Remove 

func (db *DB) Remove(key string)

Remove the given key from db

func (*DB) Removes 

func (db *DB) Removes(keys ...string) (deleted int)

Removes the given keys from db

type ExecFunc 

type ExecFunc func(db *DB, args [][]byte) redis.Reply

ExecFunc is interface for command executor args don't include cmd line

type MultiDB 

type MultiDB struct {
	// contains filtered or unexported fields
}

func MakeBasicMultiDB 

func MakeBasicMultiDB() *MultiDB

MakeBasicMultiDB create a MultiDB only with basic abilities for aof rewrite and other usages

func NewStandaloneServer 

func NewStandaloneServer() *MultiDB

NewStandaloneServer creates a standalone redis server, with multi database and all other funtions

func (*MultiDB) AfterClientClose 

func (mdb *MultiDB) AfterClientClose(c redis.Connection)

AfterClientClose does some clean after client close connection

func (*MultiDB) Close 

func (mdb *MultiDB) Close()

Close graceful shutdown database

func (*MultiDB) Exec 

func (mdb *MultiDB) Exec(c redis.Connection, cmdLine [][]byte) (result redis.Reply)

Exec executes command parameter `cmdLine` contains command and its arguments, for example: "set key value"

func (*MultiDB) ExecMulti 

func (mdb *MultiDB) ExecMulti(conn redis.Connection, watching map[string]uint32, cmdLines []CmdLine) redis.Reply

func (*MultiDB) ExecWithLock 

func (mdb *MultiDB) ExecWithLock(conn redis.Connection, cmdLine [][]byte) redis.Reply

func (*MultiDB) ForEach 

func (mdb *MultiDB) ForEach(dbIndex int, cb func(key string, data *database.DataEntity, expiration *time.Time) bool)

func (*MultiDB) GetUndoLogs 

func (mdb *MultiDB) GetUndoLogs(dbIndex int, cmdLine [][]byte) []CmdLine

func (*MultiDB) RWLocks 

func (mdb *MultiDB) RWLocks(dbIndex int, writeKeys []string, readKeys []string)

func (*MultiDB) RWUnLocks 

func (mdb *MultiDB) RWUnLocks(dbIndex int, writeKeys []string, readKeys []string)

type PreFunc 

type PreFunc func(args [][]byte) ([]string, []string)

PreFunc analyses command line when queued command to `multi` returns related write keys and read keys

type UndoFunc 

type UndoFunc func(db *DB, args [][]byte) []CmdLine

UndoFunc returns undo logs for the given command line execute from head to tail when undo

Source Files

View all Source files

Directories

Path Synopsis
Package cluster provides a server side cluster which is transparent to client.
 Package cluster provides a server side cluster which is transparent to client.
datastruct
dict
list
lock
set
sortedset
interface
database
redis
tcp
lib
consistenthash
geohash
idgenerator
logger
sync/atomic
sync/wait
timewheel
utils
wildcard
redis
client
connection
parser
reply
reply/asserts
server

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.