README
¶
Purple
An all-in-one data service with support for:
- Key/value operations
- Counters and sets
- Flags (basically key/value pairs where the value is a Boolean with a default value of
false) - Caching with TTL
Purple is meant to abstract away complex database interfaces (Redis, DynamoDB, Mongo, memory, disk, etc.) in favor of a unified set of dead-simple operations (see the full list of operations below).
You can run Purple as a gRPC server or an HTTP server (both expose the same interfaces). There's currently a gRPC client for Go only but in principle gRPC clients could be added for other languages. There are also three backends available: memory, disk, and Redis.
Since any server type can work with any backend, the following server/backend combinations are currently supported:
| Server | Backend |
|---|---|
| gRPC | Memory |
| gRPC | Disk |
| gRPC | Redis |
| HTTP | Memory |
| HTTP | Disk |
| HTTP | Redis |
The project
Goals
Microservices or FaaS functions that rely on stateful data operations can use Purple instead of needing to interact with multiple databases. This greatly simplifies the service/function development process by sharply reducing the hassle of dealing with databases (i.e. no need to install/learn/use 5 different database clients).
Does your service need something that isn't provided by Purple? File an issue or submit a PR and I'll add it!
Current status
Purple is in its very early stages. The data interfaces it provides are almost comically simple. Please do not use Purple as a production data service just yet (though I'd like to get there). Instead, use it for prototyping and experimenting.
Also be aware that Purple runs as a single instance and has no clustering built in (and thus isn't highly available). If you use the Redis backend, however, you can run multiple instances of Purple that connect to a single Redis cluster.
Future directions
In the future, I imagine Purple acting as an abstraction layer over lots of different data systems, exposing a powerful interface that covers the overwhelming majority of data use cases without exposing the system internals of any of those systems. This would entail:
- Making the current data interfaces more sophisticated and capable of covering a wider range of use cases
- Adding new interfaces, such as a timeseries interface, a simple graph interface, etc.
- Providing a relational interface that supports a subset of SQL (SQLite would likely suffice for this)
- Providing optional pluggable backends behind Purple (e.g. using Redis for caching, Elasticsearch for search, etc.)
- Providing a message queue/pub-sub interface, eliminating the need for a Kafka/Pulsar/RabbitMQ/etc. client
Want to contribute?
See the contributors guide for details.
Operations
The table below lists the available client operations for the Go client:
| Operation | Service | Semantics |
|---|---|---|
CacheGet(key string) |
Cache | Fetches the value of a key from the cache or returns a not found error if the key doesn't exist or has expired. |
CacheSet(key, value string, ttl int32) |
Cache | Sets the value associated with a key and assigns a TTL (the default is 5 seconds). Overwrites the value and TTL if the key already exists. |
CounterIncrement(key string, amount int64) |
Counter | Increments a counter by the designated amount. Returns the new value of the counter or an error. |
CounterGet(key string) |
Counter | Fetches the current value of a counter. Returns zero if the counter isn't found. |
FlagGet(key string) |
Flag | Fetches the current Boolean value of a flag. If the flag hasn't yet been set, the default value is false. |
FlagSet(key string, value bool) |
Flag | Sets the Boolean value of a flag. |
SetGet(set string) |
Set | Fetch the items currently in the specified set. Returns an empty string set ([]string) if the set isn't found. |
SetAdd(set, item string) |
Set | Adds an item to the specified set and returns the resulting set. |
SetRemove(set, item string) |
Set | Removes an item from the specified set and returns the resulting set. Returns an empty set isn't found or is already empty. |
KVGet(key string) |
KV | Gets the value associated with a key or returns a not found error. The value is currently just a byte array payload but could be made more complex later (e.g. a payload plus a content type, metadata, etc.). |
KVPut(key string, value *Value) |
KV | Sets the value associated with a key, overwriting any existing value. |
KVDelete(key string) |
KV | Deletes the value associated with a key or returns a not found error. |
Backends
There are currently three backends available for Purple:
| Backend | Explanation |
|---|---|
| Disk | Data is stored persistently on disk using the Badger library. Each service (cache, KV, etc.) is stored in its own separate on-disk DB, which guarantees key isolation. |
| Memory | Data is stored in native Go data structures (maps, slices, etc.). This backend is blazing fast but all data is lost when the service restarts. |
| Redis | The Purple server stores all data in a persistent Redis installation. Each service uses a different Redis database, which provides key isolation. |
Try it out
To try out Purple locally, you can run the Purple gRPC server in one shell session and some example client operations in another session:
git clone https://github.com/purpledb/purple && cd purple
# Start the gRPC server...
go run examples/grpc-server/main.go
# And then in a different session...
go run examples/grpc-client/main.go
Installation
gRPC server
To install the Purple gRPC server:
# Executable
go install github.com/purpledb/purple/cmd/purple-grpc
# Docker image
docker pull purpledb/purple-grpc:latest
Then you can run it:
# Executable
purple-grpc
# Docker image
docker run --rm -it -p 8081:8081 purpledb/purple-grpc:latest
You should see log output like this:
2019/07/27 14:37:09 Starting up the server on port 8081
HTTP server
To install the purple HTTP server:
# Executable
go install github.com/purpledb/purple/cmd/purple-http
# Docker image
docker pull purpledb/purple-http:latest
Then you can run it:
# Executable
purple-http
# Docker image
docker run --rm -it -p 8080:8080 purpledb/purple-http:latest
gRPC Go client
To use the Go client in your service or FaaS function:
go get github.com/purpledb/purple
To instantiate a client:
import "github.com/purpledb/purple"
// Supply the address of the purple gRPC server
client, err := purple.NewGrpcClient("localhost:8081")
if err != nil {
// Handle error
}
// Now you can run the various data operations, for example:
if err := client.CacheSet("player1-session", "a1b2c3d4e5f6", 120); err != nil {
// Handle error
}
HTTP Go client
import "github.com/purpledb/purple"
client, err := purple.NewHttpClient("http://localhost:8080")
if err != nil {
// Handle error
}
count, err := client.CounterIncrement("player1-points", 100)
Deployment
Kubernetes
There are two configuration files in the deploy directory that enable you to run the purple gRPC and HTTP servers, respectively, on Kubernetes. Both use the default namespace.
gRPC
kubectl apply -f deploy/purple-grpc-k8s.yaml
HTTP
kubectl apply -f deploy/purple-http-k8s.yaml
Accessing the service
Once you've deployed purple on Kubernetes, you can access it in your local environment using port forwarding:
# gRPC
kubectl port-forward svc/purple 8081:8081
# HTTP
kubectl port-forward svc/purple 8080:8080
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrNoKey = errors.New("no resource key provided") ErrNoValue = errors.New("no value provided") ErrNoAddress = errors.New("no server address provided") ErrNoPort = errors.New("no server port provided") ErrPortOutOfRange = errors.New("port must be between 1024 and 49151") ErrBackendNotRecognized = errors.New("backend key not recognized") ErrNoBackend = errors.New("no backend specified") )
Functions ¶
func IsNotFound ¶
Types ¶
type NotFoundError ¶
type NotFoundError struct {
// contains filtered or unexported fields
}
func NotFound ¶
func NotFound(key string) NotFoundError
func (NotFoundError) AsProtoStatus ¶
func (e NotFoundError) AsProtoStatus() error
func (NotFoundError) Error ¶
func (e NotFoundError) Error() string
type ServerConfig ¶
func (*ServerConfig) Validate ¶
func (c *ServerConfig) Validate() error
Source Files
Directories