acid

acid

An embedded in-memory key-value store with unlimited indexes. Don't use this thing in production unless you have a really good use case for ephemeral data. Totally use it in your tests, though.

Installation

Go 1.18+ required, as acid uses generics.

go get -u codeberg.org/ess/acid

Core Concepts

Buckets

A bucket is a key-value store. Think of it as a thread-safe array with secondary indexes.

The Bucket type must be instantiated with a comparable type. That covers most types that one can imagine, so you should be good.

In acid, a bucket can have any number of indexes, and they can be either plain ol' indexes, or they can be unique indexes. Either way, the index is created alongside a derivator that is used to derive the index value for a given object that you store in the bucket.

By default, a bucket has no indexes, which means that you can't really query it, so you should add some indexes.

Indexes

As is the case in any system that allows indexing, an index is effectively a map that connects a specific data type to the exact location of an item in the system.

In acid, a bucket can have any number of indexes, and they can be either plain ol' indexes, or they can be unique indexes.

There are two ways to add indexes to a bucket. You can do it during instantiation by chaining:

deriveID := func(item any) (any, error) {
  return item.(*Person).ID, nil
}

deriveName := func(item any) (any, error) {
  return item.(*Person).Name, nil
}

deriveAge := func(item any) (any, error) {
  return item.(*Person).Age, nil
}

data := acid.Bucket[Person]().
  WithIndex("id", acid.Unique[int](), deriveID).
  WithIndex("name", acid.Index[string](), deriveName).
  WithIndex("age", acid.Index[int], deriveAge).
  Finalize()

You can also add indexes procedurally:

deriveID := func(item any) (any, error) {
  return item.(*Person).ID, nil
}

deriveName := func(item any) (any, error) {
  return item.(*Person).Name, nil
}

deriveAge := func(item any) (any, error) {
  return item.(*Person).Age, nil
}

data := acid.Bucket[Person]()
data.WithIndex("id", acid.Unique[int](), deriveID)
data.WithIndex("name", acid.Index[string](), deriveName)
data.WithIndex("age", acid.Index[int](), deriveAge)
data.Finalize()

As you'll note, the last step on each of these is to Finalize() the bucket. This lets the bucket know that it's ready and we're not planning on adding any further configuration to it.

You can still attempt to make WithIndex calls after finalization, but they will not actually do anything.

Querying

Most data-related actions on a bucket require a Query. This is just a fancy way to let the action know what stored items you're interested in. For example, let's add a hypothetical Person:

data.Add(somePerson)

Note: The where clauses of a query are always an AND relationship. The And query method is just an alias for the Where query method.

Now that you have an item added (and you indexed it), you can query for it. There are two ways to do this: Some and One.

With One, the idea is that you want to retrive exactly one item from the bucket, and your query should narrow that down as far as you see fit. This is much less error-prone if you specify a unique index in your bucket and query, but it's not required. The bucket will complain if your query resolves to zero or more than one item.

person, err := data.One(acid.Where("id", 1))

On the other hand, Some will get you all items in the bucket that match your query:

// To get some items, give it a query, but think about omiting any
// unique index references
some := data.Some(acid.Where("age", 20))

Finally, while not technically a query, you can get all of the items in the bucket like this:

all := data.All()

Full Usage Example

package main

import (
	"errors"
	"fmt"

	"codeberg.org/ess/acid/v2"
	"codeberg.org/ess/acid/v2/storage"
	"codeberg.org/ess/mcnulty"
)

func main() {
	cohort := newPeople()

	p1 := &person{ID: 1, Name: "Jack Randomhacker", Age: 33}
	p2 := &person{ID: 2, Name: "Jill Randomhacker", Age: 23}
	p3 := &person{ID: 3, Name: "Rando Bystandarius", Age: 23}

	for _, p := range []*person{p1, p2, p3} {
		err := cohort.Add(p)
		if err != nil {
			panic(err)
		}
	}

	jills := cohort.Named("Jill Randomhacker")
	if len(jills) == 0 {
		panic("there are no jills!")
	}

	fmt.Println("jills:")
	if len(jills) == 0 {
		fmt.Println("\t:sadface: no jills")
	} else {
		for _, p := range jills {
			fmt.Println("\t* ", p)
		}
	}

	fmt.Println("\nerrybody:")
	for _, p := range cohort.All() {
		fmt.Println("\t* ", p)
	}

	fmt.Println()

	err := cohort.Add(&person{ID: 1, Name: "Badius Actorius", Age: 12})
	if err == nil {
		panic("should have failed to add a duplicate ID, but instead we got a bad actor")
	}

	fmt.Println("bad actor avoided")
}

type person struct {
	ID   int
	Name string
	Age  int
}

func (p *person) String() string {
	return fmt.Sprintf("[%d] %s who is %d years old", p.ID, p.Name, p.Age)
}

type people struct {
	bucket *storage.Bucket[*person]
}

func derivePersonID(item any) (any, error) {
	p, ok := item.(*person)
	if !ok {
		return mcnulty.Nil[any](), errors.New("not a person")
	}

	return p.ID, nil
}

func derivePersonName(item any) (any, error) {
	p, ok := item.(*person)
	if !ok {
		return mcnulty.Nil[any](), errors.New("not a person")
	}

	return p.Name, nil
}

func derivePersonAge(item any) (any, error) {
	p, ok := item.(*person)
	if !ok {
		return mcnulty.Nil[any](), errors.New("not a person")
	}

	return p.Age, nil
}

func newPeople() *people {
	bucket := acid.New[*person]().
		WithIndex("id", acid.Unique[int](), derivePersonID).
		WithIndex("name", acid.Index[string](), derivePersonName).
		WithIndex("age", acid.Index[int](), derivePersonAge).
		Finalize()

	return &people{bucket: bucket}
}

func (repo *people) Add(candidate *person) error {
	return repo.bucket.Add(candidate)
}

func (repo *people) Get(id int) (*person, error) {
	return repo.bucket.One(acid.Where("id", id))
}

func (repo *people) Named(name string) []*person {
	return repo.bucket.Some(acid.Where("name", name))
}

func (repo *people) All() []*person {
	return repo.bucket.All()
}

History

• v2.0.0 - Much improved API
• v1.0.3 - Improved Index.DeleteID
• v1.0.2 - Bucket.Remove now works
• v1.0.1 - Corrected some bucket errors
• v1.0.0 - Initial Release