ds

package module
v1.8.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 12, 2023 License: MIT Imports: 12 Imported by: 1

README

DS

Go Report Card Godoc Releases LICENSE

Package ds (short for "data store") is a key-value store with hash indexes. It allows for rudimentary but lightning fast retrieval of grouped or relevant data without having to iterate over all objects in the store.

Define the primary key, indexed keys, and unique keys as tags on struct fields, and DS takes care of the rest.

Usage & Examples

Examples can be found on the documentation for the library

Documentation

Overview

Package ds (short for "data store") is a key-value store with hash indexes. It allows for rudimentary but lightning fast retrieval of grouped or relevant data without having to iterate over all objects in the store.

Define the primary key, indexed keys, and unique keys as tags on struct fields, and DS takes care of the rest.

Index

Examples

Constants

View Source
const (
	ErrMissingRequiredValue     = "missing required value"
	ErrPointer                  = "pointer provided when none expected"
	ErrPropertyChanged          = "property changed"
	ErrDuplicatePrimaryKey      = "duplicate value for primary key"
	ErrDuplicateUnique          = "duplicate value for unique field"
	ErrFieldNotIndexed          = "field not indexed"
	ErrFieldNotUnique           = "field not unique"
	ErrMigrateTablePathNotFound = "table path"
	ErrBadTableFile             = "bad table file"
)

Error constant

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config struct {
	Fields          []Field
	TypeOf          string
	PrimaryKey      string
	Indexes         []string
	Uniques         []string
	LastInsertIndex uint64
	Version         int
}

Config describes ds table configuration

type Field added in v1.4.0

type Field struct {
	Name string
	Tag  string
	Type string
}

Field describes a field

type GetOptions

type GetOptions struct {
	// Should the results be sorted. Does nothing for unsorted tables
	Sorted bool
	// If results are to be sorted, should they be from most recent to oldest (true) or invese (false)
	Ascending bool
	// The maximum number of entries to return. 0 means unlimited
	Max int
}

GetOptions describes options for getting entries from a DS table

type IReadTransaction added in v1.8.0

type IReadTransaction interface {
	// Get will get a single entry by its primary key. Returns (nil, nil) if nothing found.
	Get(primaryKey interface{}) (interface{}, error)
	// GetIndex will get multiple entries that contain the same value for the specified indexed field.
	// Result is not ordered. Use GetIndexSorted to return a sorted slice.
	// Returns an empty array if nothing found.
	GetIndex(fieldName string, value interface{}, options *GetOptions) ([]interface{}, error)
	// GetUnique will get a single entry based on the value of the provided unique field.
	// Returns (nil, nil) if nothing found.
	GetUnique(fieldName string, value interface{}) (interface{}, error)
	// GetAll will get all of the entries in the table.
	GetAll(options *GetOptions) ([]interface{}, error)
}

IReadTransaction describes an interface for performing read-only operations on a table.

type IReadWriteTransaction added in v1.8.0

type IReadWriteTransaction interface {
	IReadTransaction
	// Add will add a new object to the table. o must the the same type that was used to register the table and cannot be a pointer.
	Add(o interface{}) error
	// Delete will delete the provided object and clean indexes
	Delete(o interface{}) error
	// DeletePrimaryKey will delete the object with the associated primary key and clean indexes. Does nothing if not object
	// matches the given primary key.
	DeletePrimaryKey(o interface{}) error
	// DeleteUnique will delete the object with the associated unique value and clean indexes. Does nothing if no object
	// matched the given unique fields value.
	DeleteUnique(field string, o interface{}) error
	// DeleteAllIndex will delete all objects matching the given indexed fields value
	DeleteAllIndex(fieldName string, value interface{}) error
	// DeleteAll delete all objects from the table
	DeleteAll() error
	// Update will update an existing object in the table. The primary key must match for this object
	// otherwise it will just be inserted as a new object. Updated objects do not change positions in a sorted
	// table.
	Update(o interface{}) error
}

IReadWriteTransaction describes an interface for performing read or write operations on a table.

type MigrateParams

type MigrateParams struct {
	// TablePath the path to the existing table file
	TablePath string
	// NewPath the path for the new table file. This can be the same as the old table.
	NewPath string
	// OldType an instance of a struct object that has the same definition as the existing table.
	OldType interface{}
	// NewType an instance of a struct object that has the definition that shall be used. This can be the same as the
	// OldType.
	NewType interface{}
	// DisableSorting (optional) if the current table is sorted, set this to true to disable sorting
	// Note: This is irreversible!
	DisableSorting bool
	// MigrateObject method called for each entry in the table in reverse order. Return a new type, error, or nil.
	// Migration is halted if an error is returned.
	// Return (nil, nil) and the entry will be skipped from migration, but migration will continue.
	MigrateObject func(o interface{}) (interface{}, error)
	// KeepBackup (optional) if false the backup copy of the table will be discarded if the migration was successful. If true
	// the copy is not deleted.
	KeepBackup bool
}

MigrateParams describes the parameters to perform a DS table migration. All fields are required unless otherwise specified.

type MigrationResults

type MigrationResults struct {
	// Success was the migration successful
	Success bool
	// Error if unsuccessful, this will be the error that caused the failure
	Error error
	// EntriesMigrated the number of entries migrated
	EntriesMigrated uint
	// EntriesSkipped the number of entries skipped
	EntriesSkipped uint
}

MigrationResults describes results from a migration

func Migrate

func Migrate(params MigrateParams) (results MigrationResults)

Migrate will migrate a DS table from one object type to another. You must migrate if the old data type is not compatible with the new type, such as if an existing field was changed. You don't need to migrate if you add or remove an existing field.

Before the existing data is touched, a copy is made with "_backup" appended to the filename, and a new table file is created with the migrated entries. Upon successful migration, the backup copy is deleted (by default). If the table being migrated is sorted, the original order is preserved.

Ensure you read the documentation of the MigrateParams struct, as it goes into greater detail on the parameters required for migration, and what they do.

Example
package main

import (
	"fmt"

	"github.com/ecnepsnai/ds"
)

func main() {
	// Define a struct that maps to the current type used in the table
	type oldType struct {
		Username  string `ds:"primary"`
		Email     string `ds:"unique"`
		FirstName string
		LastName  string
	}

	// Define your new struct
	type newType struct {
		Username string `ds:"primary"`
		Email    string `ds:"unique"`
		Name     string
	}

	// In this example, we're merging the "FirstName" and "LastName" fields of the User object to
	// just a single "Name" field
	result := ds.Migrate(ds.MigrateParams{
		TablePath: "/path/to/table.db",
		NewPath:   "/path/to/table.db", // You can specify the same path, or a new one if you want
		OldType:   oldType{},
		NewType:   newType{}, // NewType can be the same as the old type if you aren't changing the struct
		MigrateObject: func(o interface{}) (interface{}, error) {
			old := o.(oldType)
			// Within the MigrateObject function you can:
			// 1. Return a object of the NewType (specified in the MigrateParams)
			// 2. Return an error and the migration will abort
			// 3. Return nil and this entry will be skipped
			return newType{
				Username: old.Username,
				Email:    old.Email,
				Name:     old.FirstName + " " + old.LastName,
			}, nil
		},
	})
	if !result.Success {
		// Migration failed.
		panic(result.Error)
	}

	fmt.Printf("Migration successful. Entries migrated: %d, skipped: %d\n", result.EntriesMigrated, result.EntriesSkipped)
}
Output:

type Options

type Options struct {
	// DisableSorting disable all sorting features. This will make tables smaller, and inserts/removes/deletes faster.
	DisableSorting bool
	// contains filtered or unexported fields
}

Options describes options for DS tables. Once set, these cannot be changed.

type Table

type Table struct {
	Name string
	// contains filtered or unexported fields
}

Table describes a ds table. A table is mapped to a single registered object type and contains both the data and the indexes.

func Register

func Register(o interface{}, filePath string, options *Options) (*Table, error)

Register will register an instance of a struct with ds, creating a table (or opening an existing table) for this type at the specified file path.

Example
package main

import (
	"github.com/ecnepsnai/ds"
)

func main() {
	type User struct {
		// Primary fields represent the primary key of the object. Your object must have exactly one primary field
		// and its value is unique
		Username string `ds:"primary"`
		// Unique fields function just like primary fields except any field (other than the primary field) can be unique
		Email string `ds:"unique"`
		// Index fields represent fields where objects with identical values are grouped together so they can be fetched
		// quickly later
		Enabled bool `ds:"index"`
		// Fields with no ds tag are saved, but you can't fetch based on their value, and can have duplicate values
		// between entries
		Password string
	}

	tablePath := "user.db"

	table, err := ds.Register(User{}, tablePath, nil)
	if err != nil {
		panic(err)
	}

	// Don't forget to close your table when you're finished
	table.Close()
}
Output:

func (*Table) Close

func (table *Table) Close()

Close will close the table. This will not panic if the table has not been opened or already been closed.

func (*Table) IsIndexed

func (table *Table) IsIndexed(field string) bool

IsIndexed is the given field indexed

Example
package main

import (
	"github.com/ecnepsnai/ds"
)

func main() {
	type User struct {
		Username string `ds:"primary"`
		Email    string `ds:"email"`
		Enabled  bool   `ds:"index"`
	}

	tablePath := "user.db"
	table, err := ds.Register(User{}, tablePath, nil)
	if err != nil {
		panic(err)
	}

	table.IsIndexed("Username") // returns False
	table.IsIndexed("Enabled")  // returns True
}
Output:

func (*Table) IsUnique

func (table *Table) IsUnique(field string) bool

IsUnique is the given field unique

Example
package main

import (
	"github.com/ecnepsnai/ds"
)

func main() {
	type User struct {
		Username string `ds:"primary"`
		Email    string `ds:"email"`
		Enabled  bool   `ds:"index"`
	}

	tablePath := "user.db"
	table, err := ds.Register(User{}, tablePath, nil)
	if err != nil {
		panic(err)
	}

	table.IsUnique("Username") // returns False
	table.IsUnique("Email")    // returns True
}
Output:

func (*Table) StartRead added in v1.8.0

func (table *Table) StartRead(ctx func(tx IReadTransaction) error) error

StartRead will start a new read-only transaction on the table. ctx will be called when the transaction is ready. This method may block if there is an active write transaction.

Example
package main

import (
	"github.com/ecnepsnai/ds"
)

func main() {
	type User struct {
		Username string `ds:"primary"`
		Password string
		Email    string `ds:"unique"`
		Enabled  bool   `ds:"index"`
	}

	var table *ds.Table // Assumes the table is already registered, see ds.Register for an example

	// Get the user with username "example"
	var user *User
	table.StartRead(func(tx ds.IReadTransaction) error {
		object, err := tx.Get("example")
		if err != nil {
			return err // error fetching data
		}
		if object == nil {
			return nil // no object found
		}
		u, ok := object.(User)
		if !ok {
			panic("incorrect type") // data in table was not the same type as expected
		}
		user = &u
		return nil
	})

	if user == nil {
		panic("No user found!")
	}

	// Use the user object
}
Output:

func (*Table) StartWrite added in v1.8.0

func (table *Table) StartWrite(ctx func(tx IReadWriteTransaction) error) error

StartWrite will start a new read-write transaction on the table. ctx will be called when the transaction is ready. This method may block if there is an active transaction.

Note that any any errors or partial changes made during ctx are not reverted.

Example
package main

import (
	"github.com/ecnepsnai/ds"
)

func main() {
	type User struct {
		Username string `ds:"primary"`
		Password string
		Email    string `ds:"unique"`
		Enabled  bool   `ds:"index"`
	}

	var table *ds.Table // Assumes the table is already registered, see ds.Register for an example

	newUser := User{
		Username: "ian",
		Password: "hunter2",
		Email:    "email@domain",
		Enabled:  true,
	}

	err := table.StartWrite(func(tx ds.IReadWriteTransaction) error {
		return tx.Add(newUser)
	})
	if err != nil {
		panic(err)
	}
}
Output:

Directories

Path Synopsis
cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL