sod

package module
v1.10.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 27, 2022 License: GPL-3.0 Imports: 19 Imported by: 0

README

Version GoDoc Build Coverage

Go Simple Object Database

A simple database model to store Go structure (on disk) and search across them. It has features close to what an ORM framework can provide but has the advantage of being:

  • in pure Go (de facto portable)
  • does not depend on any DB engine (SQL, SQLite, Mongo ...) to do its job
  • everything is kept simple (one file per structure and eventually an index) It supports structure fields indexing to speed up searches on important fields.

What should you use this project for:

  • you want to implement Go struct persistency in a simple way
  • you want to do DB like operations on those structures (Update, Delete, Search ...)
  • you don't want to deploy an ORM framework

What you should not use this project for:

  • even though performances are not so bad, I don't think you can rely on it for high troughput DB operations

Examples

See examples directory for all examples.

Documentation

Index

Constants

View Source 
const (
	SchemaFilename = "schema.json"
)

Variables

View Source 
var (
	ErrUnkownField          = errors.New("unknown object field")
	ErrFieldNotIndexed      = errors.New("field not indexed")
	ErrUnkownSearchOperator = errors.New("unknown search operator")
	ErrCasting              = errors.New("casting error")

	ErrConstraintUnique = errors.New("uniqueness constraint")
)
View Source 
var (
	ErrIndexCorrupted    = errors.New("index is corrupted")
	ErrBadSchema         = errors.New("schema must be a file")
	ErrMissingObjIndex   = errors.New("schema is missing object index")
	ErrStructureChanged  = errors.New("object structure changed")
	ErrExtensionMismatch = errors.New("extension mismatch")
	ErrUnindexedField    = errors.New("field is not indexed")

	DefaultExtension   = ".json"
	DefaultCompression = false
	DefaultSchema      = Schema{
		Extension: DefaultExtension,
		Compress:  DefaultCompression}
	DefaultSchemaCompress = Schema{
		Extension: DefaultExtension,
		Compress:  true}
)
View Source 
var (
	ErrUnknownOperator           = errors.New("unknown logical operator")
	ErrNoObjectFound             = errors.New("no object found")
	ErrUnexpectedNumberOfResults = errors.New("unexpected number of results")
)
View Source 
var (
	DefaultPermissions = fs.FileMode(0700)
	LowercaseNames     = false
	ErrWrongObjectType = errors.New("wrong objet type")
)
View Source 
var (
	ErrEOI = errors.New("end of iterator")
)
View Source 
var (
	ErrFieldDescModif = errors.New("field descriptor changed")
)
View Source 
var (
	ErrInvalidObject = errors.New("object is not valid")
)
View Source 
var (
	ErrUnknownKeyType = errors.New("unknown key type")
)

Functions

func Assign added in v1.9.3 

func Assign(objs []Object, target interface{}) (err error)

func AssignOne added in v1.9.3 

func AssignOne(o Object, target interface{})

func IsIndexCorrupted added in v1.9.9 

func IsIndexCorrupted(err error) bool

func IsNoObjectFound added in v1.3.0 

func IsNoObjectFound(err error) bool

func IsUnique added in v1.3.0 

func IsUnique(err error) bool

func ToObjectChan added in v1.6.1 

func ToObjectChan(slice interface{}) (objs chan Object)

ToObjectChan is a convenient function to pre-process arguments passed to InsertOrUpdateMany function.

Types

type Async added in v1.5.0 

type Async struct {
	Enable    bool
	Threshold int
	Timeout   time.Duration
	// contains filtered or unexported fields
}

func (*Async) MarshalJSON added in v1.5.0 

func (a *Async) MarshalJSON() ([]byte, error)

func (*Async) UnmarshalJSON added in v1.5.0 

func (a *Async) UnmarshalJSON(b []byte) (err error)

type Constraints added in v1.4.0 

type Constraints struct {
	Index  bool `json:"index,omitempty"`
	Unique bool `json:"unique,omitempty"`
	Upper  bool `json:"upper,omitempty"`
	Lower  bool `json:"lower,omitempty"`
}

func (Constraints) String added in v1.8.0 

func (c Constraints) String() string

func (*Constraints) Transform added in v1.8.0 

func (c *Constraints) Transform(i interface{})

func (*Constraints) TransformField added in v1.8.0 

func (c *Constraints) TransformField(fieldPath string, o Object)

func (*Constraints) Transformer added in v1.8.0 

func (c *Constraints) Transformer() bool

type DB 

type DB struct {
	// contains filtered or unexported fields
}

func Open 

func Open(root string) *DB

Open opens a Simple Object Database

func (*DB) All 

func (db *DB) All(of Object) (out []Object, err error)

All returns all Objects in the DB

func (*DB) AssignAll added in v1.9.3 

func (db *DB) AssignAll(of Object, target interface{}) (err error)

AssignAll assigns all Objects in the DB to target

func (*DB) AssignIndex added in v1.9.6 

func (db *DB) AssignIndex(of Object, field string, target interface{}) (err error)

AssignIndex assign indexed fields to target. It prevents from fetching objects from disk if the only thing we actually want to query is some indexed fields. As indexes are all in memory this call is fast. The function panics if target is not a slice pointer or if indexed values cannot be assigned to target elements.

func (*DB) Close 

func (db *DB) Close() (last error)

Close closes gently the DB by flushing any pending async writes and by committing all the schemas to disk

func (*DB) Commit 

func (db *DB) Commit(o Object) (err error)

Commit object schema on the disk. This method must be called after Insert/Delete operations.

func (*DB) Control added in v1.4.0 

func (db *DB) Control() (err error)

Control controls checks for inconsistencies in DB

func (*DB) Count 

func (db *DB) Count(of Object) (n int, err error)

Count the number of Object in the database

func (*DB) Create 

func (db *DB) Create(o Object, s Schema) (err error)

Create a schema for an Object

func (*DB) Delete 

func (db *DB) Delete(o Object) (lastErr error)

Delete deletes a single Object from the database and commit changes

func (*DB) DeleteAll 

func (db *DB) DeleteAll(of Object) (err error)

DeleteAll deletes all Objects of the same type and commit changes

func (*DB) DeleteObjects added in v1.4.0 

func (db *DB) DeleteObjects(from *iterator) (err error)

DeleteObjects deletes Objects from an Iterator and commit changes. This primitive can be used for bulk deletions.

func (*DB) Drop 

func (db *DB) Drop() (err error)

Drop drops all the database

func (*DB) Exist 

func (db *DB) Exist(o Object) (ok bool, err error)

Exist returns true if the object exist.

func (*DB) Flush added in v1.5.0 

func (db *DB) Flush(o Object) (err error)

Flush a single object to disk. Flush does not commit schema

func (*DB) FlushAll added in v1.5.0 

func (db *DB) FlushAll(of Object) (err error)

FlushAll objects of type to disk. As Flush this function does not commit schema to disk

func (*DB) FlushAllAndCommit added in v1.6.6 

func (db *DB) FlushAllAndCommit(of Object) (last error)

FlushAllAndCommit flushes objects of type to disk and commit schema

func (*DB) FlushAndCommit added in v1.6.6 

func (db *DB) FlushAndCommit(o Object) (last error)

Flush a single object to disk and commit changes

func (*DB) Get 

func (db *DB) Get(in Object) (out Object, err error)

Get gets a single Object from the DB

func (*DB) GetByUUID added in v1.6.5 

func (db *DB) GetByUUID(in Object, uuid string) (out Object, err error)

GetByUUID gets a single Object from the DB its UUID

func (*DB) InsertOrUpdate 

func (db *DB) InsertOrUpdate(o Object) (err error)

InsertOrUpdate inserts or updates a single Object and commits changes. This method is not suited for bulk insertions as each insert will trigger a write overhead. For bulk insertion use InsertOrUpdateBulk function

func (*DB) InsertOrUpdateBulk 

func (db *DB) InsertOrUpdateBulk(in chan Object, csize int) (n int, err error)

InsertOrUpdateBulk inserts objects in bulk in the DB. A chunk size needs to be provided to commit the DB at every chunk. The DB is locked at every chunk processed, so changing the chunk size impact other concurrent DB operations. n returns the number of Objects successfully inserted.

func (*DB) InsertOrUpdateMany added in v1.5.0 

func (db *DB) InsertOrUpdateMany(objects ...Object) (n int, err error)

InsertOrUpdateMany inserts several objects into the DB and commit schema after all insertions. It is faster than calling InsertOrUpdate for every objects separately. All objects must be of the same type. This method is atomic, so all objects must satisfy constraints and be valid according to their Validate method. If this method fails no object is inserted.

func (*DB) Iterator 

func (db *DB) Iterator(of Object) (it *iterator, err error)

Iterator returns an Object Iterator

func (*DB) Lock added in v1.5.0 

func (db *DB) Lock()

func (*DB) RLock added in v1.5.0 

func (db *DB) RLock()

func (*DB) RUnlock added in v1.5.0 

func (db *DB) RUnlock()

func (*DB) Repair added in v1.8.0 

func (db *DB) Repair(of Object) (err error)

Repair repairs database schema

func (*DB) Schema 

func (db *DB) Schema(of Object) (s *Schema, err error)

Schema retrieves the schema of an Object

func (*DB) Search 

func (db *DB) Search(o Object, field, operator string, value interface{}) *Search

Search Object where field matches value according to an operator

func (*DB) Unlock added in v1.5.0 

func (db *DB) Unlock()

type FieldDescMap added in v1.8.0 

type FieldDescMap map[string]FieldDescriptor

func FieldDescriptors added in v1.7.1 

func FieldDescriptors(from Object) (desc FieldDescMap)

func (FieldDescMap) CompatibleWith added in v1.9.0 

func (m FieldDescMap) CompatibleWith(target FieldDescMap) (err error)

func (FieldDescMap) Constraint added in v1.8.0 

func (m FieldDescMap) Constraint(fpath string, c Constraints) (err error)

func (FieldDescMap) FieldsCompatibleWith added in v1.9.0 

func (m FieldDescMap) FieldsCompatibleWith(target FieldDescMap) (err error)

func (FieldDescMap) GetDescriptor added in v1.8.0 

func (m FieldDescMap) GetDescriptor(fpath string) (d FieldDescriptor, ok bool)

func (FieldDescMap) Transformers added in v1.8.0 

func (m FieldDescMap) Transformers() (t []FieldDescriptor)

type FieldDescriptor added in v1.3.0 

type FieldDescriptor struct {
	Path        string      `json:"path"`
	Type        string      `json:"type"`
	Constraints Constraints `json:"constraints"`
}

func (*FieldDescriptor) DeepEqual added in v1.9.0 

func (d *FieldDescriptor) DeepEqual(other *FieldDescriptor) bool

func (*FieldDescriptor) FieldEqual added in v1.9.0 

func (d *FieldDescriptor) FieldEqual(other *FieldDescriptor) bool

func (FieldDescriptor) String added in v1.8.0 

func (d FieldDescriptor) String() string

func (*FieldDescriptor) Transform added in v1.8.0 

func (d *FieldDescriptor) Transform(o interface{})

type Item 

type Item struct {
	// contains filtered or unexported fields
}

Item is a base structure implementing Object interface

func (*Item) Initialize 

func (o *Item) Initialize(uuid string)

func (*Item) Transform added in v1.6.2 

func (o *Item) Transform()

func (*Item) UUID 

func (o *Item) UUID() string

func (*Item) Validate added in v1.6.1 

func (o *Item) Validate() error

type Object 

type Object interface {
	// UUID returns a unique identifier used to store the
	// Object in the database
	UUID() string

	// Initialize is called to initialize the UUID associated
	// to an Object
	Initialize(string)

	// Transform is called prior to Object insertion and
	// can be used to apply some transformation on the data
	// to insert.
	Transform()

	// Validate is called every time an Object is inserted
	// if an error is returned by this function the Object
	// will not be inserted.
	Validate() error
}

func CloneObject added in v1.8.0 

func CloneObject(o Object) (out Object)

func ToObjectSlice added in v1.5.1 

func ToObjectSlice(slice interface{}) (objs []Object)

ToObjectSlice is a convenient function to pre-process arguments passed to InsertOrUpdateMany function.

type Schema 

type Schema struct {
	Fields      FieldDescMap `json:"fields"`
	Extension   string       `json:"extension"`
	Compress    bool         `json:"compress"`
	Cache       bool         `json:"cache"`
	AsyncWrites *Async       `json:"async-writes,omitempty"`
	ObjectIndex *objIndex    `json:"index"`
	// contains filtered or unexported fields
}

func NewCustomSchema added in v1.8.0 

func NewCustomSchema(fields FieldDescMap, ext string) (s Schema)

func (*Schema) Asynchrone added in v1.6.3 

func (s *Schema) Asynchrone(threshold int, timeout time.Duration)

Asynchrone makes the data described by this schema managed asynchronously Objects will be written either if more than threshold events are modified or at every timeout

func (*Schema) Indexed added in v1.8.0 

func (s *Schema) Indexed() (desc []FieldDescriptor)

Indexed returns the FieldDescriptors of indexed fields 

type Search struct {
	// contains filtered or unexported fields
}

Search helper structure to easily build search queries on objects and retrieve the results

func (*Search) And 

func (s *Search) And(field, operator string, value interface{}) *Search

And performs a new Search while "ANDing" search results

func (*Search) Assign added in v1.7.0 

func (s *Search) Assign(target interface{}) (err error)

Assign returns results found calling Collect function and assign them to target. Target must be a *[]sod.Object otherwise the function panics. If no Object is found, ErrNoObjectFound is returned

func (*Search) AssignOne added in v1.6.8 

func (s *Search) AssignOne(target interface{}) (err error)

AssignOne returns the first result found calling Collect function and assign the Object found to target. Target must be a *sod.Object otherwise the function panics. If no Object is found, ErrNoObjectFound is returned

func (*Search) AssignUnique added in v1.9.4 

func (s *Search) AssignUnique(target interface{}) (err error)

AssignUnique checks there is only one result in search and assign it to target. If search retuns more than one result, ErrUnexpectedNumberOfResults is returned

func (*Search) Collect 

func (s *Search) Collect() (out []Object, err error)

Collect all the objects resulting from the search. If a search has been made on an indexed field, results will be in descending order by default. If you want to change result order, call Reverse before. NB: only search on indexed field(s) will be garanteed to be ordered according to the last field searched.

func (*Search) Delete added in v1.4.0 

func (s *Search) Delete() (err error)

Delete deletes the objects found by the search

func (*Search) Err 

func (s *Search) Err() error

Err return any error encountered while searching

func (*Search) Expects added in v1.9.4 

func (s *Search) Expects(n int) *Search

Expects checks that the number of results is the one expected if not, next call to s.Err must return an error and any subsbequent attempt to collect results must fail

func (*Search) ExpectsZeroOrN added in v1.9.5 

func (s *Search) ExpectsZeroOrN(n int) *Search

ExpectsZeroOrN checks that the number of results is the one expected or zero. If not, next call to s.Err must return an error and any subsbequent attempt to collect results must fail

func (*Search) Iterator 

func (s *Search) Iterator() (it *iterator, err error)

Iterator returns an Iterator convenient to iterate over the objects resulting from the search

func (*Search) Len 

func (s *Search) Len() int

Len returns the number of data returned by the search

func (*Search) Limit added in v1.2.0 

func (s *Search) Limit(limit uint64) *Search

Limit the number of results collected by Collect function

func (*Search) One added in v1.3.0 

func (s *Search) One() (o Object, err error)

One returns the first result found calling Collect function. If no Object is found, ErrNoObjectFound is returned

func (*Search) Operation added in v1.6.3 

func (s *Search) Operation(operator, field, comparator string, value interface{}) *Search

Operation performs a new Search while ANDing or ORing the results operator must be in ["and", "&&", "or", "||"]

func (*Search) Or 

func (s *Search) Or(field, operator string, value interface{}) *Search

Or performs a new Search while "ORing" search results

func (*Search) Reverse added in v1.1.0 

func (s *Search) Reverse() *Search

Reverse the order the results are collected by Collect function

Source Files

View all Source files

Directories

Path Synopsis
examples
foreign_structs
regex
short
simple
unique

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.