pinion

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 2, 2022 License: MIT Imports: 8 Imported by: 0

README

pinion

MIT licensed GoDoc Build Status Language

Package pinion provides a fast and simple set of routines to manage the storage and retrieval of structured records.

Overview

Pinion automates the task of managing record storage and multiple retrieval indexes. Its simple programming interface, comprising methods like Put() and Get(), operate on types that implement the pinion.Record interface. This interface isolates key-building and value-encoding to one place in your application. When used with the piniondb/store package, a fast implementation of the interface can be made simply that does not require reflection. Any practical number of record types that satisfy the pinion.Record interface can be managed by a pinion database.

Currently, pinion does not support joined records. This is obviated to some degree with its support for structures that may include maps and slices.

The pinion package depends on boltdb. All tests pass on Linux, Mac and Windows platforms.

Example

This example manages records of type personType. This type implements the pinion.Record interface; see the heavily commented file person_test.go for more details. Note that no type assertions or explicit encoding or decoding needs to be done to populate and retrieve records.


var db *pinion.DB
var err error
var person personType
db, err = pinion.Create("example/standalobe.db", 0600, pinion.Options{})
if err == nil {
    wdb := db.Wrap()
    list := []nameType{
        {last: "Smith", middle: "J", first: "Carol"},
        {last: "Jones", middle: "W", first: "Robert"},
    }
    wdb.Add(&person, func() bool {
        if len(list) > 0 {
            person = personType{id: 0, name: list[0]}
            list = list[1:]
            return true
        }
        return false
    })
    for idx := uint8(0); idx < idxPersonCount; idx++ {
        fmt.Printf("%-12s", personIndexNames[idx])
        person = personType{} // Start search at beginning with zeroed record
        wdb.Get(&person, idx, func() bool {
            fmt.Printf(" [%s]", person)
            return true
        })
        fmt.Println("")
    }
}
if err != nil {
    fmt.Println(err)
}

Running this example produces the following output:

Output:
ID           [Carol J Smith / 1] [Robert W Jones / 2]
Last name    [Robert W Jones / 2] [Carol J Smith / 1]
First name   [Carol J Smith / 1] [Robert W Jones / 2]

Installation

To install the package on your system, run

go get github.com/piniondb/pinion

Errors

The methods of a *pinion.DB instance return an error if the operation fails. Since database activity often involves a lot of steps, you may find it useful to locally wrap the database instance with Wrap() in order to defer error handling to a single place.

Keys

In addition to the required primary index, up to 255 secondary indexes can be defined for the record type you want to manage. Only keys in the primary index (index 0) need to be unique. Keys must be sortable when inserted into the underlying database as byte slices. The piniondb/store package provides support for fixed-length key segments. Alternatively, you can use fmt.Sprintf() to format fixed-length fields.

Best practices

  • Implement the pinion.Record interface in the same location at which the structure itself is defined.
  • When working with multiple records, single calls to Add(), Put() and Get() will be faster than individual calls to AddRec(), PutRec() and GetRec().

Contributing Changes

pinion is a global community effort and you are invited to make it even better. If you have implemented a new feature or corrected a problem, please consider contributing your change to the project. Your pull request should

  • be compatible with the MIT License
  • be properly documented
  • include an example in one of the test files (for example, pinion_test.go) if appropriate

Use the Go Report Card to assure that no compliance issues have been introduced.

License

pinion is released under the MIT License.

Documentation

Overview

Package pinion provides a fast and simple set of routines to manage the storage and retrieval of structured records.

Overview

Pinion automates the task of managing record storage and multiple retrieval indexes. Its simple programming interface, comprising methods like Put() and Get(), operate on types that implement the pinion.Record interface. This interface isolates key-building and value-encoding to one place in your application. When used with the piniondb/store package, a fast implementation of the interface can be made simply that does not require reflection. Any practical number of record types that satisfy the pinion.Record interface can be managed by a pinion database.

Currently, pinion does not support joined records. This is obviated to some degree with its support for structures that may include maps and slices.

The pinion package depends on boltdb. All tests pass on Linux, Mac and Windows platforms.

Example

This example manages records of type personType. This type implements the pinion.Record interface; see the heavily commented file person_test.go for more details. Note that no type assertions or explicit encoding or decoding needs to be done to populate and retrieve records.

var db *pinion.DB
var err error
var person personType
db, err = pinion.Create("example/standalobe.db", 0600, pinion.Options{})
if err == nil {
	wdb := db.Wrap()
	list := []nameType{
		{last: "Smith", middle: "J", first: "Carol"},
		{last: "Jones", middle: "W", first: "Robert"},
	}
	wdb.Add(&person, func() bool {
		if len(list) > 0 {
			person = personType{id: 0, name: list[0]}
			list = list[1:]
			return true
		}
		return false
	})
	for idx := uint8(0); idx < idxPersonCount; idx++ {
		fmt.Printf("%-12s", personIndexNames[idx])
		person = personType{} // Start search at beginning with zeroed record
		wdb.Get(&person, idx, func() bool {
			fmt.Printf(" [%s]", person)
			return true
		})
		fmt.Println("")
	}
}
if err != nil {
	fmt.Println(err)
}

Running this example produces the following output:

Output:
ID           [Carol J Smith / 1] [Robert W Jones / 2]
Last name    [Robert W Jones / 2] [Carol J Smith / 1]
First name   [Carol J Smith / 1] [Robert W Jones / 2]

Installation

To install the package on your system, run

go get github.com/piniondb/pinion

Errors

The methods of a *pinion.DB instance return an error if the operation fails. Since database activity often involves a lot of steps, you may find it useful to locally wrap the database instance with Wrap() in order to defer error handling to a single place.

Keys

In addition to the required primary index, up to 255 secondary indexes can be defined for the record type you want to manage. Only keys in the primary index (index 0) need to be unique. Keys must be sortable when inserted into the underlying database as byte slices. The piniondb/store package provides support for fixed-length key segments. Alternatively, you can use fmt.Sprintf() to format fixed-length fields.

Best practices

• Implement the pinion.Record interface in the same location at which the structure itself is defined

• When working with multiple records, single calls to Add(), Put() and Get() will be faster than individual calls to AddRec(), PutRec() and GetRec().

Contributing Changes

pinion is a global community effort and you are invited to make it even better. If you have implemented a new feature or corrected a problem, please consider contributing your change to the project. Your pull request should

• be compatible with the MIT License

• be properly documented

• include an example in one of the test files (for example, pinion_test.go) if appropriate

Use https://goreportcard.com/report/github.com/piniondb/pinion to assure that no compliance issues have been introduced.

License

pinion is released under the MIT License.

Example

Example demonstrates record management with pinion.

var db *pinion.DB
var err error
db, err = pinion.Create("example/person.db", 0600, pinion.Options{})
if err == nil {
	wdb := db.Wrap()
	populate(wdb)
	change(wdb)
	db.Close()
	err = wdb.Error()
}
if err != nil {
	fmt.Println(err)
}
Output:

--- Initial ---
ID           [Carol J Smith / 1] [Robert W Jones / 2]
Last name    [Robert W Jones / 2] [Carol J Smith / 1]
First name   [Carol J Smith / 1] [Robert W Jones / 2]
--- Robert changed to Bob ---
ID           [Carol J Smith / 1] [Bob W Jones / 2]
Last name    [Bob W Jones / 2] [Carol J Smith / 1]
First name   [Bob W Jones / 2] [Carol J Smith / 1]
Example (Standalone)
var db *pinion.DB
var err error
var person personType
db, err = pinion.Create("example/standalobe.db", 0600, pinion.Options{})
if err == nil {
	wdb := db.Wrap()
	list := []nameType{
		{last: "Smith", middle: "J", first: "Carol"},
		{last: "Jones", middle: "W", first: "Robert"},
	}
	wdb.Add(&person, func() bool {
		if len(list) > 0 {
			person = personType{id: 0, name: list[0]}
			list = list[1:]
			return true
		}
		return false
	})
	for idx := uint8(0); idx < idxPersonCount; idx++ {
		fmt.Printf("%-12s", personIndexNames[idx])
		person = personType{} // Start search at beginning with zeroed record
		wdb.Get(&person, idx, func() bool {
			fmt.Printf(" [%s]", person)
			return true
		})
		fmt.Println("")
	}
}
if err != nil {
	fmt.Println(err)
}
Output:

ID           [Carol J Smith / 1] [Robert W Jones / 2]
Last name    [Robert W Jones / 2] [Carol J Smith / 1]
First name   [Carol J Smith / 1] [Robert W Jones / 2]

Index

Examples

Constants

View Source
const (
	// Version identifies the database compatibility level
	Version = 1
)

Variables

View Source
var (
	// ErrMissingIndex is reported when data is to be accessed and the application
	// record indicates that no indexes are present
	ErrMissingIndex = errors.New("at least one index must be defined")
	// ErrMissingRecord indicates a corrupt database in which data is not
	// associated with a valid primary key
	ErrMissingRecord = errors.New("missing record for valid index")
	// ErrNotOpen is reported when a database operation is attempted on a closed
	// database instance
	ErrNotOpen = errors.New("database is not open")
	// ErrRecNotFound is reported when no match is found for the requested record
	ErrRecNotFound = errors.New("record not found")
)

Functions

This section is empty.

Types

type DB

type DB struct {
	// contains filtered or unexported fields
}

The DB type manages data access with an underlying bbolt database. It is safe for concurrent goroutine use. Only one instance of this type should be active at a time.

Example (Quantity)

ExampleDB_quantity demonstrates various operations with multiple indexes.

var db *pinion.DB
var wdb *pinion.WrapDB
var err error
db, err = quantityDB("example/quantity.db", 0, 256)
if err == nil {
	wdb = db.Wrap()
	var q quantityType
	q.id = 99
	fmt.Println("--- ID sequence ---")
	wdb.Get(&q, idxQuantityID, func() bool {
		if q.id < 104 {
			fmt.Println(q)
			return true
		}
		return false
	})
	q.val, _ = str.QuantityEncode(72)
	count := 5
	fmt.Println("--- Word sequence ---")
	wdb.Get(&q, idxQuantityVal, func() bool {
		if count > 0 {
			fmt.Println(q)
			count--
			return true
		}
		return false
	})
	db.Close()
	err = wdb.Error()
}
if err != nil {
	fmt.Println(err)
}
Output:

--- ID sequence ---
[         99 : ninety nine]
[        100 : one hundred]
[        101 : one hundred one]
[        102 : one hundred two]
[        103 : one hundred three]
--- Word sequence ---
[         72 : seventy two]
[          6 : six]
[         16 : sixteen]
[         60 : sixty]
[         68 : sixty eight]

func Create

func Create(path string, mode os.FileMode, options Options) (db *DB, err error)

Create creates a Pinion database. The file is replaced if it already exists.

func Open

func Open(path string, mode os.FileMode, options Options) (db *DB, err error)

Open opens an existing Pinion database.

func (*DB) Add

func (db *DB) Add(recPtr Record, f func() bool) (putErr error)

Add inserts one or more records in the database. It functions like Put() except that Add() will pass an autoincremented ID by means of the NextID interface method. If the application manages its own unique primary keys, it is more efficient to call Put() instead of Add(). It is crucial that all keys of each record processed by this method be properly assigned. This assures that modified keys are properly replaced.

func (*DB) AddRec

func (db *DB) AddRec(recPtr Record) (err error)

AddRec inserts one record in the database. recPtr is a pointer to a variable that fully assigned. The requirements documented for Add() apply.

func (*DB) Close

func (db *DB) Close() (err error)

Close shuts down the database and releases all associated resources. Any subsequent calls to methods of DB will result in an error.

func (*DB) Delete

func (db *DB) Delete(recPtr Record, f func() bool) (delErr error)

Delete removes records and their associated keys from the database. recPtr is a pointer to a variable that will, each time f() returns true, be populated with a successive value to be delete. The iteration is stopped when f() returns false. Only the field or fields needed to generate the primary key (index 0) need be assigned.

func (*DB) DeleteRec

func (db *DB) DeleteRec(recPtr Record) (err error)

DeleteRec deletes one record from the database. recPtr is a pointer to a variable that has at least the field or fields that make up the primary key (index 0) assigned.

func (*DB) Get

func (db *DB) Get(recPtr Record, idx uint8, f func() bool) (getErr error)

Get returns zero or more records. It calls f iteratively until f() returns false or no more records are found. For each call of f, the record variable pointed to be recPtr will be populated with a successive value from the database. The record order is determined by the index specified by idx. The first record returned is the first one that matches the initial value of the record pointed to by recPtr. Only the field or fields that make up the key associated with index idx need to be assigned initially.

func (*DB) GetRec

func (db *DB) GetRec(recPtr Record, idx uint8) (err error)

GetRec returns zero or one record from the database. The first record that matches the key field or fields associated with index idx will be put in the variable pointed to be recPtr. In this case, an error value of nil is returned. If no match is found, ErrRecNotFound is returned.

func (*DB) HexDump

func (db *DB) HexDump(wr io.Writer)

HexDump is a diagnostic routine to help with viewing the records and keys in a database.

func (*DB) Put

func (db *DB) Put(recPtr Record, f func() bool) (putErr error)

Put inserts or replaces zero or more records in the database. recPtr is a pointer to a variable that will, each time f() returns true, be populated with a successive value to be stored. The iteration is stopped when f() returns false. If the primary key (that is, Key(0)) of a record already exists, the record will overwrite its previous value. If the primary key is unique, the record will be inserted; however, unlike Add(), the inserted record's NextID() method will not be called. It is crucial that all keys of each record processed by this method be properly assigned. This assures that modified keys are properly replaced.

func (*DB) PutRec

func (db *DB) PutRec(recPtr Record) (err error)

PutRec inserts or replaces one record in the database. recPtr is a pointer to a variable that fully assigned. The requirements documented for Put() apply.

func (*DB) Wrap

func (db *DB) Wrap() (wdb *WrapDB)

Wrap returns a wrapped database instance that simplifies error handling.

type Options

type Options struct {
	BoltOpt bbolt.Options
}

The Options type is used to configure the database when it is opened.

type Record

type Record interface {
	// Convert the record identified by the method receiver to a byte sequence.
	encoding.BinaryMarshaler
	// Convert the specified byte sequence to the record identified by the method
	// receiver.
	encoding.BinaryUnmarshaler
	// Return the number of indexes associated with the record receiver. This
	// value must remain constant.
	IndexCount() uint8
	// Name of record for database table (invariant).
	Name() string
	// Construct a key for the index specified by idx.
	Key(idx uint8) (key []byte, err error)
	// Provide a temporary buffer for internal use. The returned record should be
	// of the same type as the method receiver.
	New() Record
	// Receive an autoincremented ID prior to inserting a record. This is called
	// only when the application calls Add(), not Put(). It is called before calls
	// are made to Key().
	NextID(uint64)
}

The Record interface specifies methods that allow pinion to manage multiply indexed records. The Name() and IndexCount() methods return constants; they return information about the type of the method receiver rather than the receiver instance itself. They are included here to allow all the information needed for pinion to manage database operations to be specified in one location.

It is very important that the implementation of this interface not change after data has been stored. The implementation is in essence a contract between the application and the stored data.

type WrapDB

type WrapDB struct {
	// contains filtered or unexported fields
}

WrapDB is a wrapper around DB that maintains error state internally. Its methods are the same as those of DB except that they do not return an error value. Instead, if an error is detected, it is stored for later examination. In this case, subsequent calls will be bypassed. The use of this wrapper may simplify code paths by deferring error handling until a series of database operations have completed.

Unlike a DB instance, a WrapDB instance is not safe for concurrent use. It is intended to be used locally for a relatively small sequence of method calls and then, after examining the error value returned by Error(), allowed to fall out of scope. Multiple goroutines may wrap a single *pinion.DB instance concurrently.

Example

This code exemplifies the use of various WrapDB methods. These are like corresponding DB methods except that error values are not returned. Instead, they retain the error value internally. In this example, the DB instance is local. In a typical application, the database instance is global so that it can be shared among goroutines. Unlike the DB instance, a WrapDB instance should always be local so that errors can be managed locally and do not spill over to other goroutines.

var (
	q    quantityType
	db   *pinion.DB
	wdb  *pinion.WrapDB
	fl   *os.File
	err  error
	list []uint32
)
show := func(str string) {
	fmt.Printf("--- %s ---\n", str)
	q = quantityType{} // Start at beginning with zeroed record
	wdb.Get(&q, idxQuantityID, func() bool {
		fmt.Println(q)
		return true
	})
}
db, err = quantityDB("example/dump.db", 1234, 1236)
if err == nil {
	wdb = db.Wrap()
	list = []uint32{42, 0}
	wdb.Put(&q, func() bool {
		if len(list) > 0 {
			q = quantityRec(list[0])
			list = list[1:]
			return true
		}
		return false
	})
	fl, err = os.Create("example/dump.txt")
	if err == nil {
		wdb.HexDump(fl)
		show("Full")
		list = []uint32{1235, 0}
		wdb.Delete(&q, func() bool {
			if len(list) > 0 {
				q.id = list[0]
				list = list[1:]
				return true
			}
			return false
		})
		show("Deleted ID 0 and 1235")
		q = quantityRec(1232)
		wdb.AddRec(&q)
		show("Added 1232")
		q.id = 42
		wdb.DeleteRec(&q)
		show("Deleted ID 42")
		fl.Close()
		if wdb.Error() == nil {
			wdb.ErrorSet(pinion.ErrRecNotFound)
			if wdb.Error() != nil {
				wdb.ErrorClear()
			}
		}
		err = wdb.Error()
	}
	db.Close()
}
if err != nil {
	fmt.Println(err)
}
Output:

--- Full ---
[          0 : zero]
[         42 : forty two]
[      1,234 : one thousand two hundred thirty four]
[      1,235 : one thousand two hundred thirty five]
[      1,236 : one thousand two hundred thirty six]
--- Deleted ID 0 and 1235 ---
[         42 : forty two]
[      1,234 : one thousand two hundred thirty four]
[      1,236 : one thousand two hundred thirty six]
--- Added 1232 ---
[         42 : forty two]
[      1,232 : one thousand two hundred thirty two]
[      1,234 : one thousand two hundred thirty four]
[      1,236 : one thousand two hundred thirty six]
--- Deleted ID 42 ---
[      1,232 : one thousand two hundred thirty two]
[      1,234 : one thousand two hundred thirty four]
[      1,236 : one thousand two hundred thirty six]

func (*WrapDB) Add

func (wdb *WrapDB) Add(recPtr Record, f func() bool)

Add is the locally-wrapped version of *DB.Add().

func (*WrapDB) AddRec

func (wdb *WrapDB) AddRec(recPtr Record)

AddRec is the locally-wrapped version of *DB.AddRec().

func (*WrapDB) Delete

func (wdb *WrapDB) Delete(recPtr Record, f func() bool)

Delete is the locally-wrapped version of *DB.Delete().

func (*WrapDB) DeleteRec

func (wdb *WrapDB) DeleteRec(recPtr Record)

DeleteRec is the locally-wrapped version of *DB.DeleteRec().

func (*WrapDB) Error

func (wdb *WrapDB) Error() error

Error returns the internal error value. It does not change the internal value.

func (*WrapDB) ErrorClear

func (wdb *WrapDB) ErrorClear() (err error)

ErrorClear clears the internal error value. The current value before being cleared is returned.

func (*WrapDB) ErrorSet

func (wdb *WrapDB) ErrorSet(err error)

ErrorSet allows the application to transfer its own error to the wrapped database instance. This may simplify code paths in the application because it allows the response to an error to be handled in one place. WrapDB cannot already be in an error state. If err is nil, it is ignored and will not overwrite the internal error value.

func (*WrapDB) Get

func (wdb *WrapDB) Get(recPtr Record, idx uint8, f func() bool)

Get is the locally-wrapped version of *DB.Get().

func (*WrapDB) GetRec

func (wdb *WrapDB) GetRec(recPtr Record, idx uint8)

GetRec is the locally-wrapped version of *DB.GetRec().

func (*WrapDB) HexDump

func (wdb *WrapDB) HexDump(wr io.Writer)

HexDump is the locally-wrapped version of *DB.HexDump().

func (*WrapDB) Put

func (wdb *WrapDB) Put(recPtr Record, f func() bool)

Put is the locally-wrapped version of *DB.Put().

func (*WrapDB) PutRec

func (wdb *WrapDB) PutRec(recPtr Record)

PutRec is the locally-wrapped version of *DB.PutRec().

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL