Documentation
¶
Overview ¶
Package docstore provides a portable implementation of a document store. TODO(jba): link to an explanation of document stores (https://en.wikipedia.org/wiki/Document-oriented_database?) TODO(jba): expand package doc to batch other Go CDK APIs.
Representing Times ¶
Docstore can store and retrieve values of type time.Time, with two caveats. First, the timezone may not be preserved. Second, Docstore guarantees only that time.Time values are represented to millisecond precision. Many providers will do better, but if you need to be sure that times are stored with nanosecond precision, convert the time.Time to another type before storing, and re-create when you retrieve it. For instance, you could call the time's UnixNano method to get an int64, and get the original time back (in the local timezone) with the time.Unix function.
Revisions ¶
Docstore gives every document a revision when it is created. Docstore uses the field name "DocstoreRevision" (stored in the constant docstore.RevisionField) to hold the revision. Whenever a document is modified, its revision changes. Revisions can be used for optimistic locking: whenever a Put, Replace, Update or Delete action is given a document with a revision, then an error for which gcerrors.Code returns FailedPrecondition or NotFound is returned if the stored document's revision does not match the given document's. Thus a Get followed by one of those write actions will fail if the document was changed between the Get and the write.
Since different providers use different types for revisions, the type of the revision field is unspecified. When defining a struct for storing docstore data, define the field to be of type interface{}. For example,
type User { Name string; DocstoreRevision interface{} }
If a struct doesn't have a DocstoreRevision field, then the logic described above won't apply to documents read and written with that struct. All writes with the struct will succeed even if the document was changed since the last Get.
Queries ¶
Docstore supports querying within a collection. Call the Query method on Collection to obtain a Query value, then build your query by calling Query methods like Where, Limit and so on. Finally, call the Get method on the query to execute it. The result is an iterator, whose use is described below.
iter := coll.Query().Where("size", ">", 10).Limit(5).Get(ctx)
The Where methods defines a filter condition, much like a WHERE clause in SQL. Conditions are of the form "field op value", where field is any document field path (including dot-separated paths), op is one of "=", ">", "<", ">=" or "<=", and value can be a string or number.
You can make multiple Where calls. In some cases, parts of a Where clause may be processed on the client rather than natively by the provider, which may have performance implications for large result sets. See the provider-specific package doc for details.
The Limit method specifies the maximum number of results to return.
Docstore provides no guarantees about the ordering of results. There is no way to ask for a particular ordering.
Use the DocumentIterator returned from Query.Get by repeatedly calling its Next method until it returns io.EOF. Always call Stop when you are finished with an iterator.
iter := coll.Query().Where("size", ">", 10).Limit(5).Get(ctx)
defer iter.Stop()
for {
m := map[string]interface{}{}
err := iter.Next(ctx, m)
if err == io.EOF {
break
}
if err != nil {
return err
}
fmt.Println(m)
}
Example (Actions) ¶
package main
import (
"context"
"gocloud.dev/internal/docstore"
)
func main() {
ctx := context.Background()
type Player struct {
Name string // Name is the key
Score int
}
fred := &Player{Name: "Fred", Score: 18}
got := &Player{Name: "Fred"}
coll := docstore.NewCollection(nil)
errs := coll.Actions().
Put(fred).
Get(got).
Do(ctx)
_ = errs
}
Output:
Index ¶
- Constants
- Variables
- func Increment(amount interface{}) interface{}
- type Action
- type ActionList
- func (l *ActionList) BeforeDo(f func(asFunc func(interface{}) bool) error) *ActionList
- func (l *ActionList) Create(doc Document) *ActionList
- func (l *ActionList) Delete(doc Document) *ActionList
- func (l *ActionList) Do(ctx context.Context) error
- func (l *ActionList) Get(doc Document, fps ...FieldPath) *ActionList
- func (l *ActionList) Put(doc Document) *ActionList
- func (l *ActionList) Replace(doc Document) *ActionList
- func (l *ActionList) String() string
- func (l *ActionList) Unordered() *ActionList
- func (l *ActionList) Update(doc Document, mods Mods) *ActionList
- type ActionListError
- type Collection
- func (c *Collection) Actions() *ActionList
- func (c *Collection) As(i interface{}) bool
- func (c *Collection) Create(ctx context.Context, doc Document) error
- func (c *Collection) Delete(ctx context.Context, doc Document) error
- func (c *Collection) Get(ctx context.Context, doc Document, fps ...FieldPath) error
- func (c *Collection) Put(ctx context.Context, doc Document) error
- func (c *Collection) Query() *Query
- func (c *Collection) Replace(ctx context.Context, doc Document) error
- func (c *Collection) Update(ctx context.Context, doc Document, mods Mods) error
- type CollectionURLOpener
- type Document
- type DocumentIterator
- type FieldPath
- type Mods
- type Query
- func (q *Query) BeforeQuery(f func(asFunc func(interface{}) bool) error) *Query
- func (q *Query) Delete(ctx context.Context) error
- func (q *Query) Get(ctx context.Context, fps ...FieldPath) *DocumentIterator
- func (q *Query) Limit(n int) *Query
- func (q *Query) OrderBy(field, direction string) *Query
- func (q *Query) Plan(fps ...FieldPath) (string, error)
- func (q *Query) Update(ctx context.Context, mods Mods) error
- func (q *Query) Where(fp FieldPath, op string, value interface{}) *Query
- type URLMux
- func (mux *URLMux) CollectionSchemes() []string
- func (mux *URLMux) OpenCollection(ctx context.Context, urlstr string) (*Collection, error)
- func (mux *URLMux) OpenCollectionURL(ctx context.Context, u *url.URL) (*Collection, error)
- func (mux *URLMux) RegisterCollection(scheme string, opener CollectionURLOpener)
- func (mux *URLMux) ValidCollectionScheme(scheme string) bool
Examples ¶
Constants ¶
const ( Ascending = "asc" Descending = "desc" )
Ascending and Descending are constants for use in the OrderBy method.
const RevisionField = "DocstoreRevision"
RevisionField is the name of the document field used for document revision information, to implement optimistic locking. See the Revisions section of the package documentation.
Variables ¶
var NewCollection = newCollection
NewCollection is intended for use by provider implementations.
Functions ¶
func Increment ¶ added in v0.14.0
func Increment(amount interface{}) interface{}
Increment returns a modification that results in a field being incremented. It should only be used as a value in a Mods map, like so:
docstore.Mods{"count", docstore.Increment(1)}
The amount must be an integer or floating-point value.
Types ¶
type Action ¶
type Action struct {
// contains filtered or unexported fields
}
An Action is a read or write on a single document. Use the methods of ActionList to create and execute Actions.
type ActionList ¶
type ActionList struct {
// contains filtered or unexported fields
}
An ActionList is a group of actions that affect a single collection.
By default, the actions in the list are performed in order from the point of view of the client. However, the actions may not be performed atomically, and there is no guarantee that a Get following a write will see the value just written (for example, if the provider is eventually consistent). Execution stops with the first failed action.
If the Unordered method is called on an ActionList, then the actions may be executed in any order, perhaps concurrently. All actions will be executed, even if some fail.
func (*ActionList) BeforeDo ¶ added in v0.14.0
func (l *ActionList) BeforeDo(f func(asFunc func(interface{}) bool) error) *ActionList
BeforeDo takes a callback function that will be called before the ActionList is executed by the underlying provider's action functionality. The callback takes a parameter, asFunc, that converts its argument to provider-specific types. See https://gocloud.dev/concepts/as/ for background information.
func (*ActionList) Create ¶
func (l *ActionList) Create(doc Document) *ActionList
Create adds an action that creates a new document to the given ActionList, and returns the ActionList. The document must not already exist; an error for which gcerrors.Code returns AlreadyExists is returned if it does. If the document doesn't have key fields, it will be given key fields with unique values. TODO(jba): treat zero values for struct fields as not present?
func (*ActionList) Delete ¶
func (l *ActionList) Delete(doc Document) *ActionList
Delete adds an action that deletes a document to the given ActionList, and returns the ActionList. Only the key fields and RevisionField of doc are used. See the Revisions section of the package documentation for how revisions are handled. If doc has no revision and the document doesn't exist, nothing happens and no error is returned.
func (*ActionList) Do ¶
func (l *ActionList) Do(ctx context.Context) error
Do executes the action list.
If Do returns a non-nil error, it will be of type ActionListError. If any action fails, the returned error will contain the position in the ActionList of each failed action (but see the discussion of unordered mode, below). As a special case, none of the actions will be executed if any is invalid (for example, a Put whose document is missing its key field).
In ordered mode (when the Unordered method was not called on the list), execution will stop after the first action that fails.
In unordered mode, all the actions will be executed. Docstore tries to execute the actions as efficiently as possible. Sometimes this makes it impossible to attribute failures to specific actions; in such cases, the returned ActionListError will have entries whose Index field is negative.
func (*ActionList) Get ¶
func (l *ActionList) Get(doc Document, fps ...FieldPath) *ActionList
Get adds an action that retrieves a document to the given ActionList, and returns the ActionList. Only the key fields of doc are used. If fps is omitted, doc will contain all the fields of the retrieved document. If fps is present, only the given field paths are retrieved, in addition to the revision field. It is undefined whether other fields of doc at the time of the call are removed, unchanged, or zeroed, so for portable behavior doc should contain only the key fields.
func (*ActionList) Put ¶
func (l *ActionList) Put(doc Document) *ActionList
Put adds an action that adds or replaces a document to the given ActionList, and returns the ActionList. The key fields must be set. The document may or may not already exist. See the Revisions section of the package documentation for how revisions are handled.
func (*ActionList) Replace ¶
func (l *ActionList) Replace(doc Document) *ActionList
Replace adds an action that replaces a document to the given ActionList, and returns the ActionList. The key fields must be set. The document must already exist; an error for which gcerrors.Code returns NotFound is returned if it does not. See the Revisions section of the package documentation for how revisions are handled.
func (*ActionList) String ¶ added in v0.14.0
func (l *ActionList) String() string
func (*ActionList) Unordered ¶ added in v0.13.0
func (l *ActionList) Unordered() *ActionList
After Unordered is called, Do may execute the actions in any order. All actions will be executed, even if some fail.
func (*ActionList) Update ¶
func (l *ActionList) Update(doc Document, mods Mods) *ActionList
Update atomically applies Mods to doc, which must exist. Only the key and revision fields of doc are used. It is an error to pass an empty Mods to Update.
A modification will create a field if it doesn't exist.
No field path in mods can be a prefix of another. (It makes no sense to, say, set foo but increment foo.bar.)
See the Revisions section of the package documentation for how revisions are handled.
It is undefined whether updating a sub-field of a non-map field will succeed. For instance, if the current document is {a: 1} and Update is called with the mod "a.b": 2, then either Update will fail, or it will succeed with the result {a: {b: 2}}.
Update does not modify its doc argument. To obtain the new value of the document, call Get after calling Update.
type ActionListError ¶ added in v0.13.0
An ActionListError is returned by ActionList.Do. It contains all the errors encountered while executing the ActionList, and the positions of the corresponding actions.
func (ActionListError) Error ¶ added in v0.13.0
func (e ActionListError) Error() string
func (ActionListError) Unwrap ¶ added in v0.13.0
func (e ActionListError) Unwrap() error
Unwrap returns the error in e, if there is exactly one. If there is more than one error, Unwrap returns nil, since there is no way to determine which should be returned.
type Collection ¶
type Collection struct {
// contains filtered or unexported fields
}
A Collection is a set of documents. TODO(jba): make the docstring look more like blob.Bucket.
func OpenCollection ¶ added in v0.13.0
func OpenCollection(ctx context.Context, urlstr string) (*Collection, error)
OpenCollection opens the collection identified by the URL given. See the URLOpener documentation in provider-specific subpackages for details on supported URL formats, and https://gocloud.dev/concepts/urls/ for more information.
func (*Collection) Actions ¶
func (c *Collection) Actions() *ActionList
Actions returns an ActionList that can be used to perform actions on the collection's documents.
func (*Collection) As ¶ added in v0.14.0
func (c *Collection) As(i interface{}) bool
As converts i to provider-specific types. See https://gocloud.dev/concepts/as/ for background information, the "As" examples in this package for examples, and the provider-specific package documentation for the specific types supported for that provider.
func (*Collection) Create ¶
func (c *Collection) Create(ctx context.Context, doc Document) error
Create is a convenience for building and running a single-element action list. See ActionList.Create.
func (*Collection) Delete ¶
func (c *Collection) Delete(ctx context.Context, doc Document) error
Delete is a convenience for building and running a single-element action list. See ActionList.Delete.
func (*Collection) Get ¶
Get is a convenience for building and running a single-element action list. See ActionList.Get.
func (*Collection) Put ¶
func (c *Collection) Put(ctx context.Context, doc Document) error
Put is a convenience for building and running a single-element action list. See ActionList.Put.
func (*Collection) Query ¶ added in v0.13.0
func (c *Collection) Query() *Query
Query creates a new Query over the collection.
type CollectionURLOpener ¶ added in v0.13.0
type CollectionURLOpener interface {
OpenCollectionURL(ctx context.Context, u *url.URL) (*Collection, error)
}
CollectionURLOpener opens a collection of documents based on a URL. The opener must not modify the URL argument. It must be safe to call from multiple goroutines.
This interface is generally implemented by types in driver packages.
type Document ¶
type Document = interface{}
A Document is a set of field-value pairs. One or more fields, called the key fields, must uniquely identify the document in the collection. You specify the key fields when you open a provider collection. A field name must be a valid UTF-8 string that does not contain a '.'.
A Document can be represented as a map[string]int or a pointer to a struct. For structs, the exported fields are the document fields.
type DocumentIterator ¶ added in v0.13.0
type DocumentIterator struct {
// contains filtered or unexported fields
}
DocumentIterator iterates over documents.
Always call Stop on the iterator.
func (*DocumentIterator) As ¶ added in v0.14.0
func (it *DocumentIterator) As(i interface{}) bool
As converts i to provider-specific types. See https://gocloud.dev/concepts/as/ for background information, the "As" examples in this package for examples, and the provider-specific package documentation for the specific types supported for that provider.
func (*DocumentIterator) Next ¶ added in v0.13.0
func (it *DocumentIterator) Next(ctx context.Context, dst Document) error
Next stores the next document in dst. It returns io.EOF if there are no more documents. Once Next returns an error, it will always return the same error.
func (*DocumentIterator) Stop ¶ added in v0.13.0
func (it *DocumentIterator) Stop()
Stop stops the iterator. Calling Next on a stopped iterator will return io.EOF, or the error that Next previously returned.
type FieldPath ¶
type FieldPath string
A FieldPath is a dot-separated sequence of UTF-8 field names. Examples:
room room.size room.size.width
A FieldPath can be used select top-level fields or elements of sub-documents. There is no way to select a single list element.
type Mods ¶
type Mods map[FieldPath]interface{}
Mods is a map from field paths to modifications. At present, a modification is one of: - nil, to delete the field - an Increment value, to add a number to the field - any other value, to set the field to that value See ActionList.Update.
type Query ¶ added in v0.13.0
type Query struct {
// contains filtered or unexported fields
}
Query represents a query over a collection.
func (*Query) BeforeQuery ¶ added in v0.14.0
BeforeQuery takes a callback function that will be called before the Query is executed to the underlying provider's query functionality. The callback takes a parameter, asFunc, that converts its argument to provider-specific types. See https://gocloud.dev/concepts/as/ for background information.
func (*Query) Delete ¶ added in v0.14.0
Delete deletes all the documents specified by the query. It is an error if the query has a limit.
func (*Query) Get ¶ added in v0.13.0
func (q *Query) Get(ctx context.Context, fps ...FieldPath) *DocumentIterator
Get returns an iterator for retrieving the documents specified by the query. If field paths are provided, only those paths are set in the resulting documents.
Call Stop on the iterator when finished.
func (*Query) Limit ¶ added in v0.13.0
Limit will limit the results to at most n documents. n must be positive. It is an error to specify Limit more than once in a Get query, or at all in a Delete or Update query.
func (*Query) OrderBy ¶ added in v0.14.0
OrderBy specifies that the returned documents appear sorted by the given field in the given direction. A query can have at most one OrderBy clause. If it has none, the order of returned documents is unspecified. If a query has a Where clause and an OrderBy clause, the OrderBy clause's field must appear in a Where clause. It is an error to specify OrderBy in a Delete or Update query.
func (*Query) Plan ¶ added in v0.13.0
Plan describes how the query would be executed if its Get method were called with the given field paths. Plan uses only information available to the client, so it cannot know whether a service uses indexes or scans internally.
type URLMux ¶ added in v0.13.0
type URLMux struct {
// contains filtered or unexported fields
}
URLMux is a URL opener multiplexer. It matches the scheme of the URLs against a set of registered schemes and calls the opener that matches the URL's scheme. See https://gocloud.dev/concepts/urls/ for more information.
The zero value is a multiplexer with no registered scheme.
func DefaultURLMux ¶ added in v0.13.0
func DefaultURLMux() *URLMux
DefaultURLMux returns the URLMux used by OpenCollection.
Driver packages can use this to register their CollectionURLOpener on the mux.
func (*URLMux) CollectionSchemes ¶ added in v0.13.0
CollectionSchemes returns a sorted slice of the registered Collection schemes.
func (*URLMux) OpenCollection ¶ added in v0.13.0
OpenCollection calls OpenCollectionURL with the URL parsed from urlstr. OpenCollection is safe to call from multiple goroutines.
func (*URLMux) OpenCollectionURL ¶ added in v0.13.0
OpenCollectionURL dispatches the URL to the opener that is registered with the URL's scheme. OpenCollectionURL is safe to call from multiple goroutines.
func (*URLMux) RegisterCollection ¶ added in v0.13.0
func (mux *URLMux) RegisterCollection(scheme string, opener CollectionURLOpener)
RegisterCollection registers the opener with the given scheme. If an opener already exists for the scheme, RegisterCollection panics.
func (*URLMux) ValidCollectionScheme ¶ added in v0.13.0
ValidCollectionScheme returns true iff scheme has been registered for Collections.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package driver defines a set of interfaces that the docstore package uses to interact with the underlying services.
|
Package driver defines a set of interfaces that the docstore package uses to interact with the underlying services. |
|
Package drivertest provides a conformance test for implementations of driver.
|
Package drivertest provides a conformance test for implementations of driver. |
|
Package dynamodocstore provides a docstore implementation backed by AWS DynamoDB.
|
Package dynamodocstore provides a docstore implementation backed by AWS DynamoDB. |
|
Package firedocstore provides an implementation of the docstore API for Google Cloud Firestore.
|
Package firedocstore provides an implementation of the docstore API for Google Cloud Firestore. |
|
internal
|
|
|
fields
Package fields provides a view of the fields of a struct that follows the Go rules, amended to consider tags and case insensitivity.
|
Package fields provides a view of the fields of a struct that follows the Go rules, amended to consider tags and case insensitivity. |
|
Package memdocstore provides an in-memory implementation of the docstore API.
|
Package memdocstore provides an in-memory implementation of the docstore API. |
|
Package mongodocstore provides an implementation of the docstore API for MongoDB.
|
Package mongodocstore provides an implementation of the docstore API for MongoDB. |