datastore: go.mercari.io/datastore Index | Examples | Files | Directories

package datastore

import "go.mercari.io/datastore"

Package datastore has an abstract representation of (AppEngine | Cloud) Datastore.

repository https://github.com/mercari/datastore

Let's read https://cloud.google.com/datastore/docs/ or https://cloud.google.com/appengine/docs/standard/go/datastore/ . You should also check https://godoc.org/cloud.google.com/go/datastore or https://godoc.org/google.golang.org/appengine/datastore as datastore original library.

Japanese version https://github.com/mercari/datastore/blob/master/doc_ja.go

Basic usage

Please see https://godoc.org/go.mercari.io/datastore/clouddatastore or https://godoc.org/go.mercari.io/datastore/aedatastore . Create a Client using the FromContext function of each package.

Later in this document, notes on migration from each package are summarized. Please see also there.

This package is based on the newly designed Cloud Datastore API. We are introducing flatten tags that only exist in Cloud Datastore, we need to be careful when migrating from AE Datastore. Details will be described later. If you are worried, you may have a clue to the solution at https://godoc.org/go.mercari.io/datastore/clouddatastore .

The purpose of this package

This package has three main objectives.

1. Provide a middleware layer, and reduce the code that are not directly related to application value.
2. AppEngine, Cloud, to provide the same interface for both Datastore.
3. Enable batch processing for Single Get, Signle Put, etc.

Middleware layer

We are forced to make functions that are not directly related to the value of the application for speed, stability and operation. Such functions can be abstracted and used as middleware.

Let's think about this case. Put Entity to Datastore and set it to Memcache or Redis. Next, when getting from Datastore, Get from Memcache first, Get it again from Datastore if it fails. It is very troublesome to provide these operations for all Kind and all Entity operations. However, if the middleware intervenes with all Datastore RPCs, you can transparently process without affecting the application code.

As another case, RPC sometimes fails. If it fails, the process often succeeds simply by retrying. For easy RET retry with all RPCs, it is better to implement it as middleware.

Please refer to https://godoc.org/go.mercari.io/datastore/dsmiddleware if you want to know the middleware already provided.

Provide the same interface between AppEngine and Cloud Datastore

The same interface is provided for AppEngine Datastore and Cloud Datastore. These two are compatible, you can run it with exactly the same code after creating the Client.

For example, you can use AE Datastore in a production environment and Cloud Datastore Emulator in UnitTest. If you can avoid goapp, tests may be faster and IDE may be more vulnerable to debugging. You can also read data from the local environment via Cloud Datastore for systems running on AE Datastore.

Caution. Although the storage bodies of RPCs of AE Datastore and Cloud Datastore are shared, there is a difference in expressiveness at the API level. Please carefully read the data written in AE Datastore carelessly on Cloud Datastore and do not update it. It may become impossible to read from the API of AE Datastore side. About this, we have not strictly tested.

Batch processing

The operation of Datastore has very little latency with respect to RPC's network. When acquiring 10 entities it means that GetMulti one time is better than getting 10 times using loops. However, we are not good at putting together multiple processes at once. Suppose, for example, you want to query on Post Kind, use the list of Comment IDs of the resulting Post, and get a list of Comments. For example, you can query Post Kind and get a list of Post. In addition, consider using CommentIDs of Post and getting a list of Comment. This is enough Query + 1 GetMulti is enough if you write very clever code. However, after acquiring the data, it is necessary to link the Comment list with the appropriate Post. On the other hand, you can easily write a code that throws a query once and then GetMulti the Comment as many as Post. In summary, it is convenient to have Put or Get queued, and there is a mechanism to execute it collectively later.

Batch() is it! You can find the example at https://godoc.org/go.mercari.io/datastore/#pkg-examples .

Boom replacing goon

I love goon. So I made https://godoc.org/go.mercari.io/datastore/boom which can be used in conjunction with this package.

How to migrate to this library

Here's an overview of what you need to do to migrate your existing code.

replace *datastore.Key to datastore.Key.
replace *datastore.Query to datastore.Query.
replace *datastore.Iterator to datastore.Iterator.

from AE Datastore

import go.mercari.io/datastore and go.mercari.io/datastore/aedatastore both.
rewrite those using functions of datastore package to FromContext function and Client method calls.
replace err.(appengine.MultiError) to err.(datastore.MultiError) .
Stop using appengine.BlobKey and replace with string.
replace google.golang.org/appengine/datastore.Done to google.golang.org/api/iterator.Done .
replace key.IntID() to key.ID() .
replace key.StringID() to key.Name() .
When nesting a struct, apply `datastore:", flatten "` to the corresponding field.
Delete datastore.TransactionOptions, it is not supported.
If using google.golang.org/appengine/datastore , replace to go.mercari.io/datastore .

from Cloud Datastore

import go.mercari.io/datastore and go.mercari.io/datastore/clouddatastore .
rewrite those using functions of datastore package to FromContext function and Client method calls.
replace *datastore.Commit to datastore.Commit .
If using cloud.google.com/go/datastore , replace to go.mercari.io/datastore .

from goon to boom

replace *goon.Goon to *boom.Boom .
replace goon.FromContext(ctx) to ds, _ := aedatastore.FromContext(ctx); boom.FromClient(ctx, ds) .

Code:

ctx := context.Background()
client, err := clouddatastore.FromContext(ctx)
if err != nil {
    panic(err)
}
defer client.Close()
defer testutils.CleanUpAllEntities(ctx, client)

type Comment struct {
    Message string
}
type Post struct {
    Content    string
    CommentIDs []int64    `json:"-"`
    Comments   []*Comment `datastore:"-"`
}

// preparing entities
for i := 0; i < 4; i++ {
    post := &Post{Content: fmt.Sprintf("post #%d", i+1)}
    key, err := client.Put(ctx, client.IncompleteKey("Post", nil), post)
    if err != nil {
        panic(err)
    }

    for j := 0; j < 5; j++ {
        comment := &Comment{Message: fmt.Sprintf("comment #%d", j+1)}
        cKey, err := client.Put(ctx, client.IncompleteKey("Comment", nil), comment)
        if err != nil {
            panic(err)
        }

        post.CommentIDs = append(post.CommentIDs, cKey.ID())
    }
    _, err = client.Put(ctx, key, post)
    if err != nil {
        panic(err)
    }
}

// start fetching...
posts := make([]*Post, 0)
_, err = client.GetAll(ctx, client.NewQuery("Post").Order("Content"), &posts)
if err != nil {
    panic(err)
}

// Let's batch get!
bt := client.Batch()

for _, post := range posts {
    comments := make([]*Comment, 0)
    for _, id := range post.CommentIDs {
        comment := &Comment{}
        bt.Get(client.IDKey("Comment", id, nil), comment, nil)
        comments = append(comments, comment)
    }
    post.Comments = comments
}

err = bt.Exec(ctx)
if err != nil {
    panic(err)
}

// check result
for _, post := range posts {
    fmt.Println("Post", post.Content)
    for _, comment := range post.Comments {
        fmt.Println("Comment", comment.Message)
    }
}

Output:

Post post #1
Comment comment #1
Comment comment #2
Comment comment #3
Comment comment #4
Comment comment #5
Post post #2
Comment comment #1
Comment comment #2
Comment comment #3
Comment comment #4
Comment comment #5
Post post #3
Comment comment #1
Comment comment #2
Comment comment #3
Comment comment #4
Comment comment #5
Post post #4
Comment comment #1
Comment comment #2
Comment comment #3
Comment comment #4
Comment comment #5

Code:

ctx := context.Background()
client, err := clouddatastore.FromContext(ctx)
if err != nil {
    panic(err)
}
defer client.Close()
defer testutils.CleanUpAllEntities(ctx, client)

type Comment struct {
    Message string
}

// preparing entities...
// Put ID: 2, 4 into Datastore.
var keys []datastore.Key
for i := 1; i <= 5; i++ {
    key := client.IDKey("Comment", int64(i), nil)
    keys = append(keys, key)

    comment := &Comment{Message: fmt.Sprintf("comment #%d", i)}
    if i%2 == 0 {
        _, err = client.Put(ctx, key, comment)
        if err != nil {
            panic(err)
        }
    }
}

// Let's batch get!
bt := client.Batch()

var comments []*Comment
for _, key := range keys {
    comment := &Comment{}

    bt.Get(key, comment, func(err error) error {
        if err == datastore.ErrNoSuchEntity {
            // ignore ErrNoSuchEntity
            return nil
        } else if err != nil {
            return err
        }

        comments = append(comments, comment)

        return nil
    })
}

err = bt.Exec(ctx)
if err != nil {
    panic(err)
}

// check result
for _, comment := range comments {
    fmt.Println("Comment", comment.Message)
}

Output:

Comment comment #2
Comment comment #4

Code:

ctx := context.Background()
client, err := clouddatastore.FromContext(ctx)
if err != nil {
    panic(err)
}
defer client.Close()
defer testutils.CleanUpAllEntities(ctx, client)

type Data struct {
    Name string
}

key := client.IncompleteKey("Data", nil)
entity := &Data{Name: "mercari"}
key, err = client.Put(ctx, key, entity)
if err != nil {
    panic(err)
}

entity = &Data{}
err = client.Get(ctx, key, entity)
if err != nil {
    panic(err)
}

fmt.Println(entity.Name)

Output:

mercari

Index

Examples

Package Files

batch.go caches.go doc.go doc_ja.go errors.go export.go interfaces.go load.go option.go prop.go query.go save.go time.go tx_batch.go

Variables

var (
    // ErrInvalidEntityType is returned when functions like Get or Next are
    // passed a dst or src argument of invalid type.
    ErrInvalidEntityType = errors.New("datastore: invalid entity type")
    // ErrInvalidKey is returned when an invalid key is presented.
    ErrInvalidKey = errors.New("datastore: invalid key")
    // ErrNoSuchEntity is returned when no entity was found for a given key.
    ErrNoSuchEntity = errors.New("datastore: no such entity")
)
var ErrConcurrentTransaction = errors.New("datastore: concurrent transaction")

ErrConcurrentTransaction is returned when a transaction is rolled back due to a conflict with a concurrent transaction.

var LoadEntity = loadEntity

LoadEntity to dst struct.

var SaveEntity = saveEntity

SaveEntity convert key & struct to *Entity.

var SuppressErrFieldMismatch = true

SuppressErrFieldMismatch when this flag is true. If you want to align (AE|Cloud) Datastore's default behavior, set false.

func LoadStruct Uses

func LoadStruct(ctx context.Context, dst interface{}, p []Property) error

LoadStruct loads the properties from p to dst. dst must be a struct pointer.

The values of dst's unmatched struct fields are not modified, and matching slice-typed fields are not reset before appending to them. In particular, it is recommended to pass a pointer to a zero valued struct on each LoadStruct call.

type Batch Uses

type Batch struct {
    Client Client
    // contains filtered or unexported fields
}

Batch can queue operations on Datastore and process them in batch. Batch does nothing until you call Exec(). This helps to reduce the number of RPCs.

func (*Batch) Delete Uses

func (b *Batch) Delete(key Key, h BatchErrHandler)

Delete Entity operation into the queue.

func (*Batch) Exec Uses

func (b *Batch) Exec(ctx context.Context) error

Exec will perform all the processing that was queued. This process is done recursively until the queue is empty. The return value may be MultiError, but the order of contents is not guaranteed.

func (*Batch) Get Uses

func (b *Batch) Get(key Key, dst interface{}, h BatchErrHandler)

Get Entity operation into the queue.

func (*Batch) Put Uses

func (b *Batch) Put(key Key, src interface{}, h BatchPutHandler)

Put Entity operation into the queue. This operation doesn't Put to Datastore immediately. If a h is provided, it passes the processing result to the handler, and treats the return value as the value of the result of Putting.

type BatchErrHandler Uses

type BatchErrHandler func(err error) error

BatchErrHandler represents Entity's individual callback when batching non-Put processing.

type BatchPutHandler Uses

type BatchPutHandler func(key Key, err error) error

BatchPutHandler represents Entity's individual callback when batching Put processing.

type Client Uses

type Client interface {
    // Get loads the entity stored for key into dst, which must be a struct pointer or implement PropertyLoadSaver.
    //
    // If there is no such entity for the key, Get returns ErrNoSuchEntity.
    // The values of dst's unmatched struct fields are not modified, and matching slice-typed fields are not reset before appending to them.
    // In particular, it is recommended to pass a pointer to a zero valued struct on each Get call.
    //
    // If you set false to SuppressErrFieldMismatch variable, act like the original Datastore.
    // ErrFieldMismatch is returned when a field is to be loaded into a different type than the one it was stored from, or when a field is missing or unexported in the destination struct.
    Get(ctx context.Context, key Key, dst interface{}) error

    // GetMulti is a batch version of Get.
    //
    // dst must be a []S, []*S, []I or []P, for some struct type S, some interface type I, or some non-interface non-pointer type P such that P or *P implements PropertyLoadSaver.
    // If an []I, each element must be a valid dst for Get: it must be a struct pointer or implement PropertyLoadSaver.
    //
    // As a special case, PropertyList is an invalid type for dst, even though a PropertyList is a slice of structs.
    // It is treated as invalid to avoid being mistakenly passed when []PropertyList was intended.
    GetMulti(ctx context.Context, keys []Key, dst interface{}) error

    // Put saves the entity src into the datastore with key k.
    // src must be a struct pointer or implement PropertyLoadSaver; if a struct pointer then any unexported fields of that struct will be skipped.
    // If k is an incomplete key, the returned key will be a unique key generated by the datastore.
    Put(ctx context.Context, key Key, src interface{}) (Key, error)

    // PutMulti is a batch version of Put.
    //
    // src must satisfy the same conditions as the dst argument to GetMulti.
    PutMulti(ctx context.Context, keys []Key, src interface{}) ([]Key, error)

    // Delete deletes the entity for the given key.
    Delete(ctx context.Context, key Key) error

    // DeleteMulti is a batch version of Delete.
    DeleteMulti(ctx context.Context, keys []Key) error

    // NewTransaction starts a new transaction.
    NewTransaction(ctx context.Context) (Transaction, error)

    // RunInTransaction runs f in a transaction. f is invoked with a Transaction that f should use for all the transaction's datastore operations.
    //
    // f must not call Commit or Rollback on the provided Transaction.
    //
    // If f returns nil, RunInTransaction commits the transaction, returning the Commit and a nil error if it succeeds.
    // If the commit fails due to a conflicting transaction, RunInTransaction gives up and returns ErrConcurrentTransaction immediately.
    // If you want to retry operation, You have to retry by yourself.
    //
    // If f returns non-nil, then the transaction will be rolled back and RunInTransaction will return the same error.
    //
    // Note that when f returns, the transaction is not committed. Calling code must not assume that any of f's changes have been committed until RunInTransaction returns nil.
    RunInTransaction(ctx context.Context, f func(tx Transaction) error) (Commit, error)

    // Run runs the given query in the given context.
    Run(ctx context.Context, q Query) Iterator

    // AllocateIDs accepts a slice of incomplete keys and returns a slice of complete keys that are guaranteed to be valid in the datastore.
    AllocateIDs(ctx context.Context, keys []Key) ([]Key, error)

    // Count returns the number of results for the given query.
    //
    // The running time and number of API calls made by Count scale linearly with with the sum of the query's offset and limit.
    // Unless the result count is expected to be small, it is best to specify a limit; otherwise Count will continue until it finishes counting or the provided context expires.
    Count(ctx context.Context, q Query) (int, error)

    // GetAll runs the provided query in the given context and returns all keys that match that query, as well as appending the values to dst.
    //
    // dst must have type *[]S or *[]*S or *[]P, for some struct type S or some non-interface, non-pointer type P such that P or *P implements PropertyLoadSaver.
    //
    // As a special case, *PropertyList is an invalid type for dst, even though a PropertyList is a slice of structs.
    // It is treated as invalid to avoid being mistakenly passed when *[]PropertyList was intended.
    //
    // The keys returned by GetAll will be in a 1-1 correspondence with the entities added to dst.
    //
    // If q is a “keys-only” query, GetAll ignores dst and only returns the keys.
    //
    // The running time and number of API calls made by GetAll scale linearly with with the sum of the query's offset and limit.
    // Unless the result count is expected to be small, it is best to specify a limit; otherwise GetAll will continue until it finishes collecting results or the provided context expires.
    GetAll(ctx context.Context, q Query, dst interface{}) ([]Key, error)

    // IncompleteKey creates a new incomplete key.
    // The supplied kind cannot be empty.
    // The namespace of the new key is empty.
    IncompleteKey(kind string, parent Key) Key

    // NameKey creates a new key with a name.
    // The supplied kind cannot be empty.
    // The supplied parent must either be a complete key or nil.
    // The namespace of the new key is empty.
    NameKey(kind, name string, parent Key) Key

    // IDKey creates a new key with an ID.
    // The supplied kind cannot be empty.
    // The supplied parent must either be a complete key or nil.
    // The namespace of the new key is empty.
    IDKey(kind string, id int64, parent Key) Key

    // NewQuery creates a new Query for a specific entity kind.
    //
    // An empty kind means to return all entities, including entities created and managed by other App Engine features, and is called a kindless query.
    // Kindless queries cannot include filters or sort orders on property values.
    NewQuery(kind string) Query

    // Close closes the Client.
    Close() error

    // DecodeKey decodes a key from the opaque representation returned by Encode.
    DecodeKey(encoded string) (Key, error)

    // DecodeCursor from its base-64 string representation.
    DecodeCursor(s string) (Cursor, error)

    // Batch creates batch mode objects.
    Batch() *Batch
    // AppendMiddleware to client.
    // Middleware will apply First-In First-Apply
    AppendMiddleware(middleware Middleware)
    // RemoveMiddleware from client.
    RemoveMiddleware(middleware Middleware) bool
    // Context returns this client's context.
    Context() context.Context
    // SetContext to this client.
    SetContext(ctx context.Context)
}

Client is a client for reading and writing data in a datastore dataset.

type ClientGenerator Uses

type ClientGenerator func(ctx context.Context, opts ...ClientOption) (Client, error)

ClientGenerator represents the type of function for generating Client.

Deprecated:

var FromContext ClientGenerator

FromContext provides default ClientGenerator. this variable will be injected by go.mercari.io/datastore/aedatastore or go.mercari.io/datastore/clouddatastore package's init function.

Deprecated: use aedatastore.FromContext or clouddatastore.FromContext instead.

type ClientOption Uses

type ClientOption interface {
    Apply(*internal.ClientSettings)
}

A ClientOption is an option for a Datastore client.

func WithCredentialsFile Uses

func WithCredentialsFile(filename string) ClientOption

WithCredentialsFile returns a ClientOption that authenticates API calls with the given service account or refresh token JSON credentials file.

func WithGRPCDialOption Uses

func WithGRPCDialOption(opt grpc.DialOption) ClientOption

WithGRPCDialOption returns a ClientOption that appends a new grpc.DialOption to an underlying gRPC dial. It does not work with WithGRPCConn.

func WithHTTPClient Uses

func WithHTTPClient(client *http.Client) ClientOption

WithHTTPClient returns a ClientOption that specifies the HTTP client to use as the basis of communications. This option may only be used with services that support HTTP as their communication transport. When used, the WithHTTPClient option takes precedent over all other supplied options.

func WithProjectID Uses

func WithProjectID(projectID string) ClientOption

WithProjectID returns a ClientOption that specifies ProjectID to be used in client.

func WithScopes Uses

func WithScopes(scope ...string) ClientOption

WithScopes returns a ClientOption that overrides the default OAuth2 scopes to be used for a service.

func WithTokenSource Uses

func WithTokenSource(s oauth2.TokenSource) ClientOption

WithTokenSource returns a ClientOption that specifies an OAuth2 token source to be used as the basis for authentication.

type Commit Uses

type Commit interface {
    Key(p PendingKey) Key
}

Commit represents the result of a committed transaction.

type Cursor Uses

type Cursor interface {
    String() string
}

Cursor is an iterator's position. It can be converted to and from an opaque string. A cursor can be used from different HTTP requests, but only with a query with the same kind, ancestor, filter and order constraints.

The zero Cursor can be used to indicate that there is no start and/or end constraint for a query.

type Entity Uses

type Entity struct {
    Key        Key
    Properties []Property
}

An Entity is the value type for a nested struct. This type is only used for a Property's Value.

type ErrFieldMismatch Uses

type ErrFieldMismatch struct {
    StructType reflect.Type
    FieldName  string
    Reason     string
}

ErrFieldMismatch is returned when a field is to be loaded into a different type than the one it was stored from, or when a field is missing or unexported in the destination struct. StructType is the type of the struct pointed to by the destination argument passed to Get or to Iterator.Next.

func (*ErrFieldMismatch) Error Uses

func (e *ErrFieldMismatch) Error() string

type GeoPoint Uses

type GeoPoint struct {
    Lat, Lng float64
}

GeoPoint represents a location as latitude/longitude in degrees.

type Iterator Uses

type Iterator interface {
    // Next returns the key of the next result. When there are no more results,
    // iterator.Done is returned as the error.
    //
    // If the query is not keys only and dst is non-nil, it also loads the entity
    // stored for that key into the struct pointer or PropertyLoadSaver dst, with
    // the same semantics and possible errors as for the Get function.
    Next(dst interface{}) (Key, error)
    // Cursor returns a cursor for the iterator's current location.
    Cursor() (Cursor, error)
}

Iterator is the result of running a query.

type Key Uses

type Key interface {
    Kind() string
    ID() int64
    Name() string
    ParentKey() Key
    Namespace() string
    SetNamespace(namespace string)

    String() string
    GobEncode() ([]byte, error)
    GobDecode(buf []byte) error
    MarshalJSON() ([]byte, error)
    UnmarshalJSON(buf []byte) error
    Encode() string
    Equal(o Key) bool
    Incomplete() bool
}

Key represents the datastore key for a stored entity.

type KeyLoader Uses

type KeyLoader interface {
    // PropertyLoadSaver is embedded because a KeyLoader
    // must also always implement PropertyLoadSaver.
    PropertyLoadSaver
    LoadKey(ctx context.Context, k Key) error
}

KeyLoader can store a Key.

type Middleware Uses

type Middleware interface {
    // AllocateIDs intercepts AllocateIDs operation.
    AllocateIDs(info *MiddlewareInfo, keys []Key) ([]Key, error)
    // PutMultiWithoutTx intercepts PutMulti without Transaction operation.
    PutMultiWithoutTx(info *MiddlewareInfo, keys []Key, psList []PropertyList) ([]Key, error)
    // PutMultiWithTx intercepts PutMulti with Transaction operation.
    PutMultiWithTx(info *MiddlewareInfo, keys []Key, psList []PropertyList) ([]PendingKey, error)
    // GetMultiWithoutTx intercepts GetMulti without Transaction operation.
    GetMultiWithoutTx(info *MiddlewareInfo, keys []Key, psList []PropertyList) error
    // GetMultiWithTx intercepts GetMulti with Transaction operation.
    GetMultiWithTx(info *MiddlewareInfo, keys []Key, psList []PropertyList) error
    // DeleteMultiWithoutTx intercepts DeleteMulti without Transaction operation.
    DeleteMultiWithoutTx(info *MiddlewareInfo, keys []Key) error
    // DeleteMultiWithTx intercepts DeleteMulti with Transaction operation.
    DeleteMultiWithTx(info *MiddlewareInfo, keys []Key) error
    // PostCommit will kicked after Transaction commit.
    PostCommit(info *MiddlewareInfo, tx Transaction, commit Commit) error
    // PostRollback will kicked after Transaction rollback.
    PostRollback(info *MiddlewareInfo, tx Transaction) error
    // Run intercepts Run query operation.
    Run(info *MiddlewareInfo, q Query, qDump *QueryDump) Iterator
    // GetAll intercepts GetAll operation.
    GetAll(info *MiddlewareInfo, q Query, qDump *QueryDump, psList *[]PropertyList) ([]Key, error)
    // Next intercepts Next operation.
    Next(info *MiddlewareInfo, q Query, qDump *QueryDump, iter Iterator, ps *PropertyList) (Key, error)
    // Count intercepts Count operation.
    Count(info *MiddlewareInfo, q Query, qDump *QueryDump) (int, error)
}

Middleware hooks to the Datastore's RPC and It can modify arguments and return values. see https://godoc.org/go.mercari.io/datastore/dsmiddleware

type MiddlewareInfo Uses

type MiddlewareInfo struct {
    Context     context.Context
    Client      Client
    Transaction Transaction
    Next        Middleware
}

MiddlewareInfo provides RPC's processing state.

type MultiError Uses

type MultiError []error

MultiError is returned by batch operations when there are errors with particular elements. Errors will be in a one-to-one correspondence with the input elements; successful elements will have a nil entry.

func (MultiError) Error Uses

func (m MultiError) Error() string

type PendingKey Uses

type PendingKey interface {
    StoredContext() context.Context
}

PendingKey represents the key for newly-inserted entity. It can be resolved into a Key by calling the Key method of Commit.

type Property Uses

type Property struct {
    // Name is the property name.
    Name string
    // Value is the property value. The valid types are:
    //	- int64
    //	- bool
    //	- string
    //	- float64
    //	- Key
    //	- time.Time (retrieved as local time)
    //	- GeoPoint
    //	- []byte (up to 1 megabyte in length)
    //	- *Entity (representing a nested struct)
    // Value can also be:
    //	- []interface{} where each element is one of the above types
    // This set is smaller than the set of valid struct field types that the
    // datastore can load and save. A Value's type must be explicitly on
    // the list above; it is not sufficient for the underlying type to be
    // on that list. For example, a Value of "type myInt64 int64" is
    // invalid. Smaller-width integers and floats are also invalid. Again,
    // this is more restrictive than the set of valid struct field types.
    //
    // A Value will have an opaque type when loading entities from an index,
    // such as via a projection query. Load entities into a struct instead
    // of a PropertyLoadSaver when using a projection query.
    //
    // A Value may also be the nil interface value; this is equivalent to
    // Python's None but not directly representable by a Go struct. Loading
    // a nil-valued property into a struct will set that field to the zero
    // value.
    Value interface{}
    // NoIndex is whether the datastore cannot index this property.
    // If NoIndex is set to false, []byte and string values are limited to
    // 1500 bytes.
    NoIndex bool
}

Property is a name/value pair plus some metadata. A datastore entity's contents are loaded and saved as a sequence of Properties. Each property name must be unique within an entity.

func SaveStruct Uses

func SaveStruct(ctx context.Context, src interface{}) ([]Property, error)

SaveStruct returns the properties from src as a slice of Properties. src must be a struct pointer.

type PropertyList Uses

type PropertyList []Property

PropertyList converts a []Property to implement PropertyLoadSaver.

func (*PropertyList) Load Uses

func (l *PropertyList) Load(ctx context.Context, p []Property) error

Load loads all of the provided properties into l. It does not first reset *l to an empty slice.

func (*PropertyList) Save Uses

func (l *PropertyList) Save(ctx context.Context) ([]Property, error)

Save saves all of l's properties as a slice of Properties.

type PropertyLoadSaver Uses

type PropertyLoadSaver interface {
    Load(ctx context.Context, ps []Property) error
    Save(ctx context.Context) ([]Property, error)
}

PropertyLoadSaver can be converted from and to a slice of Properties.

type PropertyTranslator Uses

type PropertyTranslator interface {
    ToPropertyValue(ctx context.Context) (interface{}, error)
    FromPropertyValue(ctx context.Context, p Property) (dst interface{}, err error)
}

PropertyTranslator is for converting the value of Property when saving and loading.

Code:

ctx := context.Background()
client, err := clouddatastore.FromContext(ctx)
if err != nil {
    panic(err)
}
defer client.Close()
defer testutils.CleanUpAllEntities(ctx, client)

ctx = context.WithValue(ctx, contextClient{}, client)

// Each fields are saved as datastore.Key and [] datastore.Key on Datastore.
type Group struct {
    OwnerID   UserID
    MemberIDs UserIDs
}

entity, err := datastore.SaveEntity(
    ctx, client.IncompleteKey("Group", nil),
    &Group{
        OwnerID:   147,
        MemberIDs: UserIDs{147, 258, 369},
    },
)
if err != nil {
    panic(err)
}

if key, ok := entity.Properties[0].Value.(datastore.Key); !ok {
    panic("unexpected state")
} else {
    fmt.Println("OwnerID", key.ID())
}
if keys, ok := entity.Properties[1].Value.([]datastore.Key); !ok {
    panic("unexpected state")
} else {
    for _, key := range keys {
        fmt.Println("MemberID", key.ID())
    }
}

Output:

OwnerID 147
MemberID 147
MemberID 258
MemberID 369

type Query Uses

type Query interface {
    Ancestor(ancestor Key) Query
    EventualConsistency() Query
    Namespace(ns string) Query
    Transaction(t Transaction) Query
    Filter(filterStr string, value interface{}) Query
    Order(fieldName string) Query
    Project(fieldNames ...string) Query
    Distinct() Query
    // NOT IMPLEMENTED ON APPENGINE DistinctOn(fieldNames ...string) *Query
    KeysOnly() Query
    Limit(limit int) Query
    Offset(offset int) Query
    Start(c Cursor) Query
    End(c Cursor) Query

    Dump() *QueryDump
}

Query represents a datastore query.

type QueryDump Uses

type QueryDump struct {
    Kind                string
    Ancestor            Key
    EventualConsistency bool
    Namespace           string
    Transaction         Transaction
    Filter              []*QueryFilterCondition
    Order               []string
    Project             []string
    Distinct            bool
    KeysOnly            bool
    Limit               int
    Offset              int
    Start               Cursor
    End                 Cursor
}

QueryDump provides information of executed query.

func (*QueryDump) String Uses

func (dump *QueryDump) String() string

type QueryFilterCondition Uses

type QueryFilterCondition struct {
    Filter string
    Value  interface{}
}

QueryFilterCondition provides information of filter of query.

type Transaction Uses

type Transaction interface {
    // Get is the transaction-specific version of the package function Get.
    // All reads performed during the transaction will come from a single consistent snapshot.
    // Furthermore, if the transaction is set to a serializable isolation level,
    // another transaction cannot concurrently modify the data that is read or modified by this transaction.
    Get(key Key, dst interface{}) error
    // GetMulti is a batch version of Get.
    GetMulti(keys []Key, dst interface{}) error
    // Put is the transaction-specific version of the package function Put.
    //
    // Put returns a PendingKey which can be resolved into a Key using the return value from a successful Commit.
    // If key is an incomplete key, the returned pending key will resolve to a unique key generated by the datastore.
    Put(key Key, src interface{}) (PendingKey, error)
    // PutMulti is a batch version of Put. One PendingKey is returned for each element of src in the same order.
    PutMulti(keys []Key, src interface{}) ([]PendingKey, error)
    // Delete is the transaction-specific version of the package function Delete.
    // Delete enqueues the deletion of the entity for the given key,
    // to be committed atomically upon calling Commit.
    Delete(key Key) error
    // DeleteMulti is a batch version of Delete.
    DeleteMulti(keys []Key) error

    // Commit applies the enqueued operations atomically.
    Commit() (Commit, error)
    // Rollback abandons a pending transaction.
    Rollback() error

    Batch() *TransactionBatch
}

Transaction represents a set of datastore operations to be committed atomically.

Operations are enqueued by calling the Put and Delete methods on Transaction (or their Multi-equivalents). These operations are only committed when the Commit method is invoked. To ensure consistency, reads must be performed by using Transaction's Get method or by using the Transaction method when building a query.

A Transaction must be committed or rolled back exactly once.

type TransactionBatch Uses

type TransactionBatch struct {
    Transaction Transaction
    // contains filtered or unexported fields
}

TransactionBatch provides Batch operation under Transaction. TransactionBatch does nothing until you call Exec(). This helps to reduce the number of RPCs.

func (*TransactionBatch) Delete Uses

func (b *TransactionBatch) Delete(key Key, h BatchErrHandler)

Delete Entity operation into the queue.

func (*TransactionBatch) Exec Uses

func (b *TransactionBatch) Exec() error

Exec will perform all the processing that was queued. This process is done recursively until the queue is empty. The return value may be MultiError, but the order of contents is not guaranteed. Exec() doesn't call Commit() or Rollback(), You should call that manually.

func (*TransactionBatch) Get Uses

func (b *TransactionBatch) Get(key Key, dst interface{}, h BatchErrHandler)

Get Entity operation into the queue.

func (*TransactionBatch) Put Uses

func (b *TransactionBatch) Put(key Key, src interface{}, h TxBatchPutHandler)

Put Entity operation into the queue. This operation doesn't Put to Datastore immediately. If a h is provided, it passes the processing result to the handler, and treats the return value as the value of the result of Putting.

type TxBatchPutHandler Uses

type TxBatchPutHandler func(pKey PendingKey, err error) error

TxBatchPutHandler represents Entity's individual callback when batching Put with transaction processing.

Directories

PathSynopsis
aedatastorePackage aedatastore provides AppEngine Datastore implementation of datastore.Client.
aeprodtestPackage aeprodtest has nothing to use in your application.
boomPackage boom handles the troublesome processing of datastore.Key automatically.
clouddatastorePackage clouddatastore provides Cloud Datastore implementation of datastore.Client.
dsmiddlewarePackage dsmiddleware does not have anything, but it has a middleware group as a subpackage.
dsmiddleware/aememcachePackage aememcache handles Put, Get etc to Datastore and provides caching by AppEngine's Memcache.
dsmiddleware/chaosrpcPackage chaosrpc generates chaos very efficiently! This package will randomly error all RPCs with a probability of 20%.
dsmiddleware/dsmemcachePackage dsmemcache handles Put, Get etc to Datastore and provides caching by memcached.
dsmiddleware/fishbonePackage fishbone automatically rewrites the behavior based on KeysOnly + Get by Key when Run or GetAll Query, contributing to reducing the amount of charge.
dsmiddleware/localcachePackage localcache handles Put, Get etc to Datastore and provides caching by machine local memory.
dsmiddleware/noopPackage noop does "no-operation".
dsmiddleware/rediscachePackage rediscache handles Put, Get etc to Datastore and provides caching by Redis.
dsmiddleware/rpcretryPackage rpcretry automatically retries when some RPCs end in error.
dsmiddleware/splitopPackage splitop provides a avoid Datastore's limitation.
dsmiddleware/storagecachePackage storagecache provides a mechanism for using various storage as datastore's cache.
internal
internal/c/fieldsPackage fields provides a view of the fields of a struct that follows the Go rules, amended to consider tags and case insensitivity.
internal/shared
internal/testutils
testbed
testsuitePackage testsuite has nothing to use in your application.
testsuite/dsmiddleware/dslog
testsuite/dsmiddleware/fishbone
testsuite/dsmiddleware/localcache
testsuite/dsmiddleware/rpcretry
testsuite/favcliptools
testsuite/realworld/recursivebatch
testsuite/realworld/tbf

Package datastore imports 17 packages (graph) and is imported by 26 packages. Updated 2019-09-02. Refresh now. Tools for package owners.