goon: Index | Files

package goon

import ""

Package goon provides an autocaching interface to the app engine datastore similar to the python NDB package.

Goon differs from the datastore package in various ways: it remembers the appengine Context, which need only be specified once at creation time; kinds need not be specified as they are computed, by default, from a type's name; keys are inferred from specially-tagged fields on types, removing the need to pass key objects around.

In general, the difference is that Goon's API is identical to the datastore API, it's just shorter.

Keys in Goon are stored in the structs themselves. Below is an example struct with a field to specify the id (see the Key Specifications section below for full documentation).

type User struct {
	Id    string `datastore:"-" goon:"id"`
	Name  string

Thus, to get a User with id 2:

userid := 2
g := goon.NewGoon(r)
u := &User{Id: userid}

Key Specifications

For both the Key and KeyError functions, src must be a S or *S for some struct type S. The key is extracted based on various fields of S. If a field of type int64 or string has a struct tag named goon with value "id", it is used as the key's id. If a field of type *datastore.Key has a struct tag named goon with value "parent", it is used as the key's parent. If a field of type string has a struct tag named goon with value "kind", it is used as the key's kind. The "kind" field supports an optional second parameter which is the default kind name. If no kind field exists, the struct's name is used. These fields should all have their datastore field marked as "-".

Example, with kind User:

type User struct {
	Id    string `datastore:"-" goon:"id"`
	Read  time.Time

Example, with kind U if _kind is the empty string:

type User struct {
	_kind string `goon:"kind,U"`
	Id    string `datastore:"-" goon:"id"`
	Read  time.Time

To override kind of a single entity to UserKind:

u := User{_kind: "UserKind"}

An example with both parent and kind:

type UserData struct {
	Id     string         `datastore:"-" goon:"id"`
	_kind  string         `goon:"kind,UD"`
	Parent *datastore.Key `datastore:"-" goon:"parent"`
	Data   []byte


Datastore interaction with: Get, GetMulti, Put, PutMulti, Delete, DeleteMulti, Queries.

All key-based operations backed by memory and memcache.

Per-request, in-memory cache: fetch the same key twice, the second request is served from local memory.

Intelligent multi support: running GetMulti correctly fetches from memory, then memcache, then the datastore; each tier only sends keys off to the next one if they were missing.

Memcache control variance: long memcache requests are cancelled.

Transactions use a separate context, but locally cache any results on success.

Automatic kind naming: struct names are inferred by reflection, removing the need to manually specify key kinds.

Simpler API than appengine/datastore.

API comparison between goon and datastore

put with incomplete key


type Group struct {
	Name string
c := appengine.NewContext(r)
g := &Group{Name: "name"}
k := datastore.NewIncompleteKey(c, "Group", nil)
err := datastore.Put(c, k, g)


type Group struct {
	Id   int64 `datastore:"-" goon:"id"`
	Name string
n := goon.NewGoon(r)
g := &Group{Name: "name"}
err := n.Put(g)

get with known key


type Group struct {
	Name string
c := appengine.NewContext(r)
g := &Group{}
k := datastore.NewKey(c, "Group", "", 1, nil)
err := datastore.Get(c, k, g)


type Group struct {
	Id   int64 `datastore:"-" goon:"id"`
	Name string
n := goon.NewGoon(r)
g := &Group{Id: 1}
err := n.Get(g)

Memcache Control Variance

Memcache is generally fast. When it is slow, goon will timeout the memcache requests and proceed to use the datastore directly. The memcache put and get timeout variables determine how long to wait for various kinds of requests. The default settings were determined experimentally and should provide reasonable defaults for most applications.


PropertyLoadSaver support

Structs that implement the PropertyLoadSaver interface are guaranteed to call the Save() method once and only once per Put/PutMulti call and never elsewhere. Similarly the Load() method is guaranteed to be called once and only once per Get/GetMulti/GetAll/Next call and never elsewhere.


Package Files

cache.go doc.go entity.go goon.go query.go


var (
    // LogErrors issues appengine.Context.Errorf on any error.
    LogErrors = true
    // LogTimeoutErrors issues appengine.Context.Warningf on memcache timeout errors.
    LogTimeoutErrors = false

    // MemcachePutTimeoutThreshold is the number of bytes after which the large
    // timeout setting is added to the small timeout setting. This repeats.
    // Which means that with a 20 KiB threshold and a 100 KiB memcache payload,
    // the final timeout is MemcachePutTimeoutSmall + 5*MemcachePutTimeoutLarge
    MemcachePutTimeoutThreshold = 20 * 1024 // 20 KiB
    // MemcachePutTimeoutSmall is the minimum time to wait during memcache
    // Put operations before aborting them and using the datastore.
    MemcachePutTimeoutSmall = 5 * time.Millisecond
    // MemcachePutTimeoutLarge is the amount of extra time to wait for larger
    // memcache Put requests. See also MemcachePutTimeoutThreshold.
    MemcachePutTimeoutLarge = 1 * time.Millisecond
    // MemcacheGetTimeout is the amount of time to wait for all memcache Get
    // requests, per key fetched. Because we can't really know how big entities
    // we are requesting, this setting should be for the maximum size entity.
    // The final timeout is limited to the number of maximum sized entities
    // an RPC result can contain, so the timeout won't grow insanely large
    // if you're fetching a ton of small entities.
    MemcacheGetTimeout = 31250 * time.Microsecond // 31.25 milliseconds

    // IgnoreFieldMismatch decides whether *datastore.ErrFieldMismatch errors
    // should be silently ignored. This allows you to easily remove fields from structs.
    IgnoreFieldMismatch = true
var MemcacheKey = func(k *datastore.Key) string {
    return k.Encode()

MemcacheKey returns the string form of the provided datastore key.

func DefaultKindName Uses

func DefaultKindName(src interface{}) string

DefaultKindName is the default implementation to determine the Kind an Entity has. Returns the basic Type of the src (no package name included).

func NotFound Uses

func NotFound(err error, idx int) bool

NotFound returns true if err is an appengine.MultiError and err[idx] is a datastore.ErrNoSuchEntity.

type Goon Uses

type Goon struct {
    Context context.Context

    // KindNameResolver is used to determine what Kind to give an Entity.
    // Defaults to DefaultKindName
    KindNameResolver KindNameResolver
    // contains filtered or unexported fields

Goon holds the app engine context and the request memory cache.

func FromContext Uses

func FromContext(c context.Context) *Goon

FromContext creates a new Goon object from the given appengine Context. Useful with profiling packages like appstats.

func NewGoon Uses

func NewGoon(r *http.Request) *Goon

NewGoon creates a new Goon object from the given request.

func (*Goon) Count Uses

func (g *Goon) Count(q *datastore.Query) (int, error)

Count returns the number of results for the query.

func (*Goon) Delete Uses

func (g *Goon) Delete(src interface{}) error

Delete deletes the provided entity. Takes either *S or *datastore.Key.

func (*Goon) DeleteMulti Uses

func (g *Goon) DeleteMulti(src interface{}) error

DeleteMulti is a batch version of Delete. Takes either []*S or []*datastore.Key.

func (*Goon) FlushLocalCache Uses

func (g *Goon) FlushLocalCache()

FlushLocalCache clears the local memory cache.

func (*Goon) Get Uses

func (g *Goon) Get(dst interface{}) error

Get loads the entity based on dst's key into dst If there is no such entity for the key, Get returns datastore.ErrNoSuchEntity.

func (*Goon) GetAll Uses

func (g *Goon) GetAll(q *datastore.Query, dst interface{}) ([]*datastore.Key, error)

GetAll runs the query and returns all the keys that match the query, as well as appending the values to dst, setting the goon key fields of dst, and caching the returned data in local memory.

For "keys-only" queries dst can be nil, however if it is not, then GetAll appends zero value structs to dst, only setting the goon key fields. No data is cached with "keys-only" queries.


func (*Goon) GetMulti Uses

func (g *Goon) GetMulti(dst interface{}) error

GetMulti is a batch version of Get.

dst must be a *[]S, *[]*S, *[]I, []S, []*S, or []I, for some struct type S, or some interface type I. If *[]I or []I, each element must be a struct pointer.

func (*Goon) Key Uses

func (g *Goon) Key(src interface{}) *datastore.Key

Key is the same as KeyError, except nil is returned on error or if the key is incomplete.

func (*Goon) KeyError Uses

func (g *Goon) KeyError(src interface{}) (*datastore.Key, error)

KeyError returns the key of src based on its properties.

func (*Goon) Kind Uses

func (g *Goon) Kind(src interface{}) string

Kind returns src's datastore Kind or "" on error.

func (*Goon) Put Uses

func (g *Goon) Put(src interface{}) (*datastore.Key, error)

Put saves the entity src into the datastore based on src's key k. If k is an incomplete key, the returned key will be a unique key generated by the datastore.

func (*Goon) PutMulti Uses

func (g *Goon) PutMulti(src interface{}) ([]*datastore.Key, error)

PutMulti is a batch version of Put.

src must be a *[]S, *[]*S, *[]I, []S, []*S, or []I, for some struct type S, or some interface type I. If *[]I or []I, each element must be a struct pointer.

func (*Goon) Run Uses

func (g *Goon) Run(q *datastore.Query) *Iterator

Run runs the query.

func (*Goon) RunInTransaction Uses

func (g *Goon) RunInTransaction(f func(tg *Goon) error, opts *datastore.TransactionOptions) error

RunInTransaction runs f in a transaction. It calls f with a transaction context tg that f should use for all App Engine operations. Neither cache nor memcache are used or set during a transaction.

Otherwise similar to appengine/datastore.RunInTransaction:

type Iterator Uses

type Iterator struct {
    // contains filtered or unexported fields

Iterator is the result of running a query.

func (*Iterator) Cursor Uses

func (t *Iterator) Cursor() (datastore.Cursor, error)

Cursor returns a cursor for the iterator's current location.

func (*Iterator) Next Uses

func (t *Iterator) Next(dst interface{}) (*datastore.Key, error)

Next returns the entity of the next result. When there are no more results, datastore.Done is returned as the error.

If the query is not keys only and dst is non-nil, it also loads the entity stored for that key into the struct pointer dst, with the same semantics and possible errors as for the Get function. This result is cached in memory.

If the query is keys only and dst is non-nil, dst will be given the right id.

Refer to appengine/datastore.Iterator.Next:

type KindNameResolver Uses

type KindNameResolver func(src interface{}) string

KindNameResolver takes an Entity and returns what the Kind should be for Datastore.

Package goon imports 19 packages (graph) and is imported by 44 packages. Updated 2019-06-05. Refresh now. Tools for package owners.