ddbmap

package module
v0.0.11 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 10, 2019 License: MIT Imports: 18 Imported by: 0

README

Build Status GoDoc license Go Report Card

ddbmap

ddbmap is a Go (golang) library and module that presents a map-like view of an AWS DynamoDB table.

Example

package main

import (
    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/aws/external"
    "github.com/shawnsmithdev/ddbmap"
    "fmt"
)

type Person struct {
    Id   int
    Name string
    Age  int
}

func main() {
    awsCfg, _ := external.LoadDefaultAWSConfig()
    awsCfg.Retryer = aws.DefaultRetryer{NumMaxRetries: 100}

    // Assumes table already exists, will auto-discover key names
    tCfg := ddbmap.TableConfig{
        TableName:         "TestTable",
        ValueUnmarshaller: ddbmap.UnmarshallerForType(Person{}),
    }
    people, _ := tCfg.NewMap(awsCfg)

    // put
    p1 := Person{Id: 1, Name: "Bob", Age: 20}
    err := people.Store(p1)

    // get
    p2, loaded, err := people.Load(Person{Id: 1})
    if loaded && err == nil {
        fmt.Println(p2.(Person)) // same as p1
    }

    // iterate
    err = people.Range(func(p3 interface{}) bool {
        fmt.Println(p3.(Person)) // same as p1
        return true
    })
}

Your table, as a map

One way to view a DynamoDB table is as kind of a hashmap in the cloud.

This library ignores some of the features of DynamoDB, such as range key queries and batching, to provide a simple API to access a table.

  • Get a single record
  • Put a single record
  • Delete a single record
  • Conditional Put If Absent
  • Iterate over all records (serially or in parallel)

Note that you must either use capitalized DynamoDB field names, or add struct tags like dynamodbav to rename exported fields.

Item API

The ddbmap.ItemMap API may be used by implementing ddbmap.Itemable to handle conversions between the Go and DynamoDB type system, with or without using reflection.

Conditional Updates (versions)

Conditional updates, where the condition is stronger than just a record's absence, is supported by defining a numerical version field and configuring VersionName in the TableConfig to the name of that field.

Time To Live

If the TimeToLiveDuration field in the TableConfig is set, each record will be stored with a new number field set to Unix epoch seconds of time the record was stored, plus the configured duration. This is useful when using the TTL feature of DynamoDB tables. The field name is TTL by default but can be changed with TimeToLiveName in TableConfig.

Dependencies

This library depends on the AWS Go SDK v2 and golang.org/x/sync. If building with a go version older than 1.11, you will need to install these dependencies manually.

go get -u github.com/aws/aws-sdk-go-v2
go get -u golang.org/x/sync

TODO

  • Test range early termination
  • Test other set types, null

Documentation

Overview

Package ddbmap presents a map-like interface for DynamoDB tables.

Index

Constants

View Source
const (

	// DefaultTimeToLiveName is used if the TTL duration is set but the ttl attribute name is not.
	DefaultTimeToLiveName = "TTL"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type CreateTableOptions

type CreateTableOptions struct {
	// CreateTableIfAbsent determines if a table should be created when missing.
	// If true, users must also set the HashKeyType and, if there is a range key, the RangeKeyType,
	// and may also choose to set CreateTableReadCapacity and CreateTableWriteCapacity
	CreateTableIfAbsent bool
	// CreateTableReadCapacity is the read capacity of the new table, if created. 1 is used if less than 1.
	CreateTableReadCapacity int
	// CreateTableReadCapacity is the write capacity of the new table, if created. 1 is used if less than 1.
	CreateTableWriteCapacity int
	// The type of the hash key attribute.
	HashKeyType dynamodb.ScalarAttributeType
	// The type of the range key attribute, if any.
	RangeKeyType dynamodb.ScalarAttributeType
	// If true, Server Side Encryption (SSE) is enabled.
	ServerSideEncryption bool
}

CreateTableOptions contain values used when creating new DynamoDB tables

type DynamoMap

type DynamoMap struct {
	TableConfig
	Client *dynamodb.Client
}

DynamoMap is a map view of a DynamoDB table. *DynamoMap implements both Map and ItemMap.

func (*DynamoMap) CreateTable added in v0.0.6

func (d *DynamoMap) CreateTable() error

CreateTable creates a new table.

func (*DynamoMap) Delete

func (d *DynamoMap) Delete(key interface{}) (err error)

Delete delete the value stored under the same key(s) as the given value, if any.

func (*DynamoMap) DeleteItem

func (d *DynamoMap) DeleteItem(key Itemable) error

DeleteItem deletes any existing item with the same key(s) as the given item.

func (*DynamoMap) DescribeTable added in v0.0.6

func (d *DynamoMap) DescribeTable(setKeys bool) (status dynamodb.TableStatus, err error)

DescribeTable checks the table description, returning the table status or any errors. If the status is CREATING, the call will poll waiting for the status to change. If the table does not exist, the status will be empty. If setKeys is true, the keys will be set using the table description.

func (*DynamoMap) DisableTTL added in v0.0.11

func (d *DynamoMap) DisableTTL() error

DisableTTL will disable TimeToLive on the table if it is enabled.

func (*DynamoMap) EnableTTL added in v0.0.11

func (d *DynamoMap) EnableTTL() error

EnableTTL will enable TimeToLive on the table if it is not enabled, or update it if the configured time to live attribute name does not match the one currently in use.

func (*DynamoMap) Load

func (d *DynamoMap) Load(key interface{}) (value interface{}, ok bool, err error)

Load returns any value stored under the same key(s) as the given value, if any. The ok result indicates if there a value was found for the key.

func (*DynamoMap) LoadItem

func (d *DynamoMap) LoadItem(key Itemable) (item Item, ok bool, err error)

LoadItem returns the existing item, if present, with the same key(s) as the given item. The ok result returns true if the value was found.

func (*DynamoMap) LoadOrStore

func (d *DynamoMap) LoadOrStore(val interface{}) (actual interface{}, loaded bool, err error)

LoadOrStore returns any value stored that has the same key as the given value, if any, else stores and returns the given value. The loaded result is true if the value was loaded, false if stored. The first argument is ignored.

func (*DynamoMap) LoadOrStoreItem

func (d *DynamoMap) LoadOrStoreItem(val Itemable) (actual Item, loaded bool, err error)

LoadOrStoreItem returns the existing item, if present, with the same key(s) as the given item. Otherwise, it stores and returns the given item. The loaded result is true if the value was loaded, false if stored.

func (*DynamoMap) Range

func (d *DynamoMap) Range(consumer func(value interface{}) bool) error

Range iterates over the map and applies the given function to every value. Iteration eventually stops if the given function returns false. The consumed key will be nil unless KeyUnmarshaller is set. The consumed value will be an Item unless ValueUnmarshaller is set.

func (*DynamoMap) RangeItems

func (d *DynamoMap) RangeItems(consumer func(Item) bool) error

RangeItems calls the given consumer for each stored item. Iteration eventually stops if the given function returns false.

func (*DynamoMap) Store

func (d *DynamoMap) Store(val interface{}) (err error)

Store stores the given value. The first argument is ignored.

func (*DynamoMap) StoreIfAbsent

func (d *DynamoMap) StoreIfAbsent(val interface{}) (stored bool, err error)

StoreIfAbsent stores the given value if there is no existing value with the same key(s), returning true if stored. The first argument is ignored.

func (*DynamoMap) StoreIfVersion

func (d *DynamoMap) StoreIfVersion(val interface{}, version int64) (ok bool)

StoreIfVersion stores the given item if there is an existing item with the same key(s) and the given version. Returns true if the item was stored.

func (*DynamoMap) StoreItem

func (d *DynamoMap) StoreItem(val Itemable) error

StoreItem stores the given item, clobbering any existing item with the same key(s).

func (*DynamoMap) StoreItemIfAbsent

func (d *DynamoMap) StoreItemIfAbsent(val Itemable) (stored bool, err error)

StoreItemIfAbsent stores the given item if there is no existing item with the same key(s), returning true if stored.

func (*DynamoMap) StoreItemIfVersion

func (d *DynamoMap) StoreItemIfVersion(item Itemable, version int64) (ok bool, err error)

StoreItemIfVersion stores the given item if there is an existing item with the same key(s) and the given version. Returns true if the item was stored.

type Item

Item is a type underlied by the map type output by dynamodbattribute.MarshalMap. This represents a single row in a DynamoDB table or a 'Map' in the DynamoDB type system.

func MarshalItem

func MarshalItem(val interface{}) (Item, error)

MarshalItem will marshal a value into an Item using dynamodbattribute.MarshalMap, unless this can be avoided because the value is already an Item or is Itemable.

func (Item) AsItem

func (item Item) AsItem() Item

AsItem directly returns this item.

func (Item) Exists

func (item Item) Exists(attr string) bool

Exists returns true if the given attribute exists, even if it is null.

func (Item) IsNull

func (item Item) IsNull(attr string) bool

IsNull returns true if attribute exists, but is null.

func (Item) IsPresent

func (item Item) IsPresent(attr string) bool

IsPresent returns true if attribute exists and is not null.

func (Item) Project

func (item Item) Project(attrs ...string) Item

Project returns a new item based on this one, but with only the specified attributes.

func (Item) String added in v0.0.11

func (item Item) String() string

String returns a string representation of the content of the item

type ItemMap

type ItemMap interface {
	// DeleteItem deletes any existing item with the same key(s) as the given item.
	DeleteItem(keys Itemable) error

	// LoadItem returns the existing item, if present, with the same key(s) as the given item.
	// The ok result returns true if the value was found.
	LoadItem(keys Itemable) (item Item, ok bool, err error)

	// StoreItem stores the given item, clobbering any existing item with the same key(s).
	StoreItem(item Itemable) error

	// LoadOrStoreItem returns the existing item, if present, with the same key(s) as the given item.
	// Otherwise, it stores and returns the given item.
	// The loaded result is true if the value was loaded, false if stored.
	LoadOrStoreItem(item Itemable) (actual Item, loaded bool, err error)

	// StoreIfAbsent stores the given value if there is no existing value with the same key(s),
	// returning true if stored.
	StoreIfAbsent(val interface{}) (stored bool, err error)

	// StoreItemIfAbsent stores the given item if there is no existing item with the same key(s),
	// returning true if stored.
	StoreItemIfAbsent(item Itemable) (stored bool, err error)

	// RangeItems calls the given consumer for each stored item.
	// If the consumer returns false, range eventually stops the iteration.
	RangeItems(consumer func(Item) (resume bool)) error

	// StoreIfVersion stores the given item if there is an existing item with the same key(s) and the given version.
	// Returns true if the item was stored.
	StoreIfVersion(val interface{}, version int64) (ok bool)

	// StoreItemIfVersion stores the given item if there is an existing item with the same key(s) and the given version.
	// Returns true if the item was stored.
	StoreItemIfVersion(item Itemable, version int64) (ok bool, err error)
}

ItemMap is like Map except that it supports Itemable types and more conditional operations.

type ItemUnmarshaller

type ItemUnmarshaller func(Item) (interface{}, error)

ItemUnmarshaller is a function that can convert an Item into some other type

func UnmarshallerForType

func UnmarshallerForType(template interface{}) ItemUnmarshaller

UnmarshallerForType creates a new ItemUnmashaller function from a template. The template may be any value of the struct type you want items to be unmarshalled into, such as the zero value.

type Itemable

type Itemable interface {
	AsItem() Item
}

Itemable is implemented by types that can directly build representations of their data in the DynamoDB type system. This allows users to take direct control of how their data is presented to DynamoDB. Item also implements Itemable, by returning itself, so any method that take Itemable can accept an Item directly.

type KeyFromValue added in v0.0.8

type KeyFromValue func(interface{}) (interface{}, error)

KeyFromValue is a function that can generate a hashable key from a value.

type Map

type Map interface {
	// Delete delete the value stored under the given key, if any.
	Delete(toDelete interface{}) error
	// Load returns the value stored under the given key, if any.
	// The ok result indicates if there a value was found for the key.
	Load(toLoad interface{}) (result interface{}, ok bool, err error)
	// LoadOrStore returns the value stored under the same key as the given value, if any,
	// else stores and returns the given value.
	// The loaded result is true if the value was loaded, false if stored.
	LoadOrStore(value interface{}) (actual interface{}, loaded bool, err error)
	// Range iterates over the map and applies the given function to every value.
	// Range stops iteration if the given function returns false.
	Range(consumer func(value interface{}) (resume bool)) error
	// Store stores the given value.
	Store(toStore interface{}) error
	// Store stores the given value is no value with the same key is stored.
	// The stored result is true if the value was stored.
	StoreIfAbsent(toStore interface{}) (stored bool, err error)
}

Map implements a key-value store where keys can always be determined from values.

func NewSyncMap added in v0.0.8

func NewSyncMap(keyFromValue KeyFromValue) Map

NewSyncMap creates a new Map that uses sync.Map as storage. This is intended for use in tests.

type TableConfig

type TableConfig struct {
	// The name of the table.
	TableName string
	// The name of the hash key attribute.
	HashKeyName string
	// The name of the range key attribute, if any.
	RangeKeyName string
	// The name of the numeric version field, if any.
	// Used only for those conditional methods that use versions.
	VersionName string
	// The name of the ttl field, if any.
	// If empty and TimeToLiveDuration is not zero, DefaultTimeToLiveName ("TTL") will be used.
	// A ttl field should be either an int type or dynamodbattribute.UnixTime.
	TimeToLiveName string
	// The Time To Live Duration, if any.
	TimeToLiveDuration time.Duration
	// The concurrency used in table scans (Range calls).
	// If less than 2, scan is done serially.
	ScanConcurrency int
	// If the client should use strongly consistent reads.
	// This costs twice as much as eventually consistent reads.
	ReadWithStrongConsistency bool
	// If true, debug logging in this library is enabled.
	Debug bool
	// Logger is the logger used by this library for debug and error logging.
	Logger aws.Logger
	// ValueUnmarshaller can be used to change what is returned by Load, LoadOrStore, and Range.
	// These methods return an Item if ValueUnmarshaller is nil.
	// If ValueUnmarshaller is not nil, the result of passing the value item to the unmarshaller
	// is returned as the value instead of the item.
	ValueUnmarshaller ItemUnmarshaller
	// Options for creating the table
	CreateTableOptions
}

TableConfig holds details about a specific DynamoDB table and some options for using it.

func (TableConfig) NewMap

func (tc TableConfig) NewMap(cfg aws.Config) (*DynamoMap, error)

NewMap creates a map view of a DynamoDB table from a TableConfig. If the table does not exist or is being deleted or there is an error, the pointer result will be nil. If ScanTableIfNotExists is true and the table does not exist, it will be created. If ScanTableIfNotExists is false and the key names are not set, they will be looked up. If the logger has not been configured, either the AWS config's logger (if present) or stdout will be used.

func (TableConfig) Ranged

func (tc TableConfig) Ranged() bool

Ranged returns true if RangeKeyName is not empty

func (TableConfig) ToKeyItem

func (tc TableConfig) ToKeyItem(item Item) Item

ToKeyItem returns an item with only the configured key(s) copied from the given item.

Directories

Path Synopsis
Package ddbconv can be used to convert between dynamodb.AttributeValue and the Go type system Some of these functions provide little more than improved readability.
Package ddbconv can be used to convert between dynamodb.AttributeValue and the Go type system Some of these functions provide little more than improved readability.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL