README
¶
S3 Stow Implementation
Location = Amazon S3
Container = Bucket
Item = File
Helpful Links:
http://docs.aws.amazon.com/sdk-for-go/api/service/s3/#example_S3_ListBuckets
SDK Notes:
- Metadata of an S3 Object can only be set when the Object is created.
Concerns:
- An AWS account may have credentials which temporarily modifies permissions. This is specified by a token value. This feature is implemented but disabled and added as a TODO.
Things to know:
-
Paging for the list of containers doesn't exist yet, this is because there's a hard limit of about 100 containers for every account.
-
A client is required to provide a region. Manipulating buckets that reside within other regions isn't possible.
The init function of every implementation of stow must call stow.Register.
stow.Register accepts a few things:
Kind, a string argument respresenting the name of the location.
makefn a function that accepts any type that conforms to the stow.Config
interface. It first validates the values of the Config argument, and then
attempts to use the configuration to create a new client. If successful, An
instance of a data type that conforms to the stow.Location interface is
created. This Location should have fields that contain the client and
configuration.
Further calls in the hierarchy of a Location, Container, and Item depend on the values of the configuration + the client to send and receive information.
kingmatchfna function that ensures that a given URL matches theKindof the type of storage.
*stow.Register(kind string, makefn func(Config) (Locaion, error), kindmatchfn func(url.URL) bool)
-
Adds
kindandmakefninto a map that contains a list of locations. -
Adds
kindto a slice that contains all of the different kinds. -
Adds
kindas part of an anonymous function which validates the scheme of the url.URL
Once the stow.Register function is completed, a location of the given kind is returned.
Documentation
¶
Overview ¶
Package s3 provides an abstraction of Amazon S3 (Simple Storage Service). An S3 Bucket is a Stow Container and an S3 Object is a Stow Item. Recall that nested directories exist within S3.
Usage and Credentials ¶
There are three separate pieces of information required by Stow to have access to an S3 Stow Location: an AWS User's ACCESS_KEY_ID and SECRET_KEY fields, as well as the physical region of the S3 Endpoint. Ensure that the AWS User whose credentials are used to manipulate the S3 endpoint has permissions to do so.
stow.Dial requires both a string value ("s3") of the particular Stow Location Kind and a stow.Config instance. The stow.Config instance requires three entries with the specific key value attributes:
- a key of s3.ConfigAccessKeyID with a value of the AWS account's Access Key ID - a key of s3.ConfigSecretKey with a value of the AWS account's Secret Key - a key of s3.ConfigRegion with a value of the S3 endpoint's region (in all lowercase)
Location ¶
The s3.location methods allow the retrieval of an S3 endpoint's Bucket or list of Buckets (Container or Containers). A stow.Item representation of an S3 Object can also be retrieved based on the Object's URL (ItemByURL).
Additional s3.location methods provide capabilities to create and remove S3 Buckets (CreateContainer or RemoveContainer, respectively).
Container ¶
There are s3.container methods which can retrieve an S3 Bucket's:
- name (ID or Name) - Object or complete list of Objects (Item or Items) - region
Additional s3.container methods give Stow the ability to:
- remove an S3 Bucket (RemoveItem) - update or create an S3 Object (Put)
Item ¶
Methods within an s3.item allow the retrieval of an S3 Object's: - name (ID or name) - URL (ItemByUrl) - size in bytes (Size) - S3 specific metadata (Metadata, key value pairs usually found within the console) - last modified date (LastMod) - Etag (Etag) - content (Open)
Index ¶
Constants ¶
const ( // ConfigAuthType is an optional argument that defines whether to use an IAM role or access key based auth ConfigAuthType = "auth_type" // ConfigAccessKeyID is one key of a pair of AWS credentials. ConfigAccessKeyID = "access_key_id" // ConfigSecretKey is one key of a pair of AWS credentials. ConfigSecretKey = "secret_key" // ConfigRegion represents the region/availability zone of the session. ConfigRegion = "region" // ConfigEndpoint is optional config value for changing s3 endpoint // used for e.g. minio.io ConfigEndpoint = "endpoint" // ConfigCACertFile is optional config value for providing path to cacert file for custom endpoint like Minio // to establish TLS secure connection ConfigCACertFile = "cacert_file" // ConfigCACertData is optional config value for providing path to cacert data for custom endpoint like Minio // to establish TLS secure connection ConfigCACertData = "cacert_data" // ConfigDisableSSL is optional config value for disabling SSL support on custom endpoints // Its default value is "false", to disable SSL set it to "true". ConfigDisableSSL = "disable_ssl" // ConfigV2Signing is an optional config value for signing requests with the v2 signature. // Its default value is "false", to enable set to "true". // This feature is useful for s3-compatible blob stores -- ie minio. ConfigV2Signing = "v2_signing" )
const Kind = "s3"
Kind represents the name of the location/storage type.
Variables ¶
This section is empty.
Functions ¶
Types ¶
This section is empty.
Source Files