README
¶
knoxite S3 storage backend
URL Structure
The amazons3 storage backend registers the amazons3:// handler. To use it, supply a URL of the following format either as a -r parameter to your knoxite invocation or to the configuration system:
amazons3://<bucket-name>/[prefix/][?region=REGION]&[endpoint=URL]&[force_path_style=true]
Optionally, some configuration parameters may be supplied as GET style parameters:
| Parameter Name | Valid values | Description |
|---|---|---|
region |
valid AWS region descriptors | AWS Region. If this configuration is not specified, the backend falls back to other means of configuration, such as the AWS_REGION environment variable. |
endpoint |
valid URLs | For testing purposes only. This Parameter can be used to make S3 requests against backends other than those provided by AWS. This is not recommended. |
force_path_style |
true |
Use this parameter to force the underlying S3 SDK to make "path style" requests against the Amazon S3 backend. We don't recommend using this parameter if not required for compatibility reasons as path style request are being sunset by AWS. |
S3 bucket setup
For security reasons, we recommend setting up an S3 bucket and using short-lived credentials such as EC2 instance roles whenever feasible.
In situations where this is not possible, long-lived credentials such as IAM users are a good option.
In both cases, we recommend sticking to the principle of least privilege as outlined in AWS IAM's best practices.
For inspiration, you can have a look at the AWS Cloudformation template we use to deploy testing infrastructure for this backend.
Documentation
¶
Index ¶
- type AmazonS3Client
- type AmazonS3StorageBackend
- func (*AmazonS3StorageBackend) AvailableSpace() (uint64, error)
- func (*AmazonS3StorageBackend) Close() error
- func (*AmazonS3StorageBackend) CreatePath(path string) error
- func (backend *AmazonS3StorageBackend) DeleteFile(path string) error
- func (*AmazonS3StorageBackend) Description() string
- func (backend *AmazonS3StorageBackend) Location() string
- func (*AmazonS3StorageBackend) NewBackend(url url.URL) (knoxite.Backend, error)
- func (*AmazonS3StorageBackend) Protocols() []string
- func (backend *AmazonS3StorageBackend) ReadFile(path string) ([]byte, error)
- func (backend *AmazonS3StorageBackend) Stat(path string) (uint64, error)
- func (backend *AmazonS3StorageBackend) WriteFile(path string, data []byte) (uint64, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AmazonS3Client ¶
type AmazonS3Client interface {
GetObject(input *s3.GetObjectInput) (*s3.GetObjectOutput, error)
HeadObject(input *s3.HeadObjectInput) (*s3.HeadObjectOutput, error)
PutObject(input *s3.PutObjectInput) (*s3.PutObjectOutput, error)
DeleteObject(input *s3.DeleteObjectInput) (*s3.DeleteObjectOutput, error)
}
AmazonS3Client is an abstraction over the functions used from aws-golang-sdk's S3 client. Using this instead of the client directly makes it easier to mock.
type AmazonS3StorageBackend ¶
type AmazonS3StorageBackend struct {
knoxite.StorageFilesystem
// contains filtered or unexported fields
}
AmazonS3StorageBackend is the storage backend that adapts knoxite's backend interface to Amazon S3.
func (*AmazonS3StorageBackend) AvailableSpace ¶
func (*AmazonS3StorageBackend) AvailableSpace() (uint64, error)
AvailableSpace returns the free space on this backend.
func (*AmazonS3StorageBackend) Close ¶
func (*AmazonS3StorageBackend) Close() error
Close closes the StorageFileSystem.
func (*AmazonS3StorageBackend) CreatePath ¶
func (*AmazonS3StorageBackend) CreatePath(path string) error
CreatePath creates a folder in a filesystem-like storage backend.
func (*AmazonS3StorageBackend) DeleteFile ¶
func (backend *AmazonS3StorageBackend) DeleteFile(path string) error
DeleteFile deletes a file from the storage backend.
func (*AmazonS3StorageBackend) Description ¶
func (*AmazonS3StorageBackend) Description() string
Description returns a human-readable description for the Storage backend.
func (*AmazonS3StorageBackend) Location ¶
func (backend *AmazonS3StorageBackend) Location() string
Location returns the backend's URL as a string.
func (*AmazonS3StorageBackend) NewBackend ¶
NewBackend initializes an Amazon S3 Storage Backend.
func (*AmazonS3StorageBackend) Protocols ¶
func (*AmazonS3StorageBackend) Protocols() []string
Protocols returns a list of supported Protocol Handlers.
func (*AmazonS3StorageBackend) ReadFile ¶
func (backend *AmazonS3StorageBackend) ReadFile(path string) ([]byte, error)
ReadFile reads a file from the backend.
Source Files