README
¶
S3 Storage Backend
The S3 backend provides support for Amazon S3 and compatible services.
Configuration
To use the S3 backend, you need to specify the following options in your connection string:
endpoint: The endpoint URL of your S3 service.region: The AWS region.accessKey: Your AWS access key.secretKey: Your AWS secret key.sse: Server Side Encryption setting.
A connection string for the S3 backend looks like this:
s3://myBucketName/my/prefix?region=region&accessKey=accessKey&secretKey=secretKey&sse=sse
Replace myBucketName/my/prefix, region, accessKey, secretKey, and sse with your actual parameters.
Documentation
¶
Index ¶
- func New(opts Options) (types.Storage, error)
- func NewCredentials(accessKey, secretKey string) *credentials.Credentials
- type Backend
- func (b *Backend) AbortMultipartUploadWithContext(ctx context.Context, path, uploadID string) error
- func (b *Backend) CompleteMultipartUploadWithContext(ctx context.Context, path, uploadID string, ...) error
- func (b *Backend) DeleteWithContext(ctx context.Context, path string) error
- func (b *Backend) InitiateMultipartUploadWithContext(ctx context.Context, path string) (string, error)
- func (b *Backend) ListWithContext(ctx context.Context, prefix string) (*[]types.Object, error)
- func (b *Backend) MoveToBucketWithContext(ctx context.Context, srcPath, dstPath, dstBucket string) error
- func (b *Backend) MoveWithContext(ctx context.Context, fromPath string, toPath string) error
- func (b *Backend) ReadWithContext(ctx context.Context, path string, start int64, end int64) (io.ReadCloser, error)
- func (b *Backend) StatWithContext(ctx context.Context, path string) (*types.Object, error)
- func (b *Backend) WriteMultipartWithContext(ctx context.Context, path, uploadID string, partNumber int64, ...) (int64, *types.CompletedPart, error)
- func (b *Backend) WriteWithContext(ctx context.Context, path string, reader io.Reader, size int64) (int64, error)
- type Options
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewCredentials ¶
func NewCredentials(accessKey, secretKey string) *credentials.Credentials
Types ¶
type Backend ¶
type Backend struct {
Bucket string
Client *s3.S3
Downloader *s3manager.Downloader
Prefix string
Uploader *s3manager.Uploader
SSE string
}
Backend is a storage backend for S3
func (*Backend) AbortMultipartUploadWithContext ¶
AbortMultipartUpload aborts the multipart upload
func (*Backend) CompleteMultipartUploadWithContext ¶
func (b *Backend) CompleteMultipartUploadWithContext(ctx context.Context, path, uploadID string, completedParts []*types.CompletedPart) error
CompleteMultipartUpload completes a multipart upload
func (*Backend) DeleteWithContext ¶
DeleteObject removes an object from a S3 bucket, at prefix
func (*Backend) InitiateMultipartUploadWithContext ¶
func (b *Backend) InitiateMultipartUploadWithContext(ctx context.Context, path string) (string, error)
InitiateMultipartUpload initiates a multipart upload.
Use WriteWithContext over this method if the full file is already on the local machine as it will do the multipart upload for you.
func (*Backend) ListWithContext ¶
ListObjects lists all objects in a S3 bucket, at prefix Note: This function does not handle pagination and will return a maximum of 1000 objects. If there are more than 1000 objects with the specified prefix, consider using pagination to retrieve all objects.
func (*Backend) MoveToBucketWithContext ¶
func (b *Backend) MoveToBucketWithContext(ctx context.Context, srcPath, dstPath, dstBucket string) error
MoveToBucket moves an object from one S3 bucket to another
func (*Backend) MoveWithContext ¶
MoveObject moves an object from one path to another within a S3 bucket
func (*Backend) ReadWithContext ¶
func (b *Backend) ReadWithContext(ctx context.Context, path string, start int64, end int64) (io.ReadCloser, error)
ReadWithContext reads an object from S3 bucket, at given path
func (*Backend) StatWithContext ¶
Stat returns information about an object in a S3 bucket, at given path
func (*Backend) WriteMultipartWithContext ¶
func (b *Backend) WriteMultipartWithContext(ctx context.Context, path, uploadID string, partNumber int64, reader io.ReadSeeker, size int64) (int64, *types.CompletedPart, error)
WriteMultipartWithContext writes a part of a multipart upload
func (*Backend) WriteWithContext ¶
func (b *Backend) WriteWithContext(ctx context.Context, path string, reader io.Reader, size int64) (int64, error)
WriteObject uploads an object to S3, intelligently buffering large files into smaller chunks and sending them in parallel across multiple goroutines.
Always use this method if your full file is already on disk or in memory.
Source Files