s3util

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 18, 2020 License: MIT Imports: 14 Imported by: 0

README

Package s3util provides streaming transfers to and from Amazon S3.

Full documentation:
http://godoc.org/github.com/kr/s3/s3util

Documentation

Overview

Package s3util provides streaming transfers to and from Amazon S3.

To use it, open or create an S3 object, read or write data, and close the object.

You must assign valid credentials to DefaultConfig.Keys before using DefaultConfig. Be sure to close an io.WriteCloser returned by this package, to flush buffers and complete the multipart upload process.

Index

Examples

Constants

This section is empty.

Variables

View Source
var DefaultConfig = &Config{
	Service:         s3.DefaultService,
	Keys:            new(s3.Keys),
	MetricsCallback: nil,
}

Functions

func GetRespCode

func GetRespCode(e error) int

func Metadata

func Metadata(url string, c *Config) (*http.Response, error)

Metadata requests the S3 object's metadata at url. An HTTP status other than 200 is considered an error.

If c is nil, Metadata uses DefaultConfig.

func Open

func Open(url string, c *Config) (io.ReadCloser, *http.Response, error)

Open requests the S3 object at url. An HTTP status other than 200 is considered an error.

If c is nil, Open uses DefaultConfig.

Example
package main

import (
	"github.com/500px/s3/s3util"
	"io"
	"os"
)

func main() {
	s3util.DefaultConfig.AccessKey = "...access key..."
	s3util.DefaultConfig.SecretKey = "...secret key..."
	r, _, _ := s3util.Open("https://mybucket.s3.amazonaws.com/log.txt", nil)
	w, _ := os.Create("out.txt")
	io.Copy(w, r)
	w.Close()
}
Output:

Types

type Config

type Config struct {
	*s3.Service
	*s3.Keys
	*http.Client    // if nil, uses http.DefaultClient
	MetricsCallback MetricsCallbackFunc
}

type File

type File struct {
	// contains filtered or unexported fields
}

File represents an S3 object or directory.

func NewFile

func NewFile(rawurl string, c *Config) (*File, error)

NewFile returns a new File with the given URL and config.

Set rawurl to a directory on S3, such as https://mybucket.s3.amazonaws.com/myfolder. The URL cannot have query parameters or a fragment. If c is nil, DefaultConfig will be used.

func (*File) Readdir

func (f *File) Readdir(n int) ([]os.FileInfo, error)

Readdir requests a list of entries in the S3 directory represented by f and returns a slice of up to n FileInfo values, in alphabetical order. Subsequent calls on the same File will yield further FileInfos. Only direct children are returned, not deeper descendants.

type Metrics

type Metrics struct {
	TotalBytes uint64
	TotalTime  time.Duration
}

type MetricsCallbackFunc

type MetricsCallbackFunc func(Metrics)

type Stat

type Stat struct {
	Key          string
	LastModified string
	ETag         string // ETag value, without double quotes.
	Size         string
	StorageClass string
	OwnerID      string `xml:"Owner>ID"`
	OwnerName    string `xml:"Owner>DisplayName"`
}

Stat contains information about an S3 object or directory. It is the "underlying data source" returned by method Sys for each FileInfo produced by this package.

fi.Sys().(*s3util.Stat)

For the meaning of these fields, see http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGET.html.

type Uploader

type Uploader struct {
	UploadId string // written by xml decoder

	Err error
	// contains filtered or unexported fields
}

func (*Uploader) Close

func (u *Uploader) Close() error

func (*Uploader) CloseWithResponse

func (u *Uploader) CloseWithResponse() (*http.Response, error)

It's the caller's responsibility to close the response, if any.

func (*Uploader) Write

func (u *Uploader) Write(p []byte) (n int, err error)

type WriteCloserWithResponse

type WriteCloserWithResponse interface {
	io.WriteCloser
	CloseWithResponse() (*http.Response, error)
}

func Create

func Create(url string, h http.Header, c *Config) (WriteCloserWithResponse, error)

Create creates an S3 object at url and sends multipart upload requests as data is written.

If h is not nil, each of its entries is added to the HTTP request header. If c is nil, Create uses DefaultConfig.

Example
package main

import (
	"github.com/500px/s3/s3util"
	"io"
	"os"
)

func main() {
	s3util.DefaultConfig.AccessKey = "...access key..."
	s3util.DefaultConfig.SecretKey = "...secret key..."
	r, _ := os.Open("/dev/stdin")
	w, _ := s3util.Create("https://mybucket.s3.amazonaws.com/log.txt", nil, nil)
	io.Copy(w, r)
	w.Close()
}
Output:

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL