s3resource

package module
v0.0.0-...-ae57f8d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 24, 2015 License: Apache-2.0 Imports: 9 Imported by: 0

README

S3 Resource

Versions objects in an S3 bucket, by pattern-matching filenames to identify version numbers.

Source Configuration

  • access_key_id: Required. The AWS access key to use when accessing the bucket.

  • secret_access_key: Required. The AWS secret key to use when accessing the bucket.

  • bucket: Required. The name of the bucket.

  • region_name: Optional. The region the bucket is in. Defaults to us-east-1.

  • private: Optional. Indicates that the bucket is private, so that any URLs provided are signed.

  • cloudfront_url: Optional. The URL (scheme and domain) of your CloudFront distribution that is fronting this bucket. This will be used in the url file that is given to the following task.

  • endpoint: Optional. Custom endpoint for using S3 compatible provider.

File Names

One of the following two options must be specified:

  • regexp: Optional. The pattern to match filenames against. The first grouped match is used to extract the version, or if a group is explicitly named version, that group is used.

    The version extracted from this pattern is used to version the resource. Semantic versions, or just numbers, are supported.

  • versioned_file: Optional If you enable versioning for your S3 bucket then you can keep the file name the same and upload new versions of your file without resorting to version numbers. This property is the path to the file in your S3 bucket.

Behavior

check: Extract versions from the bucket.

Objects will be found via the pattern configured by regexp. The versions will be used to order them (using semver). Each object's filename is the resulting version.

in: Fetch an object from the bucket.

Places the following files in the destination:

  • (filename): The file fetched from the bucket.

  • url: A file containing the URL of the object. If private is true, this URL will be signed.

  • version: The version identified in the file name.

Parameters

None.

out: Upload an object to the bucket.

Given a path specified by from, upload it to the S3 bucket, optionally to a directory configured by to. The path must identify a single file.

Parameters

  • from: Required. A regexp specifying the file to upload. If the regexp matches more than one file, the output fails.

  • to: Optional. A destination directory in the bucket.

Example Configuration

Resource
- name: release
  type: s3
  source:
    bucket: releases
    regexp: release-name-(.*).tgz
    access_key_id: AKIA-ACCESS-KEY
    secret_access_key: SECRET
Plan
- get: release

- put: release
  params:
    from: a/release/path/release-(.*).tgz

Required IAM Permissions

Non-versioned Buckets
  • s3:PutObject
  • s3:GetObject
  • s3:ListBucket
Versioned Buckets

Everything above and...

  • s3:GetBucketVersioning
  • s3:GetObjectVersion
  • s3:ListBucketVersions

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Fatal 

func Fatal(doing string, err error)

func Sayf 

func Sayf(message string, args ...interface{})

Types

type MetadataPair 

type MetadataPair struct {
	Name  string `json:"name"`
	Value string `json:"value"`
}

type S3Client 

type S3Client interface {
	BucketFiles(bucketName string, prefixHint string) ([]string, error)
	BucketFileVersions(bucketName string, remotePath string) ([]string, error)

	UploadFile(bucketName string, remotePath string, localPath string) (string, error)
	DownloadFile(bucketName string, remotePath string, versionID string, localPath string) error

	DeleteFile(bucketName string, remotePath string) error
	DeleteVersionedFile(bucketName string, remotePath string, versionID string) error

	URL(bucketName string, remotePath string, private bool, versionID string) string
}

func NewS3Client 

func NewS3Client(accessKey string, secretKey string, regionName string, endpoint string) (S3Client, error)

type Source 

type Source struct {
	AccessKeyID     string `json:"access_key_id"`
	SecretAccessKey string `json:"secret_access_key"`
	Bucket          string `json:"bucket"`
	Regexp          string `json:"regexp"`
	VersionedFile   string `json:"versioned_file"`
	Private         bool   `json:"private"`
	RegionName      string `json:"region_name"`
	CloudfrontURL   string `json:"cloudfront_url"`
	Endpoint        string `json:"endpoint"`
}

func (Source) IsValid 

func (source Source) IsValid() (bool, string)

type Version 

type Version struct {
	Path      string `json:"path,omitempty"`
	VersionID string `json:"version_id,omitempty"`
}

Source Files

View all Source files

Directories

Path Synopsis
Godeps
_workspace/src/github.com/aws/aws-sdk-go/aws
Package aws provides core functionality for making requests to AWS services.
 Package aws provides core functionality for making requests to AWS services.
_workspace/src/github.com/aws/aws-sdk-go/aws/awserr
Package awserr represents API error interface accessors for the SDK.
 Package awserr represents API error interface accessors for the SDK.
_workspace/src/github.com/aws/aws-sdk-go/aws/awsutil
_workspace/src/github.com/aws/aws-sdk-go/aws/corehandlers
_workspace/src/github.com/aws/aws-sdk-go/aws/credentials
Package credentials provides credential retrieval and management The Credentials is the primary method of getting access to and managing credentials Values.
 Package credentials provides credential retrieval and management The Credentials is the primary method of getting access to and managing credentials Values.
_workspace/src/github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds
_workspace/src/github.com/aws/aws-sdk-go/aws/credentials/stscreds
Package stscreds are credential Providers to retrieve STS AWS credentials.
 Package stscreds are credential Providers to retrieve STS AWS credentials.
_workspace/src/github.com/aws/aws-sdk-go/aws/defaults
_workspace/src/github.com/aws/aws-sdk-go/aws/ec2metadata
_workspace/src/github.com/aws/aws-sdk-go/aws/request
_workspace/src/github.com/aws/aws-sdk-go/aws/service
_workspace/src/github.com/aws/aws-sdk-go/aws/service/serviceinfo
_workspace/src/github.com/aws/aws-sdk-go/internal/endpoints
Package endpoints validates regional endpoints for services.
 Package endpoints validates regional endpoints for services.
_workspace/src/github.com/aws/aws-sdk-go/internal/protocol/query
Package query provides serialisation of AWS query requests, and responses.
 Package query provides serialisation of AWS query requests, and responses.
_workspace/src/github.com/aws/aws-sdk-go/internal/protocol/query/queryutil
_workspace/src/github.com/aws/aws-sdk-go/internal/protocol/rest
Package rest provides RESTful serialization of AWS requests and responses.
 Package rest provides RESTful serialization of AWS requests and responses.
_workspace/src/github.com/aws/aws-sdk-go/internal/protocol/restxml
Package restxml provides RESTful XML serialisation of AWS requests and responses.
 Package restxml provides RESTful XML serialisation of AWS requests and responses.
_workspace/src/github.com/aws/aws-sdk-go/internal/protocol/xml/xmlutil
Package xmlutil provides XML serialisation of AWS requests and responses.
 Package xmlutil provides XML serialisation of AWS requests and responses.
_workspace/src/github.com/aws/aws-sdk-go/internal/signer/v4
Package v4 implements signing for AWS V4 signer
 Package v4 implements signing for AWS V4 signer
_workspace/src/github.com/aws/aws-sdk-go/service/s3
Package s3 provides a client for Amazon Simple Storage Service.
 Package s3 provides a client for Amazon Simple Storage Service.
_workspace/src/github.com/aws/aws-sdk-go/service/s3/s3iface
Package s3iface provides an interface for the Amazon Simple Storage Service.
 Package s3iface provides an interface for the Amazon Simple Storage Service.
_workspace/src/github.com/aws/aws-sdk-go/service/s3/s3manager
_workspace/src/github.com/cloudfoundry/gunk/urljoiner
_workspace/src/github.com/hashicorp/go-version
_workspace/src/github.com/mitchellh/colorstring
colorstring provides functions for colorizing strings for terminal output.
 colorstring provides functions for colorizing strings for terminal output.
_workspace/src/github.com/onsi/ginkgo
Ginkgo is a BDD-style testing framework for Golang The godoc documentation describes Ginkgo's API.
 Ginkgo is a BDD-style testing framework for Golang The godoc documentation describes Ginkgo's API.
_workspace/src/github.com/onsi/ginkgo/config
Ginkgo accepts a number of configuration options.
 Ginkgo accepts a number of configuration options.
_workspace/src/github.com/onsi/ginkgo/ginkgo
The Ginkgo CLI The Ginkgo CLI is fully documented [here](http://onsi.github.io/ginkgo/#the_ginkgo_cli) You can also learn more by running: ginkgo help Here are some of the more commonly used commands: To install: go install github.com/onsi/ginkgo/ginkgo To run tests: ginkgo To run tests in all subdirectories: ginkgo -r To run tests in particular packages: ginkgo <flags> /path/to/package /path/to/another/package To pass arguments/flags to your tests: ginkgo <flags> <packages> -- <pass-throughs> To run tests in parallel ginkgo -p this will automatically detect the optimal number of nodes to use.
 The Ginkgo CLI The Ginkgo CLI is fully documented [here](http://onsi.github.io/ginkgo/#the_ginkgo_cli) You can also learn more by running: ginkgo help Here are some of the more commonly used commands: To install: go install github.com/onsi/ginkgo/ginkgo To run tests: ginkgo To run tests in all subdirectories: ginkgo -r To run tests in particular packages: ginkgo <flags> /path/to/package /path/to/another/package To pass arguments/flags to your tests: ginkgo <flags> <packages> -- <pass-throughs> To run tests in parallel ginkgo -p this will automatically detect the optimal number of nodes to use.
_workspace/src/github.com/onsi/ginkgo/ginkgo/convert
_workspace/src/github.com/onsi/ginkgo/ginkgo/interrupthandler
_workspace/src/github.com/onsi/ginkgo/ginkgo/nodot
_workspace/src/github.com/onsi/ginkgo/ginkgo/testrunner
_workspace/src/github.com/onsi/ginkgo/ginkgo/testsuite
_workspace/src/github.com/onsi/ginkgo/ginkgo/watch
_workspace/src/github.com/onsi/ginkgo/integration
_workspace/src/github.com/onsi/ginkgo/internal/codelocation
_workspace/src/github.com/onsi/ginkgo/internal/containernode
_workspace/src/github.com/onsi/ginkgo/internal/failer
_workspace/src/github.com/onsi/ginkgo/internal/leafnodes
_workspace/src/github.com/onsi/ginkgo/internal/remote
Aggregator is a reporter used by the Ginkgo CLI to aggregate and present parallel test output coherently as tests complete.
 Aggregator is a reporter used by the Ginkgo CLI to aggregate and present parallel test output coherently as tests complete.
_workspace/src/github.com/onsi/ginkgo/internal/spec
_workspace/src/github.com/onsi/ginkgo/internal/specrunner
_workspace/src/github.com/onsi/ginkgo/internal/suite
_workspace/src/github.com/onsi/ginkgo/internal/testingtproxy
_workspace/src/github.com/onsi/ginkgo/internal/writer
_workspace/src/github.com/onsi/ginkgo/reporters
Ginkgo's Default Reporter A number of command line flags are available to tweak Ginkgo's default output.
 Ginkgo's Default Reporter A number of command line flags are available to tweak Ginkgo's default output.
_workspace/src/github.com/onsi/ginkgo/reporters/stenographer
_workspace/src/github.com/onsi/ginkgo/types
_workspace/src/github.com/onsi/gomega
Gomega is the Ginkgo BDD-style testing framework's preferred matcher library.
 Gomega is the Ginkgo BDD-style testing framework's preferred matcher library.
_workspace/src/github.com/onsi/gomega/format
Gomega's format package pretty-prints objects.
 Gomega's format package pretty-prints objects.
_workspace/src/github.com/onsi/gomega/gbytes
Package gbytes provides a buffer that supports incrementally detecting input.
 Package gbytes provides a buffer that supports incrementally detecting input.
_workspace/src/github.com/onsi/gomega/gexec
Package gexec provides support for testing external processes.
 Package gexec provides support for testing external processes.
_workspace/src/github.com/onsi/gomega/ghttp
Package ghttp supports testing HTTP clients by providing a test server (simply a thin wrapper around httptest's server) that supports registering multiple handlers.
 Package ghttp supports testing HTTP clients by providing a test server (simply a thin wrapper around httptest's server) that supports registering multiple handlers.
_workspace/src/github.com/onsi/gomega/internal/assertion
_workspace/src/github.com/onsi/gomega/internal/asyncassertion
_workspace/src/github.com/onsi/gomega/internal/fakematcher
_workspace/src/github.com/onsi/gomega/internal/testingtsupport
_workspace/src/github.com/onsi/gomega/matchers
Gomega matchers This package implements the Gomega matchers and does not typically need to be imported.
 Gomega matchers This package implements the Gomega matchers and does not typically need to be imported.
_workspace/src/github.com/onsi/gomega/matchers/support/goraph/bipartitegraph
_workspace/src/github.com/onsi/gomega/matchers/support/goraph/edge
_workspace/src/github.com/onsi/gomega/matchers/support/goraph/node
_workspace/src/github.com/onsi/gomega/matchers/support/goraph/util
_workspace/src/github.com/onsi/gomega/types
_workspace/src/github.com/vaughan0/go-ini
Package ini provides functions for parsing INI configuration files.
 Package ini provides functions for parsing INI configuration files.
cmd
check
in
out
This file was generated by counterfeiter
 This file was generated by counterfeiter

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.