s3

package

v1.3.1 Latest Latest Go to latest Published: Jun 6, 2017 License: MIT Imports: 16 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/cgilling/veneur

Links

Open Source Insights

README ¶

S3 Plugin

The S3 plugin archives every flush to S3 as a separate S3 object.

This plugin is still in an experimental state.

Documentation ¶

Index ¶

Constants
Variables
func EncodeDDMetricCSV(d samplers.DDMetric, w *csv.Writer, partitionDate *time.Time, hostName string) error
func EncodeDDMetricsCSV(metrics []samplers.DDMetric, delimiter rune, includeHeaders bool, ...) (io.ReadSeeker, error)
func S3Path(hostname string, ft filetype) *string
type S3Plugin

Constants ¶

View Source

const (
	// the order in which these appear determines the
	// order of the fields in the resultant TSV
	TsvName tsvField = iota
	TsvTags
	TsvMetricType

	// The hostName attached to the metric
	TsvHostname

	// The hostName of the server flushing the data
	TsvVeneurHostname

	TsvDeviceName
	TsvInterval

	TsvTimestamp
	TsvValue

	// This is the _partition field
	// required by the Redshift IncrementalLoader.
	// For our purposes, the current date is a good partition.
	TsvPartition
)

View Source

const PartitionDateFormat = "20060102"

View Source

const RedshiftDateFormat = "2006-01-02 03:04:05"

Variables ¶

View Source

var S3Bucket = "stripe-veneur"

View Source

var S3ClientUninitializedError = errors.New("s3 client has not been initialized")

Functions ¶

func EncodeDDMetricCSV ¶

func EncodeDDMetricCSV(d samplers.DDMetric, w *csv.Writer, partitionDate *time.Time, hostName string) error

EncodeDDMetricCSV generates a newline-terminated CSV row that describes the data represented by the DDMetric. The caller is responsible for setting w.Comma as the appropriate delimiter. For performance, encodeCSV does not flush after every call; the caller is expected to flush at the end of the operation cycle

func EncodeDDMetricsCSV ¶

func EncodeDDMetricsCSV(metrics []samplers.DDMetric, delimiter rune, includeHeaders bool, hostname string) (io.ReadSeeker, error)

EncodeDDMetricsCSV returns a reader containing the gzipped CSV representation of the DDMetrics data, one row per DDMetric. the AWS sdk requires seekable input, so we return a ReadSeeker here

func S3Path ¶

func S3Path(hostname string, ft filetype) *string

Types ¶

type S3Plugin ¶

type S3Plugin struct {
	Logger   *logrus.Logger
	Svc      s3iface.S3API
	S3Bucket string
	Hostname string
}

func (*S3Plugin) Flush ¶

func (p *S3Plugin) Flush(metrics []samplers.DDMetric, hostname string) error

func (*S3Plugin) Name ¶

func (p *S3Plugin) Name() string

func (*S3Plugin) S3Post ¶

func (p *S3Plugin) S3Post(hostname string, data io.ReadSeeker, ft filetype) error

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
mock

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL