s3

package

v14.2.0 Latest Latest Go to latest Published: May 9, 2022 License: MIT Imports: 18 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/stripe/veneur

Links

Open Source Insights

README ¶

S3 Sink

The S3 sink archives every flush to S3 as a separate S3 object.

This sink is still in an experimental state.

Config Options to connect to S3

Mandatory parameters below.

aws_s3_bucket: util.StringSecret
aws_region: string

Optional parameters below.

aws_access_key_id string
aws_secret_access_key util.StringSecret

The Go AWS SDK will load up Credentials in the following order. https://docs.aws.amazon.com/sdk-for-go/api/aws/session/

Environment Variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, AWS_PROFILE, AWS_REGION
Shared Credentials file ~/.aws/credentials
Shared Configuration file (if SharedConfig is enabled) export AWS_SDK_LOAD_CONFIG=1
EC2 Instance Metadata (credentials only).

Documentation ¶

Index ¶

Variables
func Create(server *veneur.Server, name string, logger *logrus.Entry, config veneur.Config, ...) (sinks.MetricSink, error)
func MigrateConfig(config *veneur.Config)
func ParseConfig(name string, config interface{}) (veneur.MetricSinkConfig, error)
func S3Path(hostname string, ft filetype) *string
type S3Sink
type S3SinkConfig

Constants ¶

This section is empty.

Variables ¶

View Source

var S3ClientUninitializedError = errors.New("s3 client has not been initialized")

Functions ¶

func Create ¶

func Create(
	server *veneur.Server, name string, logger *logrus.Entry,
	config veneur.Config, sinkConfig veneur.MetricSinkConfig,
) (sinks.MetricSink, error)

Create creates a new S3 sink for metrics. This function should match the signature of a value in veneur.MetricSinkTypes, and is intended to be passed into veneur.NewFromConfig to be called based on the provided configuration.

func MigrateConfig ¶

func MigrateConfig(config *veneur.Config)

TODO(arnavdugar): Remove this once the old configuration format has been removed.

func ParseConfig ¶

func ParseConfig(
	name string, config interface{},
) (veneur.MetricSinkConfig, error)

ParseConfig decodes the map config for an S3 sink into an S3SinkConfig struct.

func S3Path ¶

func S3Path(hostname string, ft filetype) *string

Types ¶

type S3Sink ¶

type S3Sink struct {
	Hostname string
	Interval int
	Logger   *logrus.Entry

	S3Bucket string
	Svc      s3iface.S3API
	// contains filtered or unexported fields
}

func (*S3Sink) Flush ¶

func (p *S3Sink) Flush(
	ctx context.Context, metrics []samplers.InterMetric,
) error

func (*S3Sink) FlushOtherSamples ¶

func (p *S3Sink) FlushOtherSamples(ctx context.Context, samples []ssf.SSFSample)

func (*S3Sink) Name ¶

func (p *S3Sink) Name() string

func (*S3Sink) S3Post ¶

func (p *S3Sink) S3Post(hostname string, data io.ReadSeeker, ft filetype) error

func (*S3Sink) Start ¶

func (p *S3Sink) Start(traceClient *trace.Client) error

type S3SinkConfig ¶

type S3SinkConfig struct {
	AccessKeyID     util.StringSecret `yaml:"access_key_id"`
	S3Bucket        string            `yaml:"s3_bucket"`
	Region          string            `yaml:"region"`
	SecretAccessKey util.StringSecret `yaml:"secret_access_key"`
}

Source Files ¶

View all Source files

s3.go

Directories ¶

Path	Synopsis
mock

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL