awss3exporter

package module

v0.99.0 Latest Latest Go to latest Published: Apr 22, 2024 License: Apache-2.0 Imports: 26 Imported by: 3

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/open-telemetry/opentelemetry-collector-contrib

Links

Open Source Insights

README ¶

AWS S3 Exporter for OpenTelemetry Collector

Status
Stability	alpha: traces, metrics, logs
Distributions	contrib
Issues
Code Owners	@atoulme, @pdelewski

Schema supported

This exporter targets to support proto/json format.

Exporter Configuration

The following exporter configuration parameters are supported.

Name	Description	Default
`region`	AWS region.	"us-east-1"
`s3_bucket`	S3 bucket
`s3_prefix`	prefix for the S3 key (root directory inside bucket).
`s3_partition`	time granularity of S3 key: hour or minute	"minute"
`role_arn`	the Role ARN to be assumed
`file_prefix`	file prefix defined by user
`marshaler`	marshaler used to produce output data	`otlp_json`
`encoding`	Encoding extension to use to marshal data. Overrides the `marshaler` configuration option if set.
`encoding_file_extension`	file format extension suffix when using the `encoding` configuration option. May be left empty for no suffix to be appended.
`endpoint`	overrides the endpoint used by the exporter instead of constructing it from `region` and `s3_bucket`
`s3_force_path_style`	set this to `true` to force the request to use path-style addressing	false
`disable_ssl`	set this to `true` to disable SSL when sending requests	false
`compression`	should the file be compressed	none

Marshaler

Marshaler determines the format of data sent to AWS S3. Currently, the following marshalers are implemented:

otlp_json (default): the OpenTelemetry Protocol format, represented as json.
otlp_proto: the OpenTelemetry Protocol format, represented as Protocol Buffers. A single protobuf message is written into each object.
sumo_ic: the Sumo Logic Installed Collector Archive format. This format is supported only for logs.
body: export the log body as string. This format is supported only for logs.

Encoding

Encoding overrides marshaler if present and sets to use an encoding extension defined in the collector configuration.

See https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/encoding.

Compression

none (default): No compression will be applied
gzip: Files will be compressed with gzip. This does not support sumo_icmarshaler.

Example Configuration

Following example configuration defines to store output in 'eu-central' region and bucket named 'databucket'.

exporters:
  awss3:
    s3uploader:
        region: 'eu-central-1'
        s3_bucket: 'databucket'
        s3_prefix: 'metric'
        s3_partition: 'minute'

Logs and traces will be stored inside 'databucket' in the following path format.

metric/year=XXXX/month=XX/day=XX/hour=XX/minute=XX

AWS Credential Configuration

This exporter follows default credential resolution for the aws-sdk-go.

Follow the guidelines for the credential configuration.

OpenTelemetry Collector Helm Chart for Kubernetes

For example, when using OpenTelemetry Collector Helm Chart you could use extraEnvs in the values.yaml.

extraEnvs:
- name: AWS_ACCESS_KEY_ID
  value: "< YOUR AWS ACCESS KEY >"
- name: AWS_SECRET_ACCESS_KEY
  value: "< YOUR AWS SECRET ACCESS KEY >"

Documentation ¶

Overview ¶

Package awss3exporter stores OpenTelemetry data as an AWS S3 exporter.

Constants ¶

View Source

const (
	SourceCategoryKey = "_sourceCategory"
	SourceHostKey     = "_sourceHost"
	SourceNameKey     = "_sourceName"
)

Variables ¶

View Source

var (
	ErrUnknownMarshaler = errors.New("unknown marshaler")
)

Functions ¶

func NewFactory ¶

func NewFactory() exporter.Factory

NewFactory creates a factory for S3 exporter.

Types ¶

type Config ¶

type Config struct {
	S3Uploader    S3UploaderConfig `mapstructure:"s3uploader"`
	MarshalerName MarshalerType    `mapstructure:"marshaler"`

	// Encoding to apply. If present, overrides the marshaler configuration option.
	Encoding              *component.ID `mapstructure:"encoding"`
	EncodingFileExtension string        `mapstructure:"encoding_file_extension"`
}

Config contains the main configuration options for the s3 exporter

func (*Config) Validate ¶ added in v0.83.0

func (c *Config) Validate() error

type MarshalerType ¶ added in v0.76.1

type MarshalerType string

const (
	OtlpProtobuf MarshalerType = "otlp_proto"
	OtlpJSON     MarshalerType = "otlp_json"
	SumoIC       MarshalerType = "sumo_ic"
	Body         MarshalerType = "body"
)

type S3UploaderConfig ¶

type S3UploaderConfig struct {
	Region           string                 `mapstructure:"region"`
	S3Bucket         string                 `mapstructure:"s3_bucket"`
	S3Prefix         string                 `mapstructure:"s3_prefix"`
	S3Partition      string                 `mapstructure:"s3_partition"`
	FilePrefix       string                 `mapstructure:"file_prefix"`
	Endpoint         string                 `mapstructure:"endpoint"`
	RoleArn          string                 `mapstructure:"role_arn"`
	S3ForcePathStyle bool                   `mapstructure:"s3_force_path_style"`
	DisableSSL       bool                   `mapstructure:"disable_ssl"`
	Compression      configcompression.Type `mapstructure:"compression"`
}

S3UploaderConfig contains aws s3 uploader related config to controls things like bucket, prefix, batching, connections, retries, etc.

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
internal
metadata

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL