s3

package

v1.6.1 Latest Latest Go to latest Published: Aug 1, 2023 License: MIT Imports: 15 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/kelindar/talaria

Links

Open Source Insights

README ¶

Amazon S3 sink, Minio, Digital Ocean Spaces

This sink implements Amazon S3 protocol and can be enabled for Digital Ocean Spaces and Minio using a custom endpoint.

This sink can be enabled by adding the following configuration in the tables section:

tables:
  eventlog:
    compact:                               # enable compaction
      interval: 60                         # compact every 60 seconds
      nameFunc: "s3://bucket/namefunc.lua" # file name function
      s3:                                  # sink to Amazon S3
        region: "ap-southeast-1"           # the region to use
        bucket: "bucket"                   # the bucket to use
        prefix: "dir1/"                    # (optional) prefix to add
        endpoint: "http://127.0.0.1"       # (optional) custom endpoint to use
        sse: ""                            # (optional) server-side encryption
        accessKey: ""                      # (optional) static access key to override
        secretKey: ""                      # (optional) static secret key to override
        concurrency: 32                    # (optional) upload concurrency, default=NUM_CPU
...

Documentation ¶

Index ¶

type Uploader
type Writer
- func New(monitor monitor.Monitor, ...) (*Writer, error)
- func (w *Writer) Write(key key.Key, blocks []block.Block) error

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Uploader ¶

type Uploader interface {
	Upload(input *s3manager.UploadInput, options ...func(*s3manager.Uploader)) (*s3manager.UploadOutput, error)
}

Uploader uploads to underlying backend

type Writer ¶

type Writer struct {
	*base.Writer
	// contains filtered or unexported fields
}

Writer represents a writer for Amazon S3 and compatible storages.

func New ¶

func New(monitor monitor.Monitor, bucket, prefix, region, endpoint, sse, access, secret, filter, encoding string, concurrency int) (*Writer, error)

New initializes a new S3 writer.

func (*Writer) Write ¶

func (w *Writer) Write(key key.Key, blocks []block.Block) error

Write writes creates object of S3 bucket prefix key in S3Writer bucket with value val

Source Files ¶

View all Source files

s3.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL