batch

package
v0.99.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 22, 2024 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Index

Constants

View Source
const (
	MaxRecordSize     = 1 << 20 // 1MiB
	MaxBatchedRecords = 500
)

Variables

View Source
var (
	// ErrPartitionKeyLength is used when the given key exceeds the allowed kinesis limit of 256 characters
	ErrPartitionKeyLength = errors.New("partition key size is greater than 256 characters")
	// ErrRecordLength is used when attempted record results in a byte array greater than 1MiB
	ErrRecordLength = consumererror.NewPermanent(errors.New("record size is greater than 1 MiB"))
)
View Source
var (
	// ErrUnsupportedEncoding is used when the encoder type does not support the type of encoding
	ErrUnsupportedEncoding = errors.New("unsupported type to encode")
	// ErrUnknownExportEncoder is used when a named encoding doesn't not exist
	ErrUnknownExportEncoder = errors.New("unknown encoding export format")
)

Functions

This section is empty.

Types

type Batch

type Batch struct {
	// contains filtered or unexported fields
}

func New

func New(opts ...Option) *Batch

func (*Batch) AddRecord added in v0.37.0

func (b *Batch) AddRecord(raw []byte, key string) error

func (*Batch) Chunk

func (b *Batch) Chunk() (chunks [][]types.PutRecordsRequestEntry)

Chunk breaks up the iternal queue into blocks that can be used to be written to he kinesis.PutRecords endpoint

type Encoder

type Encoder interface {
	Metrics(md pmetric.Metrics) (*Batch, error)

	Traces(td ptrace.Traces) (*Batch, error)

	Logs(ld plog.Logs) (*Batch, error)
}

Encoder transforms the internal pipeline format into a configurable format that is then used to export to kinesis.

func NewEncoder added in v0.37.0

func NewEncoder(named string, batchOptions ...Option) (Encoder, error)

type Option

type Option func(bt *Batch)

func WithCompression added in v0.37.0

func WithCompression(compressor compress.Compressor) Option

func WithMaxRecordSize

func WithMaxRecordSize(size int) Option

func WithMaxRecordsPerBatch

func WithMaxRecordsPerBatch(limit int) Option

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL