benthos: github.com/Jeffail/benthos/lib

Directories

PathSynopsis
apiPackage api implements a type used for creating the Benthos HTTP API.
bloblang
brokerPackage broker implements types used for routing inputs to outputs in non-trivial arrangements, such as fan-out or fan-in models.
bufferPackage buffer is both a types.Consumer and types.Producer implementation that is able to sit between other stream components, effectively decoupling their transaction channels by storing messages in a buffer implementation.
buffer/parallelPackage parallel contains implementations of various buffer types where the buffer can be consumed by any number of parallel consumer threads.
buffer/singlePackage single contains implementations of various buffer types where the buffer can only be consumed by a single thread (but any number of writers).
cachePackage cache implements the types.Cache interface for storing and retrieving key/value pairs from a range of storage strategies.
conditionPackage condition contains logical operators that, based on their configuration, return boolean values from messages under certain circumstances.
configPackage config defines the full configuration structure used by the Benthos service, including methods of normalising and linting user configuration.
inputPackage input defines consumers for aggregating data from a variety of sources.
input/readerPackage reader defines implementations of an interface for generic message reading from various third party sources.
logPackage log contains utilities for logging in a modular interface.
managerPackage manager implements the types.Manager interface used for creating and sharing resources across a Benthos service.
messagePackage message contains implementations of types.Message.
message/batchPackage batch provides tooling for creating and executing Benthos message batch policies.
message/ioPackage io provides tooling for serialising and deserialising messages.
message/mapperPackage mapper implements ways of splicing and mapping batches of messages so that their derivative parts can be processed individually and merged back into the original batch.
message/metadataPackage metadata contains implementations of types.Metadata.
message/roundtripPackage roundtrip provides tooling for composing request/response behaviour through messages.
message/tracingPackage tracing implements utility functions for recording opentracing events for messages.
metricsPackage metrics contains a type for aggregating and propagating metrics to various services based on configuration.
outputPackage output defines all sinks for sending Benthos messages to a variety of third party destinations.
output/writerPackage writer defines implementations of an interface for generic message writing that outputs to various third party sinks.
pipelinePackage pipeline contains structures that implement both the Producer and Consumer interfaces.
processorPackage processor contains implementations of types.Processor, which perform an arbitrary operation on a message and either returns >0 messages to be propagated towards a sink, or a response to be sent back to the message source.
ratelimitPackage ratelimit implements the types.RateLimit interface for limiting access to resources shared service wide.
responsePackage response contains implementations of types.Response.
serverlessPackage serverless contains shared components for serverless distributions of Benthos.
serverless/lambdaPackage lambda contains the execution logic for running Benthos as an AWS lambda function.
servicePackage service contains the main execution logic of the Benthos service, which is responsible for parsing commandline flags, running a stream and capturing termination signals.
service/blobl
service/testPackage test implements the Benthos service unit testing command.
streamPackage stream creates and manages a full Benthos stream pipeline, consisting of an input layer of consumers, an optional buffer layer, a processing pipelines layer, and an output layer of producers:
stream/managerPackage manager creates and manages multiple streams, providing an API for performing CRUD operations.
tracerPackage tracer contains components able to send opentracing events.
typesPackage types defines any general structs and interfaces used throughout the benthos code base.
util
util/aws/lambda/client
util/aws/session
util/checkpointPackage checkpoint implements a mechanism for tracking checkpointed integer offsets for sequential read at-least-once queue systems such as Kafka or Kinesis.
util/configPackage config contains utilities for reading and parsing service configuration files.
util/diskPackage disk contains cross platform disk statistics utilities.
util/http
util/http/authPackage auth provides configuration fields and implementations of HTTP request authentication strategies.
util/http/client
util/retriesPackage retries implements backoff strategies around a standard configuration scheme.
util/textPackage text includes utilities for working with text that might contain variables.
util/throttlePackage throttle implements throttle strategies.
util/tlsPackage tls provides Benthos configuration fields and wrappers for a crypto/tls config.

Updated 2021-01-22. Refresh now. Tools for package owners.