benthos: github.com/Jeffail/benthos

Directories

PathSynopsis
cmd/benthos
libPackage lib contains a collection of packages used to create a Benthos stream pipeline.
lib/apiPackage api implements a type used for creating the Benthos HTTP API.
lib/brokerPackage broker implements types used for routing inputs to outputs in non-trivial arrangements, such as fan-out or fan-in models.
lib/bufferPackage buffer is both a types.Consumer and types.Producer implementation that is able to sit between other stream components, effectively decoupling their transaction channels by storing messages in a buffer implementation.
lib/buffer/parallelPackage parallel contains implementations of various buffer types where the buffer can be consumed by any number of parallel consumer threads.
lib/buffer/singlePackage single contains implementations of various buffer types where the buffer can only be consumed by a single thread (but any number of writers).
lib/cachePackage cache implements the types.Cache interface for storing and retrieving key/value pairs from a range of storage strategies.
lib/conditionPackage condition contains logical operators that, based on their configuration, return boolean values from messages under certain circumstances.
lib/configPackage config defines the full configuration structure used by the Benthos service, including methods of normalising and linting user configuration.
lib/inputPackage input defines consumers for aggregating data from a variety of sources.
lib/input/readerPackage reader defines implementations of an interface for generic message reading from various third party sources.
lib/logPackage log contains utilities for logging in a modular interface.
lib/managerPackage manager implements the types.Manager interface used for creating and sharing resources across a Benthos service.
lib/messagePackage message contains implementations of types.Message.
lib/message/batchPackage batch provides tooling for creating and executing Benthos message batch policies.
lib/message/ioPackage io provides tooling for serialising and deserialising messages.
lib/message/mapperPackage mapper implements ways of splicing and mapping batches of messages so that their derivative parts can be processed individually and merged back into the original batch.
lib/message/metadataPackage metadata contains implementations of types.Metadata.
lib/message/roundtripPackage roundtrip provides tooling for composing request/response behaviour through messages.
lib/message/tracingPackage tracing implements utility functions for recording opentracing events for messages.
lib/metricsPackage metrics contains a type for aggregating and propagating metrics to various services based on configuration.
lib/outputPackage output defines all sinks for sending Benthos messages to a variety of third party destinations.
lib/output/writerPackage writer defines implementations of an interface for generic message writing that outputs to various third party sinks.
lib/pipelinePackage pipeline contains structures that implement both the Producer and Consumer interfaces.
lib/processorPackage processor contains implementations of types.Processor, which perform an arbitrary operation on a message and either returns >0 messages to be propagated towards a sink, or a response to be sent back to the message source.
lib/ratelimitPackage ratelimit implements the types.RateLimit interface for limiting access to resources shared service wide.
lib/responsePackage response contains implementations of types.Response.
lib/serverlessPackage serverless contains shared components for serverless distributions of Benthos.
lib/serverless/lambdaPackage lambda contains the execution logic for running Benthos as an AWS lambda function.
lib/servicePackage service contains the main execution logic of the Benthos service, which is responsible for parsing commandline flags, running a stream and capturing termination signals.
lib/service/testPackage test implements the Benthos service unit testing command.
lib/streamPackage stream creates and manages a full Benthos stream pipeline, consisting of an input layer of consumers, an optional buffer layer, a processing pipelines layer, and an output layer of producers:
lib/stream/managerPackage manager creates and manages multiple streams, providing an API for performing CRUD operations.
lib/tracerPackage tracer contains components able to send opentracing events.
lib/typesPackage types defines any general structs and interfaces used throughout the benthos code base.
lib/util
lib/util/aws/lambda/client
lib/util/aws/session
lib/util/configPackage config contains utilities for reading and parsing service configuration files.
lib/util/diskPackage disk contains cross platform disk statistics utilities.
lib/util/http
lib/util/http/authPackage auth provides configuration fields and implementations of HTTP request authentication strategies.
lib/util/http/client
lib/util/retriesPackage retries implements backoff strategies around a standard configuration scheme.
lib/util/textPackage text includes utilities for working with text that might contain variables.
lib/util/throttlePackage throttle implements throttle strategies.
lib/util/tlsPackage tls provides Benthos configuration fields and wrappers for a crypto/tls config.

Updated 2019-12-15. Refresh now. Tools for package owners.