README
¶
Double-Team
Synopsis
A HTTP Kafka producer that handles outages.
Usage
Double-Team can be used in two different modes: server and restore
Server
Server mode accepts HTTP post requests and publishes them to Kafka.
Restore
Restore mode sends messages from S3 to Kafka.
Configuration
Server
The Double-Team server ./double-team server can be configured with the following options:
| Flag | Description | Environment Variable |
|---|---|---|
| --port | The address to bind to for the http server. | PORT |
| --log-level | The log level to use (options: debug, info, warn, error). | LOG_LEVEL |
| --log-format | Log format to use (eg.: json, terminal). | LOG_FORMAT |
| --log-tags | Additional tags for logs. | LOG_TAGS |
| --stats-dsn | The statistics service to send metrics to (e.g.: l2met://, prometheus://0.0.0.0:8082). | STATS_DSN |
| --stats-prefix | Prefix for statistics. | STATS_PREFIX |
| --stats-tags | Additional tags for stats. | STATS_TAGS |
| --kafka.brokers | The kafka seed brokers connect to. Format: 'ip:port' (multiple allowed). | DOUBLE_TEAM_KAFKA_BROKERS |
| --kafka.version | Version of Kafka for producing messages: '2.3.0'. | DOUBLE_TEAM_KAFKA_VERSION |
| --kafka.retry | The number of times to retry sending to Kafka. | DOUBLE_TEAM_KAFKA_RETRY |
| --s3.endpoint | The S3 endpoint to use. This is mainly for debugging. | DOUBLE_TEAM_S3_ENDPOINT |
| --s3.region | The S3 region the bucket exists in. | DOUBLE_TEAM_S3_REGION |
| --s3.bucket | The S3 bucket to write messages to. | DOUBLE_TEAM_S3_BUCKET |
Restore
The Double-Team server ./double-team restore can be configured with the following options:
| Flag | Description | Environment Variable |
|---|---|---|
| --log-level | The log level to use (options: debug, info, warn, error). | LOG_LEVEL |
| --log-format | Log format to use (eg.: json, terminal). | LOG_FORMAT |
| --log-tags | Additional tags for logs. | LOG_TAGS |
| --stats-dsn | The statistics service to send metrics to (e.g.: l2met://, prometheus://0.0.0.0:8082). | STATS_DSN |
| --stats-prefix | Prefix for statistics. | STATS_PREFIX |
| --stats-tags | Additional tags for stats. | STATS_TAGS |
| --kafka.brokers | The kafka seed brokers connect to. Format: 'ip:port' (multiple allowed). | DOUBLE_TEAM_KAFKA_BROKERS |
| --kafka.version | Version of Kafka for producing messages: '2.3.0'. | DOUBLE_TEAM_KAFKA_VERSION |
| --kafka.retry | The number of times to retry sending to Kafka. | DOUBLE_TEAM_KAFKA_RETRY |
| --s3.endpoint | The S3 endpoint to use. This is mainly for debugging. | DOUBLE_TEAM_S3_ENDPOINT |
| --s3.region | The S3 region the bucket exists in. | DOUBLE_TEAM_S3_REGION |
| --s3.bucket | The S3 bucket to read messages from. | DOUBLE_TEAM_S3_BUCKET |
Server HTTP Endpoints
POST /
Accepts a JSON payload with the message topic and data.
{
"topic": "test",
"key": "key",
"data": "test data"
}
GET /health
Gets the current health status of the server. Returns a 200 status code if the server is healthy, otherwise a 503 status code
License
MIT-License. As is. No warranties whatsoever. Mileage may vary. Batteries not included.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Application ¶
type Application struct {
// contains filtered or unexported fields
}
Application represents the application.
func NewApplication ¶
func NewApplication(ctx context.Context, producers []streaming.Producer, queueSize int) *Application
NewApplication creates an instance of Application.
func (*Application) Close ¶
func (a *Application) Close() error
Close closes the application and cleans up.
func (*Application) IsHealthy ¶
func (a *Application) IsHealthy() error
IsHealthy checks the health of the Application.
func (*Application) Send ¶
func (a *Application) Send(topic string, key, data []byte)
Send sends a message to the producer chain.
Source Files
Directories