s3

command
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 16, 2017 License: MIT Imports: 8 Imported by: 0

README

S3 Pipeline

The S3 Connector Pipeline performs the following steps:

  1. Pull records from Kinesis and buffer them untill the desired threshold is met.
  2. Upload the batch of records to an S3 bucket.
  3. Set the current Shard checkpoint in Redis.

The pipleline config vars are loaded done with gcfg.

Environment Variables

Export the required environment vars for connecting to the Kinesis stream:

export AWS_ACCESS_KEY=
export AWS_REGION_NAME=
export AWS_SECRET_KEY=
Running the code
$ go run main.go -a appName -s streamName

Documentation

The Go Gopher

There is no documentation for this package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL