Documentation
¶
Index ¶
- Constants
- Variables
- func DownloadWithRetry(assets []string, saveDir, s3Endpoint string, retries int) error
- func List(assetsDir string) (assets []string, err error)
- func Load(assetsFile string) ([]string, error)
- func UploadWithRetry(assets []string, assetDir, bucket, s3Endpoint string, retries int) error
- func Write(s3Assets []string, output io.Writer) error
- type Downloader
- type Extractor
- type Uploader
Constants ¶
const DefaultS3Regexp = `(s3:\/\/[a-zA-Z0-9\/\-\_\.]+)`
DefaultS3Regexp matches s3://.* until a " or \n
Variables ¶
var DiscardCloser io.WriteCloser = discardCloser{}
DiscardCloser implements a noop Write and Close.
Functions ¶
func DownloadWithRetry ¶
DownloadWithRetry will try to download all s3 assets to a directory. Will accept errors `retries` times. The path in `saveDir` that assets are saved to is simply saveDir/bucket/assetKey. E.g. if there's an s3 asset in `assets` with the uri "s3://my-bucket/nested/key/file.zip" and `saveDir` is "download_dir" then the asset will be saved at the path: `download_dir/my-bucket/nested/key/file.zip` with any intermediate directories being created as needed.
func UploadWithRetry ¶
UploadWithRetry will try to upload all assets in a directory to a particular bucket. Will accept errors `retries` times. Asset paths in `assets` are expected to be relative to the `assetDir` directory. The key created in the bucket for each asset will be the path that same relative path in `assetDir`. E.g. if there's an asset path in `assets` "my-bucket/nested/key/file.zip", and an `assetDir` "download_dir", a file at the path "./download_dir/my-bucket/nested/key/file.zip" is expected to exist. And if the target `bucket` is "target-bucket" then the example file will be uploaded to the s3 uri "s3://target-bucket/my-bucket/nested/key/file.zip".
Types ¶
type Downloader ¶
type Downloader interface {
Download() error
}
Downloader implement Download.
func NewManyDownloader ¶
func NewManyDownloader(assets []string, saveDir, s3Endpoint string, retries int) (Downloader, error)
NewManyDownloader returns a Downloader that will download many s3 assets to a save directory.
func NewSingleDownloader ¶
func NewSingleDownloader(asset, saveDir string, retries int) (Downloader, error)
NewSingleDownloader returns a Downloader that will download a single s3 asset to a save directory.
type Uploader ¶
type Uploader interface {
Upload() error
}
Uploader implement Upload.
Source Files