s3sync

package
v0.0.0-...-5ac641f Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 6, 2019 License: MIT Imports: 11 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CopyWorker

func CopyWorker(source <-chan string, destpath string, downloader *s3manager.Downloader, bucket string, ignoreFiles map[string]bool, wg *sync.WaitGroup)

CopyWorker - go routine worker for doing copies from s3 to fs

func DirectoryContentsWorker

func DirectoryContentsWorker(ticker <-chan time.Time, destdir string, ignoreFiles map[string]bool, done <-chan bool, wg *sync.WaitGroup)

DirectoryContentsWorker uses the local file system and retrives files that are already on disk

func S3ListWorker

func S3ListWorker(ticker <-chan time.Time, toCopy chan<- string, SourceBucket string, s3svc *s3.S3, ignoredFiles map[string]bool, done <-chan bool, wg *sync.WaitGroup)

S3ListWorker periodically checks the contents of the bucket and queues new files for download

Types

type Syncer

type Syncer struct {
	S3SVC        *s3.S3
	SourceBucket string
	DestDir      string
	// contains filtered or unexported fields
}

Syncer struct

func NewSyncer

func NewSyncer(srcBkt, destDir, endpointURL, awsregion, awsaccessid, awsaccesskey string, disableSSL, s3ForcePathStyle bool) (syncer *Syncer, err error)

NewSyncer returns a Syncer or an error if construction fails

func (*Syncer) Close

func (syncer *Syncer) Close()

Close - shuts down the syncer correctly (ie, close the toCopy channel)

func (*Syncer) Start

func (syncer *Syncer) Start(workerNum int)

Start - starts the sync

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL