synchronizer

package

v0.0.0-...-246cd80 Latest Latest Go to latest Published: Mar 14, 2024 License: Apache-2.0 Imports: 13 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/routeviews/google-cloud-storage

Documentation ¶

Overview ¶

package synchronizer uploads missing MRT archives to the gRPC archive server. It is only compatible with RouteViews files currently.

Index ¶

type Config
type Synchronizer
- func New(cfg *Config) (*Synchronizer, error)
- func (s *Synchronizer) Sync(ctx context.Context, start, end time.Time) error

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Config ¶

type Config struct {
	FTPServer string
	FTPUser   string
	FTPPass   string

	GCSCli          *storage.Client
	UploadServerCli pb.RVClient
	ArchiveBucket   string
	HTTPRoot        string
}

type Synchronizer ¶

type Synchronizer struct {
	// contains filtered or unexported fields
}

func New ¶

func New(cfg *Config) (*Synchronizer, error)

func (*Synchronizer) Sync ¶

func (s *Synchronizer) Sync(ctx context.Context, start, end time.Time) error

Sync syncs MRT archives with the GCS backup. It uses FTP to find all missing files from the given time span, check if any of them are missing, download the missing ones through HTTP (better stability), and upload them through the gRPC upload server. Files are uploaded in order of time per collector.

Source Files ¶

View all Source files

synchronizer.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL