backfill

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 15, 2020 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Load

func Load(ctx context.Context, r io.Reader, rjc pb.RoutedJournalClient, selector pb.LabelSelector, name string) error

Load a Reader of hex-encoded key/values into DeltaEvent partitions identified by |selector|. Each DeltaEvent row is sequenced under extractor |name| and written atomically with its commit acknowledgement. This ensures that multiple calls to Load with interspersed failures will not result into double-counting of rows by the VTable service, so long as each invocation uses the same |name| and reads identical input sequences from the Reader.

func Main

func Main(extractFns factable.ExtractFns)

Types

type JobSpec

type JobSpec struct {
	SchemaSpec factable.SchemaSpec
	Inputs     map[pb.Journal]pb.JournalSpec
}

JobSpec defines common specifications of a backfill job.

type MapTaskSpec

type MapTaskSpec struct {
	*JobSpec `json:",ignore"`
	Fragment pb.Fragment
	URL      string
	MVTags   []factable.MVTag
}

MapTaskSpec defines a map tasks to be performed as part of a backfill.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL