Documentation
¶
Overview ¶
Package pipeline contains Beam pipeline library functions for the SumDB verifiable map.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CreateEntries ¶
func CreateEntries(s beam.Scope, treeID int64, records beam.PCollection) beam.PCollection
CreateEntries converts the PCollection<Metadata> into a PCollection<Entry> that will be committed to by the map.
func MakeVersionLogs ¶
func MakeVersionLogs(s beam.Scope, treeID int64, metadata beam.PCollection) (beam.PCollection, beam.PCollection)
MakeVersionLogs takes the Metadata for all modules and processes this by module in order to create logs of versions. The versions for each module are sorted (by ID in the original log), and a log is constructed for each module. This method returns two PCollections: the first is of type Entry and is the key/value data to include in the map, the second is of type ModuleVersionLog.
Types ¶
type InputLog ¶
type InputLog interface {
// Head returns the metadata of available entries.
Head() (checkpoint []byte, count int64, err error)
// Entries returns a PCollection of Metadata, containing entries in range [start, end).
Entries(s beam.Scope, start, end int64) beam.PCollection
}
InputLog allows access to entries from the SumDB.
type InputLogMetadata ¶
InputLogMetadata describes the provenance information of the input log to be passed around atomically.
type MapBuilder ¶
type MapBuilder struct {
// contains filtered or unexported fields
}
MapBuilder contains the static configuration for a map, and allows maps at different log sizes to be built using its methods.
func NewMapBuilder ¶
func NewMapBuilder(source InputLog, treeID int64, prefixStrata int, versionLogs bool) MapBuilder
NewMapBuilder returns a MapBuilder for a map with the given configuration.
func (*MapBuilder) Create ¶
func (b *MapBuilder) Create(s beam.Scope, size int64) (beam.PCollection, beam.PCollection, InputLogMetadata, error)
Create builds a map from scratch, using the first `size` entries in the input log. If there aren't enough entries then it will fail. It returns a PCollection of *Tile as the first output, and any logs built will be output in the second PCollection (of type ModuleVersionLog).
func (*MapBuilder) Update ¶
func (b *MapBuilder) Update(s beam.Scope, lastTiles beam.PCollection, provenance InputLogMetadata, size int64) (beam.PCollection, InputLogMetadata, error)
Update builds a map using the last version built, and updating it to include all the first `size` entries from the input log. If there aren't enough entries then it will fail. It returns a PCollection of *Tile as the first output.
type Metadata ¶
Metadata is the audit.Metadata object with the addition of an ID field. It must map to the scheme of the leafMetadata table.
type ModuleVersionLog ¶
ModuleVersionLog represents the versions found for a single Go Module within the SumDB log. The versions are sorted by the order they are logged in SumDB.
Source Files