luci: go.chromium.org/luci/server/tq/internal/partition Index | Files

package partition

import "go.chromium.org/luci/server/tq/internal/partition"

Package partition encapsulates partitioning and querying large keyspace which can't be expressed even as uint64.

All to/from string functions use hex encoding.

Index

Package Files

partition.go

type Partition Uses

type Partition struct {
    Low  big.Int // inclusive
    High big.Int // exclusive. May be equal to max SHA2 hash value + 1.
}

Partition represents a range [Low..High).

func FromInts Uses

func FromInts(low, high int64) *Partition

func FromString Uses

func FromString(s string) (*Partition, error)

func SpanInclusive Uses

func SpanInclusive(low, highInclusive string) (*Partition, error)

func Universe Uses

func Universe(keySpaceBytes int) *Partition

func (Partition) Copy Uses

func (p Partition) Copy() *Partition

func (Partition) EducatedSplitAfter Uses

func (p Partition) EducatedSplitAfter(exclusive string, beforeItems, targetItems, maxShards int) SortedPartitions

EducatedSplitAfter splits partition after a given boundary assuming constant density s.t. each shard has approximately targetItems.

Caps the number of resulting partitions to at most maxShards. panics if called on invalid data.

func (Partition) MarshalJSON Uses

func (p Partition) MarshalJSON() ([]byte, error)

func (Partition) QueryBounds Uses

func (p Partition) QueryBounds(keySpaceBytes int) (low, high string)

func (Partition) Split Uses

func (p Partition) Split(shards int) SortedPartitions

func (Partition) String Uses

func (p Partition) String() string

func (*Partition) UnmarshalJSON Uses

func (p *Partition) UnmarshalJSON(bs []byte) error

type SortedPartitions Uses

type SortedPartitions []*Partition

SortedPartitions are disjoint partitions sorted by ascending .Low field.

func (SortedPartitions) OnlyIn Uses

func (ps SortedPartitions) OnlyIn(n int, key func(i int) string, use func(l, h int), keySpaceBytes int)

OnlyIn efficiently returns a subsequence of the `n` sorted by key objects whose key belongs to one of the partitions.

Calls use(i,j) for each objects[i:j] which belong to the range.

type SortedPartitionsBuilder Uses

type SortedPartitionsBuilder struct {
    // contains filtered or unexported fields
}

SortedPartitionsBuilder constructs a sequence of partitions by excluding chunks from a starting partion.

Not intended to scale to large number of exclusion operations.

func NewSortedPartitionsBuilder Uses

func NewSortedPartitionsBuilder(p *Partition) SortedPartitionsBuilder

func (*SortedPartitionsBuilder) Exclude Uses

func (b *SortedPartitionsBuilder) Exclude(exclude *Partition)

func (*SortedPartitionsBuilder) IsEmpty Uses

func (b *SortedPartitionsBuilder) IsEmpty() bool

func (*SortedPartitionsBuilder) Result Uses

func (b *SortedPartitionsBuilder) Result() SortedPartitions

Package partition imports 6 packages (graph) and is imported by 8 packages. Updated 2020-09-30. Refresh now. Tools for package owners.