luci: go.chromium.org/luci/server/tq/internal/partition Index | Files

package partition

import "go.chromium.org/luci/server/tq/internal/partition"

Package partition encapsulates partitioning and querying large keyspace which can't be expressed even as uint64.

All to/from string functions use hex encoding.

Index

Package Files

partition.go

type Partition Uses

❖
type Partition struct {
    Low  big.Int // inclusive
    High big.Int // exclusive. May be equal to max SHA2 hash value + 1.
}

Partition represents a range [Low..High).

func FromInts Uses

❖
func FromInts(low, high int64) *Partition

func FromString Uses

❖
func FromString(s string) (*Partition, error)

func SpanInclusive Uses

❖
func SpanInclusive(low, highInclusive string) (*Partition, error)

func Universe Uses

❖
func Universe(keySpaceBytes int) *Partition

func (Partition) Copy Uses

❖
func (p Partition) Copy() *Partition

func (Partition) EducatedSplitAfter Uses

❖
func (p Partition) EducatedSplitAfter(exclusive string, beforeItems, targetItems, maxShards int) SortedPartitions

EducatedSplitAfter splits partition after a given boundary assuming constant density s.t. each shard has approximately targetItems.

Caps the number of resulting partitions to at most maxShards. panics if called on invalid data.

func (Partition) MarshalJSON Uses

❖
func (p Partition) MarshalJSON() ([]byte, error)

func (Partition) QueryBounds Uses

❖
func (p Partition) QueryBounds(keySpaceBytes int) (low, high string)

func (Partition) Split Uses

❖
func (p Partition) Split(shards int) SortedPartitions

func (Partition) String Uses

❖
func (p Partition) String() string

func (*Partition) UnmarshalJSON Uses

❖
func (p *Partition) UnmarshalJSON(bs []byte) error

type SortedPartitions Uses

❖
type SortedPartitions []*Partition

SortedPartitions are disjoint partitions sorted by ascending .Low field.

func (SortedPartitions) OnlyIn Uses

❖
func (ps SortedPartitions) OnlyIn(n int, key func(i int) string, use func(l, h int), keySpaceBytes int)

OnlyIn efficiently returns a subsequence of the `n` sorted by key objects whose key belongs to one of the partitions.

Calls use(i,j) for each objects[i:j] which belong to the range.

type SortedPartitionsBuilder Uses

❖
type SortedPartitionsBuilder struct {
    // contains filtered or unexported fields
}

SortedPartitionsBuilder constructs a sequence of partitions by excluding chunks from a starting partion.

Not intended to scale to large number of exclusion operations.

func NewSortedPartitionsBuilder Uses

❖
func NewSortedPartitionsBuilder(p *Partition) SortedPartitionsBuilder

func (*SortedPartitionsBuilder) Exclude Uses

❖
func (b *SortedPartitionsBuilder) Exclude(exclude *Partition)

func (*SortedPartitionsBuilder) IsEmpty Uses

❖
func (b *SortedPartitionsBuilder) IsEmpty() bool

func (*SortedPartitionsBuilder) Result Uses

❖
func (b *SortedPartitionsBuilder) Result() SortedPartitions

Package partition imports 6 packages (graph) and is imported by 8 packages. Updated 2020-09-30. Refresh now. Tools for package owners.