beam: github.com/apache/beam/sdks/go/pkg/beam/io/rtrackers/offsetrange Index | Files

package offsetrange

import "github.com/apache/beam/sdks/go/pkg/beam/io/rtrackers/offsetrange"

Package offsetrange defines a restriction and restriction tracker for offset ranges. An offset range is just a range, with a start and end, that can begin at an offset, and is commonly used to represent byte ranges for files or indices for iterable containers.

Index

Package Files

offsetrange.go

type Restriction Uses

type Restriction struct {
    Start, End int64
}

Restriction is an offset range restriction, which represents a range of integers as a half-closed interval with boundaries [start, end).

func (Restriction) EvenSplits Uses

func (r Restriction) EvenSplits(num int64) (splits []Restriction)

EvenSplits splits a restriction into a number of evenly sized restrictions. Each split restriction is guaranteed to not be empty, and each unit from the original restriction is guaranteed to be contained in one split restriction.

Num should be greater than 0. Otherwise there is no way to split the restriction and this function will return the original restriction.

func (Restriction) Size Uses

func (r Restriction) Size() float64

Size returns the restriction's size as the difference between Start and End.

type Tracker Uses

type Tracker struct {
    // contains filtered or unexported fields
}

Tracker tracks a restriction that can be represented as a range of integer values, for example for byte offsets in a file, or indices in an array. Note that this tracker makes no assumptions about the positions of blocks within the range, so users must handle validation of block positions if needed.

func NewTracker Uses

func NewTracker(rest Restriction) *Tracker

NewTracker is a constructor for an Tracker given a start and end range.

func (*Tracker) GetError Uses

func (tracker *Tracker) GetError() error

GetError returns the error that caused the tracker to stop, if there is one.

func (*Tracker) GetProgress Uses

func (tracker *Tracker) GetProgress() (done, remaining float64)

GetProgress reports progress based on the claimed size and unclaimed sizes of the restriction.

func (*Tracker) GetRestriction Uses

func (tracker *Tracker) GetRestriction() interface{}

GetRestriction returns a copy of the tracker's underlying offsetrange.Restriction.

func (*Tracker) IsDone Uses

func (tracker *Tracker) IsDone() bool

IsDone returns true if the most recent claimed element is past the end of the restriction.

func (*Tracker) TryClaim Uses

func (tracker *Tracker) TryClaim(rawPos interface{}) bool

TryClaim accepts an int64 position representing the starting position of a block of work. It successfully claims it if the position is greater than the previously claimed position and within the restriction. Claiming a position at or beyond the end of the restriction signals that the entire restriction has been processed and is now done, at which point this method signals to end processing.

The tracker stops with an error if a claim is attempted after the tracker has signalled to stop, if a position is claimed before the start of the restriction, or if a position is claimed before the latest successfully claimed.

func (*Tracker) TrySplit Uses

func (tracker *Tracker) TrySplit(fraction float64) (primary, residual interface{}, err error)

TrySplit splits at the nearest integer greater than the given fraction of the remainder. If the fraction given is outside of the [0, 1] range, it is clamped to 0 or 1.

Package offsetrange imports 7 packages (graph) and is imported by 2 packages. Updated 2020-09-27. Refresh now. Tools for package owners.