dvid: github.com/janelia-flyem/dvid/datatype/common/labels Index | Files

package labels

import "github.com/janelia-flyem/dvid/datatype/common/labels"

Package labels supports label-based data types like labelblk, labelvol, labelsurf, labelsz, etc. Basic 64-bit label data and deltas are kept here so all label-based data types can use them without cyclic package dependencies, especially when writing code to synchronize across data instances.

Index

Package Files

compressed.go compressed_old.go events.go index.go labels.go mutlog.go

Constants

const (
    IngestBlockEvent          = imageblk.IngestBlockEvent
    MutateBlockEvent          = imageblk.MutateBlockEvent
    DeleteBlockEvent          = imageblk.DeleteBlockEvent
    SparsevolStartEvent       = "SPARSEVOL_START"
    SparsevolModEvent         = "SPARSEVOL_MOD"
    SparsevolEndEvent         = "SPARSEVOL_END"
    ChangeSizeEvent           = "LABEL_SIZE_CHANGE"
    MergeStartEvent           = "MERGE_START"
    MergeBlockEvent           = "MERGE_BLOCK"
    MergeEndEvent             = "MERGE_END"
    SplitStartEvent           = "SPLIT_START"
    SplitLabelEvent           = "SPLIT_LABEL"
    SplitEndEvent             = "SPLIT_END"
    CleaveStartEvent          = "CLEAVE_START"
    CleaveLabelEvent          = "CLEAVE_LABEL"
    CleaveEndEvent            = "CLEAVE_END"
    SupervoxelSplitStartEvent = "SV_SPLIT_START"
    SupervoxelSplitEvent      = "SV_SPLIT"
    SupervoxelSplitEndEvent   = "SV_SPLIT_END"
)

Label change event identifiers

const DefaultBlockSize = DefaultSubBlocksPerBlock * SubBlockSize
const DefaultSubBlocksPerBlock = 8
const (
    // MaxAllowedLabel is the largest label that should be allowed by DVID if we want
    // to take into account the maximum integer size within Javascript (due to its
    // underlying use of a double float for numbers, leading to max int = 2^53 - 1).
    // This would circumvent the need to use strings within JSON (e.g., the Google
    // solution) to represent integer labels that could exceed the max javascript
    // number.  It would require adding a value check on each label voxel of a
    // mutation request, which might be too much of a hit to handle an edge case.
    MaxAllowedLabel = 9007199254740991
)
const MaxBlockSize = 1024 // N^3 < max uint32, so N <= 2^10
const MaxSubBlockSize = MaxBlockSize / SubBlockSize
const SubBlockSize = 8

func BlockIndexToIZYXString Uses

func BlockIndexToIZYXString(zyx uint64) dvid.IZYXString

BlockIndexToIZYXString decodes a packed block index into an IZYXString. At most, each block int32 coordinate can be 20 bits.

func DecodeBlockIndex Uses

func DecodeBlockIndex(zyx uint64) (x, y, z int32)

DecodeBlockIndex decodes a packed block index into int32 coordinates. At most, each block int32 coordinate can be 20 bits.

func DownresLabels Uses

func DownresLabels(hires []byte, hisize dvid.Point3d) (lores []byte, err error)

DownresLabels takes an array of uint64 in []byte format and returns a 2x down-sampled array of uint64. An error is returned if size is not multiple of two or passed array is incorrect size.

func EncodeBlockIndex Uses

func EncodeBlockIndex(x, y, z int32) (zyx uint64)

EncodeBlockIndex converts signed (x,y,z) block coordinate into a single uint64, which is packed in ZYX order with MSB empty, the most-significant 21 bits is Z (21st bit is sign flag), next 21 bits is Y, then least-significant 21 bits is X.

func IZYXStringToBlockIndex Uses

func IZYXStringToBlockIndex(s dvid.IZYXString) (zyx uint64, err error)

IZYXStringToBlockIndex returns an encoded Block Index for a given IZYXString, returning an error if the IZYXString is formatted incorrectly.

func LogAffinity Uses

func LogAffinity(d dvid.Data, v dvid.VersionID, aff Affinity) error

func LogCleave Uses

func LogCleave(d dvid.Data, v dvid.VersionID, op CleaveOp) error

LogCleave logs the cleave of supervoxels to a label.

func LogMapping Uses

func LogMapping(d dvid.Data, v dvid.VersionID, op MappingOp) error

LogMapping logs the mapping of supervoxels to a label.

func LogMappings Uses

func LogMappings(d dvid.Data, v dvid.VersionID, ops proto.MappingOps) error

LogMappings logs a collection of mapping operations to a UUID.

func LogMerge Uses

func LogMerge(d dvid.Data, v dvid.VersionID, op MergeOp) error

LogMerge logs the merge of supervoxels to a label.

func LogSplit Uses

func LogSplit(d dvid.Data, v dvid.VersionID, op SplitOp) error

LogSplit logs the split of a set of voxels from the underlying label.

func LogSupervoxelSplit Uses

func LogSupervoxelSplit(d dvid.Data, v dvid.VersionID, op SplitSupervoxelOp) error

LogSupervoxelSplit logs the split of a supervoxel into two separate supervoxels.

func MergeStart Uses

func MergeStart(iv dvid.InstanceVersion, op MergeOp) error

MergeStart handles label map caches during an active merge operation. Note that if there are multiple synced label instances, the InstanceVersion will always be the labelblk instance. Multiple merges into a single label are allowed, but chained merges are not. For example, you can merge label 1, 2, and 3 into 4, then later merge 6 into 4. However, you cannot concurrently merge label 4 into some other label because there can be a race condition between 3 -> 4 and 4 -> X.

func MergeStop Uses

func MergeStop(iv dvid.InstanceVersion, op MergeOp)

MergeStop marks the end of a merge operation.

func SplitStart Uses

func SplitStart(iv dvid.InstanceVersion, op DeltaSplitStart) error

SplitStart checks current label map to see if the split conflicts.

func SplitStop Uses

func SplitStop(iv dvid.InstanceVersion, op DeltaSplitEnd)

SplitStop marks the end of a split operation.

func StreamLog Uses

func StreamLog(d dvid.Data, v dvid.VersionID, ch chan storage.LogMessage, wg *sync.WaitGroup) error

func WriteBinaryBlocks Uses

func WriteBinaryBlocks(mainLabel uint64, lbls Set, op *OutputOp, bounds dvid.Bounds)

WriteBinaryBlocks writes a compact serialization of a binarized Block to the supplied Writer. The serialization is a header + stream of blocks. The header is the following:

 3 * uint32      values of gx, gy, and gz
 uint64          foreground label

The format of each binary block in the stream is detailed by the WriteBinaryBlock() function.

func WriteRLEs Uses

func WriteRLEs(lbls Set, op *OutputOp, bounds dvid.Bounds)

WriteRLEs, like WriteBinaryBlocks, writes a compact serialization of a binarized Block to the supplied Writer. In this case, the serialization uses little-endian encoded integers and RLEs with the repeating units of the following format:

int32   Coordinate of run start (dimension 0)
int32   Coordinate of run start (dimension 1)
int32   Coordinate of run start (dimension 2)
int32   Length of run in X direction

The offset is the DVID space offset to the first voxel in the Block. After the RLEs have been written to the io.Writer, an error message is sent down the given errCh.

type Affinity Uses

type Affinity struct {
    Label1 uint64
    Label2 uint64
    Value  float32
}

Affinity represents a float value associated with a two-tuple of labels.

type BinaryBlock Uses

type BinaryBlock struct {
    Offset dvid.Point3d // voxel offset for the first voxels in this block
    Size   dvid.Point3d
    Label  uint64
    Voxels []bool
}

func ReceiveBinaryBlocks Uses

func ReceiveBinaryBlocks(r io.Reader) ([]BinaryBlock, error)

ReceiveBinaryBlocks returns a slice of BinaryBlock, easily parseable but not necessarily optimally compressed format.

func (*BinaryBlock) Read Uses

func (b *BinaryBlock) Read(r io.Reader, gx, gy, gz int32, label uint64) error

func (BinaryBlock) String Uses

func (b BinaryBlock) String() string

type Block Uses

type Block struct {
    Labels []uint64     // labels in Block.
    Size   dvid.Point3d // # voxels in each dimension for this block

    NumSBLabels []uint16 // # of labels for each sub-block
    SBIndices   []uint32 // indices into Labels array
    SBValues    []byte   // compressed voxel values giving index into SBIndices.
    // contains filtered or unexported fields
}

Block is the unit of storage for compressed DVID labels. It is inspired by the Neuroglancer compression scheme and makes the following changes: (1) a block-level label list with sub-block indices into the list (minimal required bits vs 64 bits in original Neuroglancer scheme), (2) the number of bits for encoding values is not required to be a power of two. A block-level label list allows easy sharing of labels between sub-blocks, and sub-block storage can be more efficient due to the smaller index (at the cost of an indirection) and better encoded value packing (at the cost of byte alignment). In both cases memory is gained for increased computation.

Blocks cover nx * ny * nz voxels. This implementation allows any choice of nx, ny, and nz with two restrictions: (1) nx, ny, and nz must be a multiple of 8 greater than 16, and (2) the total number of labels cannot exceed the capacity of a uint32.

Internally, labels are stored in 8x8x8 sub-blocks. There are gx * gy * gz sub-blocks where gx = nx / 8; gy = ny / 8; gz = nz / 8.

The byte layout will be the following if there are N labels in the Block:

     3 * uint32      values of gx, gy, and gz
     uint32          # of labels (N), cannot exceed uint32.
     N * uint64      packed labels in little-endian format.  Label 0 can be
							used to represent deleted labels, e.g., after a merge
							operation to avoid changing all sub-block indices.

     ----- Data below is only included if N > 1, otherwise it is a solid block.
           Nsb = # sub-blocks = gx * gy * gz

     Nsb * uint16        # of labels for sub-blocks (Ns[i]).
                             Each uint16 Ns[i] = # labels for sub-block i.
                             If Ns[i] == 0, the sub-block has no data
								(uninitialized), which is useful for constructing
								Blocks with sparse data.

     Nsb * Ns * uint32   label indices for sub-blocks where Ns = sum of Ns[i]
								over all sub-blocks. For each sub-block i, we have
								Ns[i] label indices of lBits.

     Nsb * values        sub-block indices for each voxel.
                             Data encompasses 512 * ceil(log2(Ns[i])) bits,
								padded so no two sub-blocks have indices in the
								same byte. At most we use 9 bits per voxel for up
								to the 512 labels in sub-block. A value gives the
								sub-block index which points to the index into
                             the N labels.  If Ns[i] <= 1, there are no values.
								If Ns[i] = 0, the 8x8x8 voxels are set to label 0.
								If Ns[i] = 1, all voxels are the given label index.

func MakeBlock Uses

func MakeBlock(uint64array []byte, bsize dvid.Point3d) (*Block, error)

MakeBlock returns a compressed label Block given a packed little-endian uint64 label array. It is the inverse of MakeLabelVolume(). There is no sharing of underlying memory between the returned Block and the given byte slice.

func MakeSolidBlock Uses

func MakeSolidBlock(label uint64, blockSize dvid.Point3d) *Block

MakeSolidBlock returns a Block that represents a single label of the given block size.

func SubvolumeToBlock Uses

func SubvolumeToBlock(sv *dvid.Subvolume, lbls []byte, idx dvid.IndexZYX, bsize dvid.Point3d) (*Block, error)

SubvolumeToBlock converts a portion of the given label array into a compressed Block. It accepts a packed little-endian uint64 label array and a description of its subvolume, i.e., its extents in dvid space, and returns a compressed Block for the given chunk when tiling dvid space with the given chunk size.

func (Block) CalcNumLabels Uses

func (b Block) CalcNumLabels(prev *Block) map[uint64]int32

CalcNumLabels calculates the change in the number of voxels under each non-zero label. If a previous Block is given, the change is calculated from the previous numbers.

func (Block) CompressGZIP Uses

func (b Block) CompressGZIP() ([]byte, error)

CompressGZIP returns a gzip compressed encoding of the serialized block data.

func (*Block) Downres Uses

func (b *Block) Downres(octants [8]*Block) error

Downres takes eight Blocks that represent higher-resolution octants (by 2x) of the receiving block, and modifies the receiving Block to be a half-resolution representation. If a given octant is a nil Block, the receiving Block is not modified for that portion of the higher-resolution octant.

func (*Block) DownresSlow Uses

func (b *Block) DownresSlow(octants [8]*Block) error

DownresSlow is same as Downres() but uses simpler and more memory/compute-intensive approach to computing down-res block.

func (*Block) GetPointLabels Uses

func (b *Block) GetPointLabels(pts []dvid.Point3d) []uint64

GetPointLabels returns the labels associated with each point by traversing the compressed data. If the point is outside the block, a zero is returned.

func (Block) MakeLabelVolume Uses

func (b Block) MakeLabelVolume() (uint64array []byte, size dvid.Point3d)

MakeLabelVolume returns a byte slice with packed little-endian uint64 labels in ZYX order, i.e., a uint64 for each voxel where consecutive values are in the (x,y,z) order: (0,0,0), (1,0,0), (2,0,0) ... (0,1,0) There is no sharing of memory between the returned byte slice and the Block data.

func (Block) MarshalBinary Uses

func (b Block) MarshalBinary() ([]byte, error)

MarshalBinary implements the encoding.BinaryMarshaler interface. Note that for efficiency, the returned byte slice will share memory with the receiver Block.

func (*Block) MergeLabels Uses

func (b *Block) MergeLabels(op MergeOp) (merged *Block, err error)

MergeLabels returns a new block that has computed the given MergeOp.

func (*Block) ReplaceLabel Uses

func (b *Block) ReplaceLabel(target, newLabel uint64) (replace *Block, replaceSize uint64, err error)

ReplaceLabel replaces references to the target label with newLabel.

func (*Block) ReplaceLabels Uses

func (b *Block) ReplaceLabels(mapping map[uint64]uint64) (replace *Block, replaced bool, err error)

ReplaceLabels replaces labels according to mapping and doesn't compute sizes.

func (Block) StringDump Uses

func (b Block) StringDump(verbose bool) string

StringDump returns a string that lists pretty-printed data from the block.

func (*Block) UnmarshalBinary Uses

func (b *Block) UnmarshalBinary(data []byte) error

UnmarshalBinary implements the encoding.BinaryUnmarshaler interface. The source byte slice is copied into a new 8-byte aligned slice so the receiver block does not depend on the passed slice.

func (*Block) Value Uses

func (b *Block) Value(pos dvid.Point3d) uint64

Value returns the label for a voxel using its 3d location within block. If the given location is outside the block extent, label 0 is returned. Note that this function is inefficient for multi-voxel value retrieval.

func (Block) WriteGoogleCompression Uses

func (b Block) WriteGoogleCompression(w io.Writer) error

GoogleCompression writes label compression compliant with the Google Neuroglancer specification: https://goo.gl/IyQbzL

type CleaveOp Uses

type CleaveOp struct {
    MutID              uint64
    Target             uint64
    CleavedLabel       uint64
    CleavedSupervoxels []uint64
}

CleaveOp represents a cleave of a label using supervoxels.

type Counts Uses

type Counts struct {
    sync.RWMutex
    // contains filtered or unexported fields
}

Counts is a thread-safe type for counting label references.

func (*Counts) Decr Uses

func (c *Counts) Decr(label uint64)

Decr decrements the count for a label.

func (*Counts) Empty Uses

func (c *Counts) Empty() bool

Empty returns true if there are no counts.

func (*Counts) Incr Uses

func (c *Counts) Incr(label uint64)

Incr increments the count for a label.

func (*Counts) Value Uses

func (c *Counts) Value(label uint64) int

Value returns the count for a label.

type DeltaDeleteSize Uses

type DeltaDeleteSize struct {
    Label    uint64
    OldSize  uint64
    OldKnown bool // true if OldSize is valid, otherwise delete all size k/v for this label.
}

DeltaDeleteSize gives info to delete a label's size.

type DeltaMerge Uses

type DeltaMerge struct {
    MergeOp
    Blocks       dvid.IZYXSlice               // not nil if labelarray used.
    BlockMap     map[dvid.IZYXString]struct{} // not nil if labelblk used, to be deprecated.
    TargetVoxels uint64
    MergedVoxels uint64
}

DeltaMerge describes the labels and blocks affected by a merge operation. It is sent during a MergeBlockEvent.

type DeltaMergeEnd Uses

type DeltaMergeEnd struct {
    MergeOp
}

DeltaMergeEnd is the data sent during a MergeEndEvent.

type DeltaMergeStart Uses

type DeltaMergeStart struct {
    MergeOp
}

DeltaMergeStart is the data sent during a MergeStartEvent.

type DeltaModSize Uses

type DeltaModSize struct {
    Label      uint64
    SizeChange int64 // Adds to old label size
}

DeltaModSize gives info to modify an existing label size without knowing the old size.

type DeltaNewSize Uses

type DeltaNewSize struct {
    Label uint64
    Size  uint64
}

DeltaNewSize is a new label being introduced.

type DeltaReplaceSize Uses

type DeltaReplaceSize struct {
    Label   uint64
    OldSize uint64
    NewSize uint64
}

DeltaReplaceSize gives info to precisely remove an old label size and add the updated size.

type DeltaSparsevol Uses

type DeltaSparsevol struct {
    Label uint64
    Mods  dvid.BlockRLEs
}

DeltaSparsevol describes a change to an existing label.

type DeltaSplit Uses

type DeltaSplit struct {
    OldLabel     uint64
    NewLabel     uint64
    Split        dvid.BlockRLEs
    SortedBlocks dvid.IZYXSlice
    SplitVoxels  uint64
}

DeltaSplit describes the voxels modified during a split operation. The Split field may be null if this is a coarse split only defined by block indices.

type DeltaSplitEnd Uses

type DeltaSplitEnd struct {
    OldLabel uint64
    NewLabel uint64
}

DeltaSplitEnd is the data sent during a SplitEndEvent.

type DeltaSplitStart Uses

type DeltaSplitStart struct {
    OldLabel uint64
    NewLabel uint64
}

DeltaSplitStart is the data sent during a SplitStartEvent.

type Index Uses

type Index struct {
    proto.LabelIndex
}

func (*Index) Add Uses

func (idx *Index) Add(idx2 *Index) error

Add adds the given Index to the receiver.

func (*Index) Cleave Uses

func (idx *Index) Cleave(cleaveLabel uint64, toCleave []uint64) *Index

Cleave the given supervoxels from an index and returns a new index, modifying both receiver and creating new cleaved index.

func (Index) Equal Uses

func (idx Index) Equal(idx2 Index) bool

Equal returns true if the receiver and passed Index are equivalent.

func (*Index) FitToBounds Uses

func (idx *Index) FitToBounds(bounds *dvid.OptionalBounds) error

FitToBounds modifies the receiver to fit the given optional block bounds.

func (*Index) GetBlockIndices Uses

func (idx *Index) GetBlockIndices() dvid.IZYXSlice

GetBlockIndices returns the block coordinates within the Index.

func (*Index) GetProcessedBlockIndices Uses

func (idx *Index) GetProcessedBlockIndices(scale uint8, bounds dvid.Bounds) (dvid.IZYXSlice, error)

GetProcessedBlockIndices returns the blocks for an index, possibly with bounds and down-res. The returned blocks are not sorted.

func (*Index) GetSupervoxelCount Uses

func (idx *Index) GetSupervoxelCount(supervoxel uint64) (count uint64)

GetSupervoxelCount returns the # of voxels for a supervoxel in an Index. Note that the counts are uint64 because although each block might only hold a # of voxels < max uint32, a massive supervoxel could hold many more.

func (*Index) GetSupervoxelCounts Uses

func (idx *Index) GetSupervoxelCounts() (counts map[uint64]uint64)

GetSupervoxelCounts returns the # of voxels for each supervoxel in an Index. Note that the counts are uint64 because although each block might only hold a # of voxels < max uint32, a massive supervoxel could hold many more.

func (*Index) GetSupervoxels Uses

func (idx *Index) GetSupervoxels() Set

GetSupervoxels returns a set of supervoxels within the receiver Index.

func (*Index) GetSupervoxelsBlocks Uses

func (idx *Index) GetSupervoxelsBlocks(supervoxels Set) map[dvid.IZYXString]struct{}

GetSupervoxelsBlocks returns the blocks for a given list of supervoxels.

func (*Index) LimitToSupervoxel Uses

func (idx *Index) LimitToSupervoxel(supervoxel uint64) (*Index, error)

LimitToSupervoxel returns a copy of the index but with only the given supervoxel

func (*Index) ModifyBlocks Uses

func (idx *Index) ModifyBlocks(label uint64, sc SupervoxelChanges) error

ModifyBlocks modifies the receiver Index to incorporate supervoxel changes among the given blocks.

func (Index) NumVoxels Uses

func (idx Index) NumVoxels() uint64

NumVoxels returns the number of voxels for the Index.

func (Index) StringDump Uses

func (idx Index) StringDump(showMutationInfo bool) string

StringDump returns a description of the data within the Index. If showMutationInfo is true, the mutation ID and information about modification is also printed.

func (*Index) SupervoxelsPresent Uses

func (idx *Index) SupervoxelsPresent(supervoxels Set) (present map[uint64]bool)

SupervoxelsPresent checks whether each label from a Set are within the index.

type Mapping Uses

type Mapping struct {
    sync.RWMutex
    // contains filtered or unexported fields
}

Mapping is a thread-safe, mapping of labels to labels in both forward and backward direction. Mutation of a Mapping instance can only be done through labels.MergeCache.

func LabelMap Uses

func LabelMap(iv dvid.InstanceVersion) *Mapping

LabelMap returns a label mapping for a version of a data instance. If no label mapping is available, a nil is returned.

func (*Mapping) ConstituentLabels Uses

func (m *Mapping) ConstituentLabels(final uint64) Set

ConstituentLabels returns a set of labels that will be mapped to the given label. The set will always include the given label.

func (*Mapping) FinalLabel Uses

func (m *Mapping) FinalLabel(start uint64) (uint64, bool)

FinalLabel follows mappings from a start label until a final mapped label is reached.

func (*Mapping) Get Uses

func (m *Mapping) Get(label uint64) (uint64, bool)

Get returns the mapping or false if no mapping exists.

type MappingOp Uses

type MappingOp struct {
    MutID    uint64
    Mapped   uint64
    Original Set
}

MappingOp represents a mapping of a set of original labels into a mapped lapel.

func ReadMappingLog Uses

func ReadMappingLog(d dvid.Data, v dvid.VersionID) ([]MappingOp, error)

func (MappingOp) Marshal Uses

func (op MappingOp) Marshal() (serialization []byte, err error)

Marshal returns a proto.MappingOp serialization

type MergeOp Uses

type MergeOp struct {
    MutID  uint64
    Target uint64
    Merged Set
}

MergeOp represents the merging of a set of labels into a target label.

func (MergeOp) String Uses

func (op MergeOp) String() string

type MergeTuple Uses

type MergeTuple []uint64

MergeTuple represents a merge of labels. Its first element is the destination label and all later elements in the slice are labels to be merged. It's an easy JSON representation as a list of labels.

func (MergeTuple) Op Uses

func (t MergeTuple) Op() (MergeOp, error)

Op converts a MergeTuple into a MergeOp.

type OutputOp Uses

type OutputOp struct {
    sync.Mutex // lock on writing
    // contains filtered or unexported fields
}

OutputOp provides a way to communicate with writing goroutines, TODO: concurrency support on the given io.Writer.

func NewOutputOp Uses

func NewOutputOp(w io.Writer) *OutputOp

func (OutputOp) Finish Uses

func (op OutputOp) Finish() error

Finish signals all input to an OutputOp is done and waits for completion. Any error from the OutputOp is returned.

func (OutputOp) Process Uses

func (op OutputOp) Process(pb *PositionedBlock)

type PositionedBlock Uses

type PositionedBlock struct {
    Block
    BCoord dvid.IZYXString
}

PositionedBlock is a Block that also knows its position in DVID space via a chunk coordinate.

func (PositionedBlock) DoSplitWithStats Uses

func (pb PositionedBlock) DoSplitWithStats(op SplitOp, m *SVSplitMap, newLabelFunc func() (uint64, error)) (split *Block, counts map[uint64]SVSplitCount, err error)

DoSplitWithStats writes a new label into the RLEs defined by the split and returns how each supervoxel (counts key) was split. This is done by doing full expansion of block into uint64 array.

func (PositionedBlock) OffsetDVID Uses

func (pb PositionedBlock) OffsetDVID() (dvid.Point3d, error)

OffsetDVID returns the DVID voxel coordinate corresponding to the first voxel of the Block, i.e., the lowest (x,y,z).

func (PositionedBlock) Split Uses

func (pb PositionedBlock) Split(op SplitOp) (split *Block, keptSize, splitSize uint64, err error)

Split a target label using RLEs within a block. Only the target label is split. A nil split block is returned if target label is not within block. TODO: If RLEs continue to be used for splits, refactor / split up to make this more readable.

func (PositionedBlock) SplitStats Uses

func (pb PositionedBlock) SplitStats(rles dvid.RLEs, m *SVSplitMap, newLabelFunc func() (uint64, error)) (counts map[uint64]SVSplitCount, err error)

SplitStats checks voxels under the split RLEs and returns how each supervoxel should be split without doing a split. This is done by doing full expansion of block into uint64 array.

func (PositionedBlock) SplitSupervoxel Uses

func (pb PositionedBlock) SplitSupervoxel(op SplitSupervoxelOp) (split *Block, keptSize, splitSize uint64, err error)

SplitSupervoxel splits a target supervoxel using RLEs within a block.

func (PositionedBlock) SplitSupervoxels Uses

func (pb PositionedBlock) SplitSupervoxels(rles dvid.RLEs, svsplits map[uint64]SVSplit) (split *Block, err error)

SplitSupervoxels replaces all split supervoxels in a block with either a split label or a remain label depending on whether it falls under the split RLEs.

func (PositionedBlock) String Uses

func (pb PositionedBlock) String() string

func (*PositionedBlock) WriteBinaryBlock Uses

func (pb *PositionedBlock) WriteBinaryBlock(indices map[uint32]struct{}, hasBackground bool, op *OutputOp, bounds dvid.Bounds) error

WriteBinaryBlock writes the binary version of a Block to the supplied Writer, where the serialized data represents just the label voxels. By definition, a binary block has at most two labels (0 = background, 1 = given label) and encoding is a bit per voxel. The binary format is related to the Google and internal DVID label block compression but is simplified, the DVID space offset of the block is included, and the sub-block data are arranged to allow streaming.

Internally, the mask is stored in 8x8x8 sub-blocks. There are gx * gy * gz sub-blocks where gx = nx / 8; gy = ny / 8; gz = nz / 8, and (gx, gy, gz) is relayed in a header outside of the data returned by this function. For example, for a full sparse volume response, there would be a header followed by some number of these binary blocks.

The byte layout will be the following:

3 * int32       offset of first voxel of Block in DVID space (x, y, z)
byte            content flag:
                0 = background ONLY  (no more data for this block)
                1 = foreground ONLY  (no more data for this block)
                2 = both background and foreground so stream of sub-blocks required.

Stream of gx * gy * gz sub-blocks with the following data:

byte            content flag:
                0 = background ONLY  (no more data for this sub-block)
                1 = foreground ONLY  (no more data for this sub-block)
                2 = both background and foreground so mask data required.
mask            64 byte bitmask where each voxel is 0 (background) or 1 (foreground)

type SVSplit Uses

type SVSplit struct {
    Split  uint64 // label corresponding to split sparse volume
    Remain uint64 // relabeling of supervoxel that remains after split
}

SVSplit provides labels after a supervoxel split

type SVSplitCount Uses

type SVSplitCount struct {
    SVSplit
    Voxels uint32 // number of voxels split
}

SVSplitCount provides both labels and the # voxels after a supervoxel split.

type SVSplitMap Uses

type SVSplitMap struct {
    sync.RWMutex
    Splits map[uint64]SVSplit
}

SVSplitMap is a thread-safe mapping of supervoxels labels to their new split labels.

type Set Uses

type Set map[uint64]struct{}

Set is a set of labels.

func NewSet Uses

func NewSet(lbls ...uint64) Set

Merge returns a set made of the given labels.

func (Set) Copy Uses

func (s Set) Copy() Set

Copy returns a duplicate of the Set.

func (Set) Exists Uses

func (s Set) Exists(i uint64) bool

Exists returns true if the given uint64 is present in the Set.

func (Set) Merge Uses

func (s Set) Merge(s2 Set)

Merge adds the elements in the given set to the receiver.

func (Set) String Uses

func (s Set) String() string

type SplitOp Uses

type SplitOp struct {
    MutID    uint64
    Target   uint64
    NewLabel uint64
    RLEs     dvid.RLEs
    Coarse   bool // true if the RLEs are block coords (coarse split), not voxels.
    SplitMap map[uint64]SVSplit
}

SplitOp represents a split with the sparse volume of the new label.

type SplitSupervoxelOp Uses

type SplitSupervoxelOp struct {
    MutID            uint64
    Supervoxel       uint64
    SplitSupervoxel  uint64
    RemainSupervoxel uint64
    Split            dvid.BlockRLEs
}

SplitSupervoxelOp describes a supervoxel split.

type SupervoxelChanges Uses

type SupervoxelChanges map[uint64]map[dvid.IZYXString]int32

SupervoxelChanges tabulates changes in voxels among supervoxels across blocks.

Package labels imports 13 packages (graph) and is imported by 12 packages. Updated 2019-04-12. Refresh now. Tools for package owners.