posting

package
v83.0.0-...-38e0eed Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 12, 2017 License: AGPL-3.0, Apache-2.0 Imports: 29 Imported by: 0

Documentation

Overview

  • Copyright (C) 2017 Dgraph Labs, Inc. and Contributors *
  • This program is free software: you can redistribute it and/or modify
  • it under the terms of the GNU Affero General Public License as published by
  • the Free Software Foundation, either version 3 of the License, or
  • (at your option) any later version. *
  • This program is distributed in the hope that it will be useful,
  • but WITHOUT ANY WARRANTY; without even the implied warranty of
  • MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
  • GNU Affero General Public License for more details. *
  • You should have received a copy of the GNU Affero General Public License
  • along with this program. If not, see <http://www.gnu.org/licenses/>.

Package posting takes care of posting lists. It contains logic for mutation layers, merging them with RocksDB, etc.

Package lru implements an LRU cache.

Index

Constants

View Source 
const (
	// Set means overwrite in mutation layer. It contributes 0 in Length.
	Set uint32 = 0x01
	// Del means delete in mutation layer. It contributes -1 in Length.
	Del uint32 = 0x02
	// Add means add new element in mutation layer. It contributes 1 in Length.
	Add uint32 = 0x03
)
View Source 
const (
	MB = 1 << 20
)

Variables

View Source 
var (
	// ErrRetry can be triggered if the posting list got deleted from memory due to a hard commit.
	// In such a case, retry.
	ErrRetry = fmt.Errorf("Temporary Error. Please retry.")
	// ErrNoValue would be returned if no value was found in the posting list.
	ErrNoValue = fmt.Errorf("No value found")
)

Functions

func CommitLists 

func CommitLists(numRoutines int, gid uint32)

func DeleteAll 

func DeleteAll() error

func DeleteCountIndex 

func DeleteCountIndex(ctx context.Context, attr string) error

func DeleteIndex 

func DeleteIndex(ctx context.Context, attr string) error

func DeletePredicate 

func DeletePredicate(ctx context.Context, attr string) error

func DeleteReverseEdges 

func DeleteReverseEdges(ctx context.Context, attr string) error

func EvictLRU 

func EvictLRU()

This doesn't sync, so call this only when you don't care about dirty posting lists in memory(for example before populating snapshot) or after calling syncAllMarks

func Init 

func Init(ps *badger.KV)

Init initializes the posting lists package, the in memory and dirty list hash.

func NewPosting 

func NewPosting(t *protos.DirectedEdge) *protos.Posting

func RebuildCountIndex 

func RebuildCountIndex(ctx context.Context, attr string) error

func RebuildIndex 

func RebuildIndex(ctx context.Context, attr string) error

RebuildIndex rebuilds index for a given attribute.

func RebuildReverseEdges 

func RebuildReverseEdges(ctx context.Context, attr string) error

RebuildReverseEdges rebuilds the reverse edges for a given attribute.

func SyncMarks 

func SyncMarks() *x.WaterMark

func TypeID 

func TypeID(edge *protos.DirectedEdge) types.TypeID

TypeID returns the typeid of destination vertex

func UnmarshalOrCopy 

func UnmarshalOrCopy(val []byte, metadata byte, pl *protos.PostingList)

Copies the val if it's uid only posting, be careful

Types

type ByUid 

type ByUid []*protos.Posting

func (ByUid) Len 

func (pa ByUid) Len() int

func (ByUid) Less 

func (pa ByUid) Less(i, j int) bool

func (ByUid) Swap 

func (pa ByUid) Swap(i, j int)

type CacheStats 

type CacheStats struct {
	Length    int
	Size      uint64
	NumEvicts uint64
}

type List 

type List struct {
	x.SafeMutex
	// contains filtered or unexported fields
}

func Get 

func Get(key []byte) (rlist *List)

Get stores the List corresponding to key, if it's not there already. to lru cache and returns it.

plist := Get(key, group) ... // Use plist TODO: This should take a node id and index. And just append all indices to a list. When doing a commit, it should update all the sync index watermarks. worker pkg would push the indices to the watermarks held by lists. And watermark stuff would have to be located outside worker pkg, maybe in x. That way, we don't have a dependency conflict.

func GetNoStore 

func GetNoStore(key []byte) (rlist *List)

GetNoStore takes a key. It checks if the in-memory map has an updated value and returns it if it exists or it gets from the store and DOES NOT ADD to lru cache.

func (*List) AddMutation 

func (l *List) AddMutation(ctx context.Context, t *protos.DirectedEdge) (bool, error)

AddMutation adds mutation to mutation layers. Note that it does not write anything to disk. Some other background routine will be responsible for merging changes in mutation layers to RocksDB. Returns whether any mutation happens.

func (*List) AddMutationWithIndex 

func (l *List) AddMutationWithIndex(ctx context.Context, t *protos.DirectedEdge) error

AddMutationWithIndex is AddMutation with support for indexing. It also supports reverse edges.

func (*List) AllValues 

func (l *List) AllValues() (vals []types.Val, rerr error)

func (*List) EstimatedSize 

func (l *List) EstimatedSize() uint32

func (*List) Facets 

func (l *List) Facets(param *protos.Param, langs []string) (fs []*protos.Facet,
	ferr error)

Facets gives facets for the posting representing value.

func (*List) Iterate 

func (l *List) Iterate(afterUid uint64, f func(obj *protos.Posting) bool)

Iterate will allow you to iterate over this Posting List, while having acquired a read lock. So, please keep this iteration cheap, otherwise mutations would get stuck. The iteration will start after the provided UID. The results would not include this UID. The function will loop until either the Posting List is fully iterated, or you return a false in the provided function, which will indicate to the function to break out of the iteration. 

	pl.Iterate(func(p *protos.Posting) bool {
   // Use posting p
   return true  // to continue iteration.
   return false // to break iteration.
 })

func (*List) LastCompactionTs 

func (l *List) LastCompactionTs() time.Time

func (*List) Length 

func (l *List) Length(afterUid uint64) int

Length iterates over the mutation layer and counts number of elements.

func (*List) Postings 

func (l *List) Postings(opt ListOptions, postFn func(*protos.Posting) bool)

Postings calls postFn with the postings that are common with uids in the opt ListOptions.

func (*List) SetForDeletion 

func (l *List) SetForDeletion()

SetForDeletion will mark this List to be deleted, so no more mutations can be applied to this.

func (*List) SyncIfDirty 

func (l *List) SyncIfDirty(delFromCache bool) (committed bool, err error)

func (*List) Uids 

func (l *List) Uids(opt ListOptions) *protos.List

Uids returns the UIDs given some query params. We have to apply the filtering before applying (offset, count). WARNING: Calling this function just to get Uids is expensive

func (*List) Value 

func (l *List) Value() (rval types.Val, rerr error)

Returns Value from posting list. This function looks only for "default" value (one without language).

func (*List) ValueFor 

func (l *List) ValueFor(langs []string) (rval types.Val, rerr error)

Returns Value from posting list, according to preferred language list (langs). If list is empty, value without language is returned; if such value is not available, value with smallest Uid is returned. If list consists of one or more languages, first available value is returned; if no language from list match the values, processing is the same as for empty list.

func (*List) ValueForTag 

func (l *List) ValueForTag(tag string) (rval types.Val, rerr error)

type ListOptions 

type ListOptions struct {
	AfterUID  uint64       // Any UID returned must be after this value.
	Intersect *protos.List // Intersect results with this list of UIDs.
}

ListOptions is used in List.Uids (in posting) to customize our output list of UIDs, for each posting list. It should be internal to this package.

type Options 

type Options struct {
	Mu             sync.Mutex
	AllottedMemory float64

	CommitFraction float64
}
var Config Options

type PIterator 

type PIterator struct {
	// contains filtered or unexported fields
}

func (*PIterator) Init 

func (it *PIterator) Init(pl *protos.PostingList, afterUid uint64)

func (*PIterator) Next 

func (it *PIterator) Next()

func (*PIterator) Posting 

func (it *PIterator) Posting() *protos.Posting

func (*PIterator) Valid 

func (it *PIterator) Valid() bool

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.