repository

package
v1.0.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 15, 2024 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Index

Constants

View Source
const STATS_KEY = "REPO-STATS"

accessor for stored object stats in the repo itself

Variables

This section is empty.

Functions

This section is empty.

Types

type BadgerQueryFunc

type BadgerQueryFunc func(txn *badger.Txn) error

Allow definition of query at call-site by abstracting to function

type BadgerRepo

type BadgerRepo struct {
	// contains filtered or unexported fields
}

Data repositoryusing badger kv db engine

func NewBadgerRepo

func NewBadgerRepo(dbFolderName string) (*BadgerRepo, error)

create a new badger repo in the nominated directory.

func OpenExistingBadgerRepo

func OpenExistingBadgerRepo(dbFolderName string) (*BadgerRepo, error)

open a repo that's already got data in it

func (*BadgerRepo) Close

func (br *BadgerRepo) Close()

ensure all writes to disk are completed

func (*BadgerRepo) Commit

func (br *BadgerRepo) Commit()

on small data files write-batch is so fast it doesn't commit prior to reporting starting, this method can be used to force a flush of the current write-batch

func (*BadgerRepo) DB

func (br *BadgerRepo) DB() *badger.DB

access to underlying data store for iterator queries

func (*BadgerRepo) GetStats

func (br *BadgerRepo) GetStats() ObjectStats

returns the last reocrded set of ingest object counts for this repo

func (*BadgerRepo) SaveStats

func (br *BadgerRepo) SaveStats(s ObjectStats) error

persists a copy of the recorded object counts back into the repository, particularly used for sizing progress bars etc. when reports are run against the repo with no prior ingest phase.

func (*BadgerRepo) Store

func (br *BadgerRepo) Store(r sec.Result, idxf IndexFunc) error

saves data into the badger db, uses the indexfunc to create the lookup key for the data item

type IndexFunc

type IndexFunc func(r sec.Result) ([]byte, error)

signature for an indexing function to use on the data objects; takes in a conversion result, returns the key for that json blob as bytes.

func IdxByTypeSchoolAndTest

func IdxByTypeSchoolAndTest() IndexFunc

index object (e.g. score-summary) by student and test Creates an index key in the form ObjectName:SchoolInfoRefId:NAPTestRefId

func IdxByTypeStudentAndTest

func IdxByTypeStudentAndTest() IndexFunc

index object (e.g. result-set) by student and test Creates an index key in the form ObjectName:StudentPersonalRefId:NAPTestRefId

func IdxSifObjectByTypeAndRefId

func IdxSifObjectByTypeAndRefId() IndexFunc

index func to retrieve sif object refid. Creates an index key in the form ObjectName:RefId - only index needed for majority of the objects

type ObjectStats

type ObjectStats map[string]int

keep totals for objects added to db

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL