qri: github.com/qri-io/qri/repo Index | Files | Directories

package repo

import "github.com/qri-io/qri/repo"

Package repo represents a repository of qri information Analogous to a git repository, repo expects a rigid structure filled with rich types specific to qri.


Package Files

mem_refstore.go mem_repo.go parse.go refstore.go repo.go serialize.go


var (

    // DefaultQriLocation is where qri data defaults to storing. The keyword $HOME
    // (and only $HOME) will be replaced with the current user home directory
    DefaultQriLocation = "$HOME/.qri"

    // ErrNotFound is the err implementers should return when stuff isn't found
    ErrNotFound = fmt.Errorf("repo: not found")
    // ErrNoHistory is the err implementers should return when no versions exist in history
    ErrNoHistory = fmt.Errorf("repo: no history")
    // ErrPeerIDRequired is for when a peerID is missing-but-expected
    ErrPeerIDRequired = fmt.Errorf("repo: peerID is required")
    // ErrPeernameRequired is for when a peername is missing-but-expected
    ErrPeernameRequired = fmt.Errorf("repo: peername is required")
    // ErrNoRepo is for when a repo is missing-but-expected
    ErrNoRepo = fmt.Errorf("repo: no repo found")
    // ErrNameRequired is for when a name is missing-but-expected
    ErrNameRequired = fmt.Errorf("repo: name is required")
    // ErrPathRequired is for when a path is missing-but-expected
    ErrPathRequired = fmt.Errorf("repo: path is required")
    // ErrNameTaken is for when a name name is already taken
    ErrNameTaken = fmt.Errorf("repo: name already in use")
    // ErrRepoEmpty is for when the repo has no datasets
    ErrRepoEmpty = fmt.Errorf("repo: this repo contains no datasets")
    // ErrNotPinner is for when the repo doesn't have the concept of pinning as a feature
    ErrNotPinner = fmt.Errorf("repo: backing store doesn't support pinning")
    // ErrNoRegistry indicates no regsitry is currently configured
    ErrNoRegistry = fmt.Errorf("no configured registry")
    // ErrEmptyRef indicates that the given reference is empty
    ErrEmptyRef = fmt.Errorf("repo: empty dataset reference")

func CanonicalizeDatasetRef Uses

func CanonicalizeDatasetRef(r Repo, ref *reporef.DatasetRef) error

CanonicalizeDatasetRef uses the user's repo to turn any local aliases into full dataset references using known canonical peernames and paths. If the provided reference is not in the local repo, still do the work of handling aliases, but return a repo.ErrNotFound error, which callers can respond to by possibly contacting remote repos.

func CanonicalizeProfile Uses

func CanonicalizeProfile(r Repo, ref *reporef.DatasetRef) error

CanonicalizeProfile populates dataset reporef.DatasetRef ProfileID and Peername properties, changing aliases to known names, and adding ProfileID from a peerstore

func CompareDatasetRef Uses

func CompareDatasetRef(a, b reporef.DatasetRef) error

CompareDatasetRef compares two Dataset References, returning an error describing any difference between the two references

func DeleteVersionInfoShim Uses

func DeleteVersionInfoShim(r Repo, ref dsref.Ref) (*dsref.VersionInfo, error)

DeleteVersionInfoShim is a shim for getting away from the old stack of DatasetRef, CanonicalizeDatasetRef, and RefStore while still safely interacting with the repo.Refstore API

func FlatbufferBytes Uses

func FlatbufferBytes(rs RefList) []byte

FlatbufferBytes turns refs into a byte slice of flatbuffer data

func GetVersionInfoShim Uses

func GetVersionInfoShim(r Repo, ref dsref.Ref) (*dsref.VersionInfo, error)

GetVersionInfoShim is a shim for getting away from the old DatasetRef, CanonicalizeDatasetRef, RefStore stack

func ListDatasetsShim Uses

func ListDatasetsShim(r Repo, offset, limit int) ([]dsref.VersionInfo, error)

ListDatasetsShim gets away from the old stack of DatasetRef, CanonicalizeDatasetRef and RefStore

func MarshalFlatbuffer Uses

func MarshalFlatbuffer(r reporef.DatasetRef, builder *flatbuffers.Builder) flatbuffers.UOffsetT

MarshalFlatbuffer writes a ref to a builder

func ParseDatasetRef Uses

func ParseDatasetRef(ref string) (reporef.DatasetRef, error)

ParseDatasetRef decodes a dataset reference from a string value It’s possible to refer to a dataset in a number of ways. The full definition of a dataset reference is as follows:

dataset_reference = peer_name/dataset_name@peer_id/network/hash

we swap in defaults as follows, all of which are represented as empty strings:

network - defaults to /ipfs/
hash - tip of version history (latest known commit)

these defaults are currently enforced by convention. TODO - make Dataset Ref parsing the responisiblity of the Repo interface, replacing empty strings with actual defaults

dataset names & hashes are disambiguated by checking if the input parses to a valid multihash after base58 decoding. through defaults & base58 checking the following should all parse:


see tests for more exmples

TODO - add validation that prevents peernames from being valid base58 multihashes and makes sure hashes are actually valid base58 multihashes TODO - figure out how IPFS CID's play into this

func PutVersionInfoShim Uses

func PutVersionInfoShim(r Repo, vi *dsref.VersionInfo) error

PutVersionInfoShim is a shim for getting away from old stack of DatasetRef, CanonicalizeDatasetRef, and RefStores while still safely interacting with the repo.Refstore API

func UnmarshalFlatbuffer Uses

func UnmarshalFlatbuffer(rfb *repofb.DatasetRef) (r reporef.DatasetRef, err error)

UnmarshalFlatbuffer decodes a job from a flatbuffer

type MemRefstore Uses

type MemRefstore []reporef.DatasetRef

MemRefstore is an in-memory implementation of the Namestore interface

func (*MemRefstore) DeleteRef Uses

func (r *MemRefstore) DeleteRef(del reporef.DatasetRef) error

DeleteRef removes a name from the store

func (MemRefstore) GetRef Uses

func (r MemRefstore) GetRef(get reporef.DatasetRef) (ref reporef.DatasetRef, err error)

GetRef completes a reference with , refs can have either Path or Peername & Name specified, GetRef should fill out the missing pieces

func (*MemRefstore) PutRef Uses

func (r *MemRefstore) PutRef(put reporef.DatasetRef) error

PutRef adds a reference to the namestore. Only complete references may be added

func (MemRefstore) RefCount Uses

func (r MemRefstore) RefCount() (int, error)

RefCount returns the total number of names in the store

func (MemRefstore) References Uses

func (r MemRefstore) References(offset, limit int) ([]reporef.DatasetRef, error)

References grabs a set of names from the Store's namespace

type MemRepo Uses

type MemRepo struct {
    // contains filtered or unexported fields

MemRepo is an in-memory implementation of the Repo interface

func NewMemRepo Uses

func NewMemRepo(ctx context.Context, p *profile.Profile, fs *muxfs.Mux, bus event.Bus) (*MemRepo, error)

NewMemRepo creates a new in-memory repository TODO (b5) - this constructor should have an event.bus argument

func (*MemRepo) Done Uses

func (r *MemRepo) Done() <-chan struct{}

Done returns a channel that the repo will send on when the repo is closed

func (*MemRepo) DoneErr Uses

func (r *MemRepo) DoneErr() error

DoneErr gives an error that occured during the shutdown process

func (*MemRepo) Dscache Uses

func (r *MemRepo) Dscache() *dscache.Dscache

Dscache returns a dscache

func (*MemRepo) Filesystem Uses

func (r *MemRepo) Filesystem() *muxfs.Mux

Filesystem gives access to the underlying filesystem

func (*MemRepo) Logbook Uses

func (r *MemRepo) Logbook() *logbook.Book

Logbook accesses the mem repo logbook

func (*MemRepo) PrivateKey Uses

func (r *MemRepo) PrivateKey() crypto.PrivKey

PrivateKey returns this repo's private key

func (*MemRepo) Profile Uses

func (r *MemRepo) Profile() (*profile.Profile, error)

Profile returns the peer profile for this repository

func (*MemRepo) Profiles Uses

func (r *MemRepo) Profiles() profile.Store

Profiles gives this repo's Peer interface implementation

func (*MemRepo) RefCache Uses

func (r *MemRepo) RefCache() Refstore

RefCache gives access to the ephemeral Refstore

func (*MemRepo) RemoveLogbook Uses

func (r *MemRepo) RemoveLogbook()

RemoveLogbook drops a MemRepo's logbook pointer. MemRepo gets used in tests a bunch, where logbook manipulation is helpful

func (*MemRepo) ResolveRef Uses

func (r *MemRepo) ResolveRef(ctx context.Context, ref *dsref.Ref) (string, error)

ResolveRef implements the dsref.Resolver interface

func (*MemRepo) SetFilesystem Uses

func (r *MemRepo) SetFilesystem(fs *muxfs.Mux)

SetFilesystem implements QFSSetter, currently used during lib contstruction

func (*MemRepo) SetLogbook Uses

func (r *MemRepo) SetLogbook(book *logbook.Book)

SetLogbook assigns MemRepo's logbook. MemRepo gets used in tests a bunch, where logbook manipulation is helpful

func (*MemRepo) SetProfile Uses

func (r *MemRepo) SetProfile(p *profile.Profile) error

SetProfile updates this repo's profile

func (*MemRepo) Store Uses

func (r *MemRepo) Store() cafs.Filestore

Store returns the underlying cafs.Filestore for this repo

type QFSSetter Uses

type QFSSetter interface {

QFSSetter sets a qfs.Filesystem the whole interface is a short-term hack that should only need to be called in one context: when lib is setting up a Repo TODO (b5): either decouple repo & qfs completely, or merge them

type RefList Uses

type RefList []reporef.DatasetRef

RefList is a list of refs

func UnmarshalRefsFlatbuffer Uses

func UnmarshalRefsFlatbuffer(data []byte) (ls RefList, err error)

UnmarshalRefsFlatbuffer turns a repo flatbuffer into a list of refs

func (RefList) Len Uses

func (rs RefList) Len() int

Len returns the length of refs. Used for sort.Interface

func (RefList) Less Uses

func (rs RefList) Less(i, j int) bool

Less returns true if i comes before j. Used for sort.Interface

func (RefList) Swap Uses

func (rs RefList) Swap(i, j int)

Swap flips the positions of i and j. Used for sort.Interface

type Refstore Uses

type Refstore interface {
    // PutRef adds a reference to the store. References must be complete with
    // Peername, Name, and Path/FSIPath specified
    PutRef(ref reporef.DatasetRef) error
    // GetRef "completes" a passed in alias (reporef.DatasetRef with at least Peername
    // and Name field specified), filling in missing fields with a stored ref
    GetRef(ref reporef.DatasetRef) (reporef.DatasetRef, error)
    // DeleteRef removes a reference from the store
    DeleteRef(ref reporef.DatasetRef) error
    // References returns a set of references from the store
    References(offset, limit int) ([]reporef.DatasetRef, error)
    // RefCount returns the number of references in the store
    RefCount() (int, error)

Refstore keeps a collection of dataset references, Refstores require complete references (with both alias and identifiers), and can carry only one of a given alias eg: putting peer/dataset@a/ipfs/b when a ref with alias peer/dataset is already in the store will overwrite the stored reference

type Repo Uses

type Repo interface {
    // All repositories wraps a content-addressed filestore as the cannonical
    // record of this repository's data. Store gives direct access to the
    // cafs.Filestore instance any given repo is using.
    // TODO (b5) - deprecate store in favour of the Filesystem interface
    Store() cafs.Filestore

    // Filesystem is currently a read-only source of Filesystem-like data
    // Filestores can multiplex to read from multiple sources like the local
    // filesystem, over http, or content-addressed filesystems.
    // the long term-plan is to merge Filestore & Store
    Filesystem() *muxfs.Mux

    // A Repo can resolve dataset references it knows about locally from its
    // Refstore
    // NOTE (b5): this implementation will be dropped when the Refstore interface
    // is removed from the repo, delegating local ref resolution to dscache
    // All Repos must keep a Refstore, defining a store of known datasets
    // NOTE(dlong): Refstore is going away soon, everything is going to move to Dscache
    // Dscache is a cache of datasets that have been built according to logbook
    Dscache() *dscache.Dscache

    // Repos have a logbook for recording & storing operation logs
    Logbook() *logbook.Book

    // A repository must maintain profile information about the owner of this dataset.
    // The value returned by Profile() should represent the peer.
    Profile() (*profile.Profile, error)
    // SetProfile sets this repo's current profile. Profiles must contain a private key
    SetProfile(*profile.Profile) error
    // PrivateKey hands over this repo's private key
    PrivateKey() crypto.PrivKey
    // A repository must maintain profile information about encountered peers.
    // Decsisions regarding retentaion of peers is left to the the implementation
    Profiles() profile.Store

    // Done returns a channel that the repo will send on when the repo is closed
    Done() <-chan struct{}
    // DoneErr gives any error that occured in the shutdown process
    DoneErr() error

Repo is the interface for working with a qri repository qri repos are stored graph of resources:datasets, known peers, analytics data, change requests, etc. Repos are connected to a single peer profile. Repos must wrap an underlying cafs.Filestore, which is intended to act as the canonical store of state across all peers that this repo may interact with.

type SearchParams Uses

type SearchParams struct {
    Q             string
    Limit, Offset int

SearchParams encapsulates parameters provided to Searchable.Search

type Searchable Uses

type Searchable interface {
    Search(p SearchParams) ([]reporef.DatasetRef, error)

Searchable is an opt-in interface for supporting repository search


buildrepoPackage buildrepo initializes a qri repo
fsPackage fsrepo is a file-system implementation of repo
genPackage gen contains routines that perform expensive cryptographic operations.
profilePackage profile defines a qri peer profile
test/specPackage spec contains a set of tests to ensure a repo implementation conforms to expected behaviors, calling RunRepoTests on a given repo implementation should pass all checks in order to properly work with Qri.

Package repo imports 20 packages (graph) and is imported by 20 packages. Updated 2020-07-10. Refresh now. Tools for package owners.