qri: github.com/qri-io/qri/repo Index | Files | Directories

package repo

import "github.com/qri-io/qri/repo"

Package repo represents a repository of qri information Analogous to a git repository, repo expects a rigid structure filled with rich types specific to qri.


Package Files

mem_refstore.go mem_repo.go ref.go refs.go repo.go


var (
    // DefaultQriLocation is where qri data defaults to storing. The keyword $HOME
    // (and only $HOME) will be replaced with the current user home directory
    DefaultQriLocation = "$HOME/.qri"

    // ErrNotFound is the err implementers should return when stuff isn't found
    ErrNotFound = fmt.Errorf("repo: not found")
    // ErrNoHistory is the err implementers should return when no versions exist in history
    ErrNoHistory = fmt.Errorf("repo: no history")
    // ErrPeerIDRequired is for when a peerID is missing-but-expected
    ErrPeerIDRequired = fmt.Errorf("repo: peerID is required")
    // ErrPeernameRequired is for when a peername is missing-but-expected
    ErrPeernameRequired = fmt.Errorf("repo: peername is required")
    // ErrNameRequired is for when a name is missing-but-expected
    ErrNameRequired = fmt.Errorf("repo: name is required")
    // ErrPathRequired is for when a path is missing-but-expected
    ErrPathRequired = fmt.Errorf("repo: path is required")
    // ErrNameTaken is for when a name name is already taken
    ErrNameTaken = fmt.Errorf("repo: name already in use")
    // ErrRepoEmpty is for when the repo has no datasets
    ErrRepoEmpty = fmt.Errorf("repo: this repo contains no datasets")
    // ErrNotPinner is for when the repo doesn't have the concept of pinning as a feature
    ErrNotPinner = fmt.Errorf("repo: backing store doesn't support pinning")
    // ErrNoRegistry indicates no regsitry is currently configured
    ErrNoRegistry = fmt.Errorf("no configured registry")
    // ErrEmptyRef indicates that the given reference is empty
    ErrEmptyRef = fmt.Errorf("repo: empty dataset reference")

func CanonicalizeDatasetRef Uses

func CanonicalizeDatasetRef(r Repo, ref *DatasetRef) error

CanonicalizeDatasetRef uses the user's repo to turn any local aliases into full dataset references using known canonical peernames and paths. If the provided reference is not in the local repo, still do the work of handling aliases, but return a repo.ErrNotFound error, which callers can respond to by possibly contacting remote repos.

func CanonicalizeProfile Uses

func CanonicalizeProfile(r Repo, ref *DatasetRef) error

CanonicalizeProfile populates dataset DatasetRef ProfileID and Peername properties, changing aliases to known names, and adding ProfileID from a peerstore

func CompareDatasetRef Uses

func CompareDatasetRef(a, b DatasetRef) error

CompareDatasetRef compares two Dataset References, returning an error describing any difference between the two references

func ConvertToDsref Uses

func ConvertToDsref(ref DatasetRef) dsref.Ref

ConvertToDsref is a shim function to transition from a DatasetRef to a dsref.Ref while we experiment with dsref as the home of name parsing

func IsRefString Uses

func IsRefString(path string) bool

IsRefString checks to see if a reference is a valid dataset ref string

type DatasetInfo Uses

type DatasetInfo struct {
    // Reference for this version
    Ref dsref.Ref `json:"ref,omitempty"`

    // FSIPath is this dataset's link to the local filesystem if one exists
    FSIPath string `json:"fsiPath,omitempty"`
    // Published indicates whether this reference is listed as an available dataset
    Published bool `json:"published"`
    // If true, this version doesn't exist locally
    Foreign bool `json:"foreign,omitempty"`

DatasetInfo describes info aboud a dataset version in a repository

func ConvertToDsinfo Uses

func ConvertToDsinfo(ref DatasetRef) DatasetInfo

ConvertToDsinfo is a shim function to transition from a DatasetRef to Info

type DatasetRef Uses

type DatasetRef struct {
    // Peername of dataset owner
    Peername string `json:"peername,omitempty"`
    // ProfileID of dataset owner
    ProfileID profile.ID `json:"profileID,omitempty"`
    // Unique name reference for this dataset
    Name string `json:"name,omitempty"`
    // Content-addressed path for this dataset
    Path string `json:"path,omitempty"`
    // FSIPath is this dataset's link to the local filesystem if one exists
    FSIPath string `json:"fsiPath,omitempty"`
    // Dataset is a pointer to the dataset being referenced
    Dataset *dataset.Dataset `json:"dataset,omitempty"`
    // Published indicates whether this reference is listed as an available dataset
    Published bool `json:"published"`
    // If true, this reference doesn't exist locally
    Foreign bool `json:"foreign,omitempty"`

DatasetRef encapsulates a reference to a dataset. This needs to exist to bind ways of referring to a dataset to a dataset itself, as datasets can't easily contain their own hash information, and names are unique on a per-repository basis.

Deprecated: DatasetRef will be removed in future versions of Qri, use dsref.Ref and this package's Info struct instead

func MustParseDatasetRef Uses

func MustParseDatasetRef(refstr string) DatasetRef

MustParseDatasetRef panics if the reference is invalid. Useful for testing

func ParseDatasetRef Uses

func ParseDatasetRef(ref string) (DatasetRef, error)

ParseDatasetRef decodes a dataset reference from a string value It’s possible to refer to a dataset in a number of ways. The full definition of a dataset reference is as follows:

dataset_reference = peer_name/dataset_name@peer_id/network/hash

we swap in defaults as follows, all of which are represented as empty strings:

network - defaults to /ipfs/
hash - tip of version history (latest known commit)

these defaults are currently enforced by convention. TODO - make Dataset Ref parsing the responisiblity of the Repo interface, replacing empty strings with actual defaults

dataset names & hashes are disambiguated by checking if the input parses to a valid multihash after base58 decoding. through defaults & base58 checking the following should all parse:


see tests for more exmples

TODO - add validation that prevents peernames from being valid base58 multihashes and makes sure hashes are actually valid base58 multihashes TODO - figure out how IPFS CID's play into this

func ParseDsref Uses

func ParseDsref(r dsref.Ref) (DatasetRef, error)

ParseDsref is a shim to convert dsrefs back to repo.DatasetRef

func (DatasetRef) Absolute Uses

func (r DatasetRef) Absolute() (s string)

Absolute implements the same thing as String(), but append ProfileID if it exist

func (DatasetRef) AliasString Uses

func (r DatasetRef) AliasString() (s string)

AliasString returns the alias components of a DatasetRef as a string

func (DatasetRef) Complete Uses

func (r DatasetRef) Complete() bool

Complete returns true if a dataset has Peername, Name, ProfileID and Path properties set

func (DatasetRef) Equal Uses

func (r DatasetRef) Equal(b DatasetRef) bool

Equal returns true only if Peername Name and Path are equal

func (DatasetRef) FlatbufferBytes Uses

func (r DatasetRef) FlatbufferBytes() []byte

FlatbufferBytes formats a ref as a flatbuffer byte slice

func (DatasetRef) IsEmpty Uses

func (r DatasetRef) IsEmpty() bool

IsEmpty returns true if none of it's fields are set

func (DatasetRef) IsPeerRef Uses

func (r DatasetRef) IsPeerRef() bool

IsPeerRef returns true if only Peername is set

func (DatasetRef) MarshalFlatbuffer Uses

func (r DatasetRef) MarshalFlatbuffer(builder *flatbuffers.Builder) flatbuffers.UOffsetT

MarshalFlatbuffer writes a ref to a builder

func (DatasetRef) Match Uses

func (r DatasetRef) Match(b DatasetRef) bool

Match checks returns true if Peername and Name are equal, and/or path is equal

func (DatasetRef) String Uses

func (r DatasetRef) String() (s string)

String implements the Stringer interface for DatasetRef

func (*DatasetRef) UnmarshalFlatbuffer Uses

func (r *DatasetRef) UnmarshalFlatbuffer(rfb *repofb.DatasetRef) (err error)

UnmarshalFlatbuffer decodes a job from a flatbuffer

type MemRefstore Uses

type MemRefstore []DatasetRef

MemRefstore is an in-memory implementation of the Namestore interface

func (*MemRefstore) DeleteRef Uses

func (r *MemRefstore) DeleteRef(del DatasetRef) error

DeleteRef removes a name from the store

func (MemRefstore) GetRef Uses

func (r MemRefstore) GetRef(get DatasetRef) (ref DatasetRef, err error)

GetRef completes a reference with , refs can have either Path or Peername & Name specified, GetRef should fill out the missing pieces

func (*MemRefstore) PutRef Uses

func (r *MemRefstore) PutRef(put DatasetRef) error

PutRef adds a reference to the namestore. Only complete references may be added

func (MemRefstore) RefCount Uses

func (r MemRefstore) RefCount() (int, error)

RefCount returns the total number of names in the store

func (MemRefstore) References Uses

func (r MemRefstore) References(offset, limit int) ([]DatasetRef, error)

References grabs a set of names from the Store's namespace

type MemRepo Uses

type MemRepo struct {
    // contains filtered or unexported fields

MemRepo is an in-memory implementation of the Repo interface

func NewMemRepo Uses

func NewMemRepo(p *profile.Profile, store cafs.Filestore, fsys qfs.Filesystem, ps profile.Store) (*MemRepo, error)

NewMemRepo creates a new in-memory repository TODO (b5) - need a better mem-repo constructor, we don't need a logbook for all test cases

func (*MemRepo) Filesystem Uses

func (r *MemRepo) Filesystem() qfs.Filesystem

Filesystem gives access to the underlying filesystem

func (*MemRepo) Logbook Uses

func (r *MemRepo) Logbook() *logbook.Book

Logbook accesses the mem repo logbook

func (*MemRepo) PrivateKey Uses

func (r *MemRepo) PrivateKey() crypto.PrivKey

PrivateKey returns this repo's private key

func (*MemRepo) Profile Uses

func (r *MemRepo) Profile() (*profile.Profile, error)

Profile returns the peer profile for this repository

func (*MemRepo) Profiles Uses

func (r *MemRepo) Profiles() profile.Store

Profiles gives this repo's Peer interface implementation

func (*MemRepo) RefCache Uses

func (r *MemRepo) RefCache() Refstore

RefCache gives access to the ephemeral Refstore

func (*MemRepo) RemoveLogbook Uses

func (r *MemRepo) RemoveLogbook()

RemoveLogbook drops a MemRepo's logbook pointer. MemRepo gets used in tests a bunch, where logbook manipulation is helpful

func (*MemRepo) SetFilesystem Uses

func (r *MemRepo) SetFilesystem(fs qfs.Filesystem)

SetFilesystem implements QFSSetter, currently used during lib contstruction

func (*MemRepo) SetLogbook Uses

func (r *MemRepo) SetLogbook(book *logbook.Book)

SetLogbook assigns MemRepo's logbook. MemRepo gets used in tests a bunch, where logbook manipulation is helpful

func (*MemRepo) SetProfile Uses

func (r *MemRepo) SetProfile(p *profile.Profile) error

SetProfile updates this repo's profile

func (*MemRepo) Store Uses

func (r *MemRepo) Store() cafs.Filestore

Store returns the underlying cafs.Filestore for this repo

type QFSSetter Uses

type QFSSetter interface {

QFSSetter sets a qfs.Filesystem the whole interface is a short-term hack that should only need to be called in one context: when lib is setting up a Repo TODO (b5): either decouple repo & qfs completely, or merge them

type Refs Uses

type Refs []DatasetRef

Refs is a list of refs

func UnmarshalRefsFlatbuffer Uses

func UnmarshalRefsFlatbuffer(data []byte) (ls Refs, err error)

UnmarshalRefsFlatbuffer turns a repo flatbuffer into a list of refs

func (Refs) FlatbufferBytes Uses

func (rs Refs) FlatbufferBytes() []byte

FlatbufferBytes turns refs into a byte slice of flatbuffer data

func (Refs) Len Uses

func (rs Refs) Len() int

Len returns the length of refs

func (Refs) Less Uses

func (rs Refs) Less(i, j int) bool

Less returns true if i comes before j

func (Refs) Remove Uses

func (rs Refs) Remove(i int) Refs

Remove deletes an entry from the list of refs at an index

func (Refs) Swap Uses

func (rs Refs) Swap(i, j int)

Swap flips the positions of i and j

type Refstore Uses

type Refstore interface {
    // PutRef adds a reference to the store. References must be complete with
    // Peername, Name, and Path specified
    PutRef(ref DatasetRef) error
    // GetRef "completes" a passed in alias (DatasetRef with at least Peername
    // and Name field specified), filling in missing fields with a stored ref
    // TODO - should we rename this to "CompleteRef"?
    GetRef(ref DatasetRef) (DatasetRef, error)
    // DeleteRef removes a reference from the store
    DeleteRef(ref DatasetRef) error
    // References returns a set of references from the store
    References(offset, limit int) ([]DatasetRef, error)
    // RefCount returns the number of references in the store
    RefCount() (int, error)

Refstore keeps a collection of dataset references, Refstores require complete references (with both alias and identifiers), and can carry only one of a given alias eg: putting peer/dataset@a/ipfs/b when a ref with alias peer/dataset is already in the store will overwrite the stored reference

type Repo Uses

type Repo interface {
    // All repositories wraps a content-addressed filestore as the cannonical
    // record of this repository's data. Store gives direct access to the
    // cafs.Filestore instance any given repo is using.
    Store() cafs.Filestore

    // Filesystem is currently a read-only source of Filesystem-like data
    // Filestores can multiplex to read from multiple sources like the local
    // filesystem, over http, or content-addressed filesystems.
    // the long term-plan is to merge Filestore & Store
    Filesystem() qfs.Filesystem

    // All Repos must keep a Refstore, defining a store of known datasets

    // Repos have a logbook for recording & storing operation logs
    Logbook() *logbook.Book

    // A repository must maintain profile information about the owner of this dataset.
    // The value returned by Profile() should represent the peer.
    Profile() (*profile.Profile, error)
    // SetProfile sets this repo's current profile. Profiles must contain a private key
    SetProfile(*profile.Profile) error
    // PrivateKey hands over this repo's private key
    PrivateKey() crypto.PrivKey
    // A repository must maintain profile information about encountered peers.
    // Decsisions regarding retentaion of peers is left to the the implementation
    Profiles() profile.Store

Repo is the interface for working with a qri repository qri repos are stored graph of resources:datasets, known peers, analytics data, change requests, etc. Repos are connected to a single peer profile. Repos must wrap an underlying cafs.Filestore, which is intended to act as the canonical store of state across all peers that this repo may interact with.

type SearchParams Uses

type SearchParams struct {
    Q             string
    Limit, Offset int

SearchParams encapsulates parameters provided to Searchable.Search

type Searchable Uses

type Searchable interface {
    Search(p SearchParams) ([]DatasetRef, error)

Searchable is an opt-in interface for supporting repository search


fsPackage fsrepo is a file-system implementation of repo
genPackage gen contains routines that perform expensive cryptographic operations.
profilePackage profile defines a qri peer profile
testPackage test contains a set of tests to ensure a repo implementation conforms to expected behaviors, calling RunRepoTests on a given repo implementation should pass all checks in order to properly work with Qri.

Package repo imports 16 packages (graph) and is imported by 14 packages. Updated 2019-11-02. Refresh now. Tools for package owners.