borges

package module
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 9, 2019 License: Apache-2.0 Imports: 7 Imported by: 6

README

GoDoc Build Status codecov.io Go Report Card

go-borges

This library abstracts read and write access to a set of go-git repositories. It comes with several implementations to support different storage methods:

  • plain: stored in the filesystem, supports transactions.
  • siva: rooted repositories in siva files, supports transactions. These files can be generated with gitcollector.
  • legacysiva: siva file generated by borges. This implementation only supports reading and does not support transactions.

When transactions are supported the writes to the repositories will be atomic and could only be seen by new readers when Commit function is called. That is, after opening a repository in read only mode any writes to it by another thread or process won't modify its contents. This is useful when the storage that is being used for reading repositories is being updated at the same time. More information and example in siva package documentation.

Installation

go-borges supports go modules and can be added to your project with:

$ go get github.com/src-d/go-borges

Example of utilization

This example lists the repositories downloaded by gitcollector.

package main

import (
	"fmt"
	"os"

	"github.com/src-d/go-borges"
	"github.com/src-d/go-borges/siva"
	"gopkg.in/src-d/go-billy.v4/osfs"
)

func main() {
	if len(os.Args) != 2 {
		fmt.Println("you need to provide the path of your siva files")
		os.Exit(1)
	}
	fs := osfs.New(os.Args[1])

	lib, err := siva.NewLibrary("library", fs, &siva.LibraryOptions{
		Bucket:        2,
		RootedRepo:    true,
		Transactional: true,
	})
	if err != nil {
		panic(err)
	}

	repos, err := lib.Repositories(borges.ReadOnlyMode)
	if err != nil {
		panic(err)
	}

	err = repos.ForEach(func(r borges.Repository) error {
		id := r.ID().String()
		head, err := r.R().Head()
		if err != nil {
			return err
		}

		fmt.Printf("repository: %v, HEAD: %v\n", id, head.Hash().String())
		return nil
	})
}

Contribute

Contributions are more than welcome, if you are interested please take a look to our Contributing Guidelines.

Code of Conduct

All activities under source{d} projects are governed by the source{d} code of conduct.

License

Apache License Version 2.0, see LICENSE.

Documentation

Overview

This library abstracts read and write access to a set of go-git repositories. It comes with several implementations to support different storage methods:

* plain: stored in the filesystem, supports transactions.

* siva: rooted repositories in siva files, supports transactions. These files can be generated with gitcollector.

* legacysiva: siva file generated by borges. This implementation only supports reading and does not support transactions.

This example lists the repositories downloaded by gitcollector:

package main

import (
	"fmt"
	"os"

	"github.com/src-d/go-borges"
	"github.com/src-d/go-borges/siva"
	"gopkg.in/src-d/go-billy.v4/osfs"
)

func main() {
	if len(os.Args) != 2 {
		fmt.Println("you need to provide the path of your siva files")
		os.Exit(1)
	}
	fs := osfs.New(os.Args[1])

	lib, err := siva.NewLibrary("library", fs, &siva.LibraryOptions{
		Bucket:        2,
		RootedRepo:    true,
		Transactional: true,
	})
	if err != nil {
		panic(err)
	}

	repos, err := lib.Repositories(borges.ReadOnlyMode)
	if err != nil {
		panic(err)
	}

	err = repos.ForEach(func(r borges.Repository) error {
		id := r.ID().String()
		head, err := r.R().Head()
		if err != nil {
			return err
		}

		fmt.Printf("repository: %v, HEAD: %v\n", id, head.Hash().String())
		return nil
	})
}

More information:

* go-git: https://github.com/src-d/go-git

* rooted repositories: https://github.com/src-d/gitcollector#storing-repositories-using-rooted-repositories

* siva files: https://github.com/src-d/go-siva

* gitcollector: https://github.com/src-d/gitcollector

* borges: https://github.com/src-d/borges

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrNotImplemented is returned by method of any implementation that are
	// not implemented on this specific implementation.
	ErrNotImplemented = errors.NewKind("not implemented")
	// ErrModeNotSupported is returned in the case of a request to open a
	// repository with a Mode not supported.
	ErrModeNotSupported = errors.NewKind("repository mode %q not supported")
	// ErrLocationNotExists when a Location is requested and can't be found.
	ErrLocationNotExists = errors.NewKind("location %s not exists")
	// ErrLibraryNotExists when a Library is requested and can't be found.
	ErrLibraryNotExists = errors.NewKind("library %s not exists")
	// ErrRepositoryExists an error returned on a request of Init on a location
	// with a repository with this RepositoryID already exists.
	ErrRepositoryExists = errors.NewKind("repository %s already exists")
	// ErrRepositoryNotExists when a Repository is requested and can't be found.
	ErrRepositoryNotExists = errors.NewKind("repository %s not exists")
	// ErrNonTransactional returned when Repository.Commit is called on a
	// repository that not support transactions.
	ErrNonTransactional = errors.NewKind("non transactional repository")
)
View Source
var (
	//ErrStop is used to stop a ForEach function in an Iter
	ErrStop = errors.New("stop iter")
)

Functions

This section is empty.

Types

type Library

type Library interface {
	// ID returns the LibraryID for this Library.
	ID() LibraryID
	// Init initializes a new Repository in a Location, the chosen Location
	// is dependant on the implementation, if this this not supported should
	// return ErrNotImplemented. If a repository with the given RepositoryID
	// already exists ErrRepositoryExists is returned.
	Init(RepositoryID) (Repository, error)
	// Get open a repository with the given RepositoryID, it itereates all the
	// library locations until this repository is found. If a repository with
	// the given RepositoryID can't be found the ErrRepositoryNotExists is
	// returned.
	Get(RepositoryID, Mode) (Repository, error)
	// GetOrInit open or initilizes a Repository at a Location, if this this not
	// supported should return ErrNotImplemented. If the repository is opened
	// this will be done in RWMode.
	GetOrInit(RepositoryID) (Repository, error)
	// Has returns true, the LibraryID and the LocationID if the given
	// RepositoryID matches any repository at any location belonging to this
	// Library.
	Has(RepositoryID) (bool, LibraryID, LocationID, error)
	// Repositories returns a RepositoryIterator that iterates through all
	// the repositories contained in all Location contained in this Library.
	Repositories(Mode) (RepositoryIterator, error)
	// Location returns the Location with the given LocationID, if a location
	// can't be found ErrLocationNotExists is returned.
	Location(LocationID) (Location, error)
	// Locations returns a LocationIterator that iterates through all locations
	// contained in this Library.
	Locations() (LocationIterator, error)
}

Library interface represents a group of different libraries and locations, it allows access to any repository stored on any library or location. Also allows the iteration of the libraries and locations to perform full scan operations. Library is the default entrypoint for accessing the repositories, should be used when the Location is not important.

type LibraryID

type LibraryID string

LibraryID represents a Library identifier.

type LibraryIterator

type LibraryIterator interface {
	// Next returns the next library from the iterator. If the iterator has
	// reached the end it will return io.EOF as an error.
	Next() (Library, error)
	// ForEach call the function for each object contained on this iter until
	// an error happens or the end of the iter is reached. If ErrStop is sent
	// the iteration is stop but no error is returned. The iterator is closed.
	//
	// util.ForEachLibraryIterator should be used to implement this function
	// unless that performance reason exists.
	ForEach(func(Library) error) error
	// Close releases any resources used by the iterator.
	Close()
}

LibraryIterator represents a Location iterator.

type Location

type Location interface {
	// ID returns the LocationID for this Location.
	ID() LocationID
	// Library returns the the Library the Location belongs to if any.
	Library() Library
	// Init initializes a new Repository at this Location.
	Init(RepositoryID) (Repository, error)
	// Get open a repository with the given RepositoryID, this operation doesn't
	// perform any read operation. If a repository with the given RepositoryID
	// already exists ErrRepositoryExists is returned.
	Get(RepositoryID, Mode) (Repository, error)
	// GetOrInit open or initilizes a Repository at this Location. If a
	// repository with the given RepositoryID can't be found the
	// ErrRepositoryNotExists is returned. If the repository is opened this will
	// be done in RWMode.
	GetOrInit(RepositoryID) (Repository, error)
	// Has returns true if the given RepositoryID matches any repository at
	// this location.
	Has(RepositoryID) (bool, error)
	// Repositories returns a RepositoryIterator that iterates through all
	// the repositories contained in this Location.
	Repositories(Mode) (RepositoryIterator, error)
}

Location interface represents a physical location where the repositories are stored, it allows access only to the repositories contained in this location.

type LocationID

type LocationID string

LocationID represents a Location identifier.

type LocationIterator

type LocationIterator interface {
	// Next returns the next location from the iterator. If the iterator has
	// reached the end it will return io.EOF as an error.
	Next() (Location, error)
	// ForEach call the function for each object contained on this iter until
	// an error happens or the end of the iter is reached. If ErrStop is sent
	// the iteration is stop but no error is returned. The iterator is closed.
	//
	// util.ForEachLocatorIterator should be used to implement this function
	// unless that performance reason exists.
	ForEach(func(Location) error) error
	// Close releases any resources used by the iterator.
	Close()
}

LocationIterator represents a Location iterator.

type Mode

type Mode int

Mode is the different modes to open a Repository.

const (
	// RWMode allows to perform read and write operations over a repository.
	RWMode Mode = iota
	// ReadOnlyMode allows only read-only operations over a repository.
	ReadOnlyMode
)

type Repository

type Repository interface {
	// ID returns the RepositoryID.
	ID() RepositoryID
	// Location returns the the Library the Location belongs to.
	Location() Location
	// Mode returns the Mode how it was opened.
	Mode() Mode
	// Commit persists all the write operations done since was open, if the
	// repository doesn't provide Transactional capabilities should return
	// ErrNonTransactional.
	Commit() error
	// Close closes the repository, if the repository was opened in transactional
	// Mode, will delete any write operation pending to be written.
	Close() error
	// R returns the git.Repository.
	R() *git.Repository
	// FS returns the filesystem to read or write directly to the repository or
	// nil if not available.
	FS() billy.Filesystem
}

Repository interface represents a git.Repository, with information about how it was open and where is located. Also provides Transactional capabilities through the method Commit.

type RepositoryID

type RepositoryID string

RepositoryID represents a Repository identifier, these IDs regularly are based on a http or git remore URL, but can be based on any other concept.

func NewRepositoryID

func NewRepositoryID(endpoint string) (RepositoryID, error)

NewRepositoryID returns a new RepositoryID based on a given endpoint. Eg.: git@github.com:src-d/go-borges becomes github.com/src-d/go-borges.git

func (RepositoryID) String

func (id RepositoryID) String() string

type RepositoryIterator

type RepositoryIterator interface {
	// Next returns the next repository from the iterator. If the iterator has
	// reached the end it will return io.EOF as an error.
	Next() (Repository, error)
	// ForEach call the function for each object contained on this iter until
	// an error happens or the end of the iter is reached. If ErrStop is sent
	// the iteration is stop but no error is returned. The iterator is closed.
	//
	// util.ForEachRepositoryIterator should be used to implement this function
	// unless that performance reason exists.
	ForEach(func(Repository) error) error
	// Close releases any resources used by the iterator.
	Close()
}

RepositoryIterator represents a Repository iterator.

Directories

Path Synopsis
Package legacysiva implements a go-borges library that uses siva files as its storage backend.
Package legacysiva implements a go-borges library that uses siva files as its storage backend.
Package siva implements a go-borges library that uses siva files as its storage backend.
Package siva implements a go-borges library that uses siva files as its storage backend.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL