car

package module
v0.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 30, 2021 License: Apache-2.0, MIT Imports: 19 Imported by: 1

README

go-car (go!)

Go Reference Coverage Status

A library to interact with merkledags stored as a single file

This is a Go implementation of the CAR specifications, both CARv1 and CARv2.

Note that there are two major module versions:

  • go-car/v2 is geared towards reading and writing CARv2 files, and also supports consuming CARv1 files and using CAR files as an IPFS blockstore.
  • go-car v0, in the root directory, just supports reading and writing CARv1 files.

Most users should use v2, especially for new software, since the v2 API transparently supports both CAR formats.

Features

CARv2 features:

  • Generate index from an existing CARv1 file
  • Wrap CARv1 files into a CARv2 with automatic index generation.
  • Random-access to blocks in a CAR file given their CID via Read-Only blockstore API, with transparent support for both CARv1 and CARv2
  • Write CARv2 files via Read-Write blockstore API, with support for appending blocks to an existing CARv2 file, and resumption from a partially written CARv2 files.
  • Individual access to inner CARv1 data payload and index of a CARv2 file via the Reader API.

Install

To install the latest version of go-car/v2 module, run:

go get github.com/ipld/go-car/v2

Alternatively, to install the v0 module, run:

go get github.com/ipld/go-car

API Documentation

See docs on pkg.go.dev.

Examples

Here is a shortlist of other examples from the documentation

Maintainers

Contribute

PRs are welcome!

When editing the Readme, please conform to the standard-readme specification.

License

Apache-2.0/MIT © Protocol Labs

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DefaultWalkFunc

func DefaultWalkFunc(nd format.Node) ([]*format.Link, error)

func HeaderSize

func HeaderSize(h *CarHeader) (uint64, error)

func WriteCar

func WriteCar(ctx context.Context, ds format.NodeGetter, roots []cid.Cid, w io.Writer) error

func WriteCarWithWalker

func WriteCarWithWalker(ctx context.Context, ds format.NodeGetter, roots []cid.Cid, w io.Writer, walk WalkFunc) error

func WriteHeader

func WriteHeader(h *CarHeader, w io.Writer) error

Types

type Block

type Block struct {
	BlockCID cid.Cid
	Data     []byte
	Offset   uint64
	Size     uint64
}

Block is all information and metadata about a block that is part of a car file

type CarHeader

type CarHeader struct {
	Roots   []cid.Cid
	Version uint64
}

func LoadCar

func LoadCar(s Store, r io.Reader) (*CarHeader, error)

func ReadHeader

func ReadHeader(br *bufio.Reader) (*CarHeader, error)

type CarReader

type CarReader struct {
	Header *CarHeader
	// contains filtered or unexported fields
}

func NewCarReader

func NewCarReader(r io.Reader) (*CarReader, error)

func (*CarReader) Next

func (cr *CarReader) Next() (blocks.Block, error)

type Dag

type Dag struct {
	Root     cid.Cid
	Selector ld.Node
}

Dag is a root/selector combo to put into a car

type OnCarHeaderFunc

type OnCarHeaderFunc func(CarHeader) error

OnCarHeaderFunc is called during traversal when the header is created

type OnNewCarBlockFunc

type OnNewCarBlockFunc func(Block) error

OnNewCarBlockFunc is called during traveral when a new unique block is encountered

type ReadStore

type ReadStore interface {
	Get(cid.Cid) (blocks.Block, error)
}

type SelectiveCar

type SelectiveCar struct {
	// contains filtered or unexported fields
}

SelectiveCar is a car file based on root + selector combos instead of just a single root and complete dag walk

func NewSelectiveCar

func NewSelectiveCar(ctx context.Context, store ReadStore, dags []Dag) SelectiveCar

NewSelectiveCar creates a new SelectiveCar for the given car file based a block store and set of root+selector pairs

func (SelectiveCar) Prepare

func (sc SelectiveCar) Prepare(userOnNewCarBlocks ...OnNewCarBlockFunc) (SelectiveCarPrepared, error)

Prepare traverse a car file and collects data on what is about to be written, but does not actually write the file

func (SelectiveCar) Write

func (sc SelectiveCar) Write(w io.Writer, userOnNewCarBlocks ...OnNewCarBlockFunc) error

type SelectiveCarPrepared

type SelectiveCarPrepared struct {
	SelectiveCar
	// contains filtered or unexported fields
}

SelectiveCarPrepared is a SelectiveCar that has already been traversed, such that it can be written quicker with Dump. It also contains metadata already collection about the Car file like size and number of blocks that go into it

func (SelectiveCarPrepared) Cids

func (sc SelectiveCarPrepared) Cids() []cid.Cid

Cids returns the list of unique block cids that will be written to the car file

func (SelectiveCarPrepared) Dump

func (sc SelectiveCarPrepared) Dump(w io.Writer) error

Dump writes the car file as quickly as possible based on information already collected

func (SelectiveCarPrepared) Header

func (sc SelectiveCarPrepared) Header() CarHeader

Header returns the header for the car file that will be written

func (SelectiveCarPrepared) Size

func (sc SelectiveCarPrepared) Size() uint64

Size returns the total size in bytes of the car file that will be written

type Store

type Store interface {
	Put(blocks.Block) error
}

type WalkFunc

type WalkFunc func(format.Node) ([]*format.Link, error)

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL