car

package module
v0.6.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 3, 2023 License: Apache-2.0, MIT Imports: 21 Imported by: 186

README

go-car (go!)

Go Reference Coverage Status

Work with car (Content addressed ARchive) files!

This is a Golang implementation of the CAR specifications, both CARv1 and CARv2.

As a format, there are two major module versions:

  • go-car/v2 is geared towards reading and writing CARv2 files, and also supports consuming CARv1 files and using CAR files as an IPFS blockstore.
  • go-car, in the root directory, only supports reading and writing CARv1 files.

Most users should use v2, especially for new software, since the v2 API transparently supports both CAR formats.

Usage / Installation

This repository provides a car binary that can be used for creating, extracting, and working with car files.

To install the latest version of car, run:

go install github.com/ipld/go-car/cmd/car@latest

More information about this binary is available in cmd/car

Features

CARv2 features:

  • Generate index from an existing CARv1 file
  • Wrap CARv1 files into a CARv2 with automatic index generation.
  • Random-access to blocks in a CAR file given their CID via Read-Only blockstore API, with transparent support for both CARv1 and CARv2
  • Write CARv2 files via Read-Write blockstore API, with support for appending blocks to an existing CARv2 file, and resumption from a partially written CARv2 files.
  • Individual access to inner CARv1 data payload and index of a CARv2 file via the Reader API.

API Documentation

See docs on pkg.go.dev.

Examples

Here is a shortlist of other examples from the documentation

Maintainers

Contribute

PRs are welcome!

When editing the Readme, please conform to the standard-readme specification.

License

Apache-2.0/MIT © Protocol Labs

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DefaultWalkFunc 

func DefaultWalkFunc(nd format.Node) ([]*format.Link, error)

func HeaderSize 

func HeaderSize(h *CarHeader) (uint64, error)

func WriteCar 

func WriteCar(ctx context.Context, ds format.NodeGetter, roots []cid.Cid, w io.Writer, options ...merkledag.WalkOption) error

func WriteCarWithWalker 

func WriteCarWithWalker(ctx context.Context, ds format.NodeGetter, roots []cid.Cid, w io.Writer, walk WalkFunc, options ...merkledag.WalkOption) error

func WriteHeader 

func WriteHeader(h *CarHeader, w io.Writer) error

Types

type Block 

type Block struct {
	BlockCID cid.Cid
	Data     []byte
	Offset   uint64
	Size     uint64
}

Block is all information and metadata about a block that is part of a car file

type CarHeader 

type CarHeader struct {
	Roots   []cid.Cid
	Version uint64
}

func LoadCar 

func LoadCar(ctx context.Context, s Store, r io.Reader) (*CarHeader, error)

func ReadHeader 

func ReadHeader(br *bufio.Reader) (*CarHeader, error)

type CarReader 

type CarReader struct {
	Header *CarHeader
	// contains filtered or unexported fields
}

func NewCarReader 

func NewCarReader(r io.Reader) (*CarReader, error)

func NewCarReaderWithOptions added in v0.6.2 

func NewCarReaderWithOptions(r io.Reader, opts ...CarReaderOption) (*CarReader, error)

func (*CarReader) Next 

func (cr *CarReader) Next() (blocks.Block, error)

type CarReaderOption added in v0.6.2 

type CarReaderOption func(*CarReader) error

func WithErrorOnEmptyRoots added in v0.6.2 

func WithErrorOnEmptyRoots(flag bool) CarReaderOption

WithErrorOnEmptyRoots is an option that can be passed to NewCarReader to specify the behavior when reading a car file that does not have any root cids set in the CAR header. Setting this option to true will cause CarReader to error on CAR that has no root CIDs listed in the header.

type Dag 

type Dag struct {
	Root     cid.Cid
	Selector ipld.Node
}

Dag is a root/selector combo to put into a car

type OnCarHeaderFunc 

type OnCarHeaderFunc func(CarHeader) error

OnCarHeaderFunc is called during traversal when the header is created

type OnNewCarBlockFunc 

type OnNewCarBlockFunc func(Block) error

OnNewCarBlockFunc is called during traveral when a new unique block is encountered

type Option added in v0.3.2 

type Option func(*options)

Option describes an option which affects behavior when interacting with the interface. 

func MaxTraversalLinks(MaxTraversalLinks uint64) Option

MaxTraversalLinks changes the allowed number of links a selector traversal can execute before failing.

Note that setting this option may cause an error to be returned from selector execution when building a SelectiveCar.

func TraverseLinksOnlyOnce added in v0.3.2 

func TraverseLinksOnlyOnce() Option

TraverseLinksOnlyOnce prevents the traversal engine from repeatedly visiting the same links more than once.

This can be an efficient strategy for an exhaustive selector where it's known that repeat visits won't impact the completeness of execution. However it should be used with caution with most other selectors as repeat visits of links for different reasons during selector execution can be valid and necessary to perform full traversal.

type ReadStore 

type ReadStore interface {
	Get(context.Context, cid.Cid) (blocks.Block, error)
}

type SelectiveCar 

type SelectiveCar struct {
	// contains filtered or unexported fields
}

SelectiveCar is a car file based on root + selector combos instead of just a single root and complete dag walk

func NewSelectiveCar 

func NewSelectiveCar(ctx context.Context, store ReadStore, dags []Dag, opts ...Option) SelectiveCar

NewSelectiveCar creates a new SelectiveCar for the given car file based a block store and set of root+selector pairs

func (SelectiveCar) Prepare 

func (sc SelectiveCar) Prepare(userOnNewCarBlocks ...OnNewCarBlockFunc) (SelectiveCarPrepared, error)

Prepare traverse a car file and collects data on what is about to be written, but does not actually write the file

func (SelectiveCar) Write 

func (sc SelectiveCar) Write(w io.Writer, userOnNewCarBlocks ...OnNewCarBlockFunc) error

type SelectiveCarPrepared 

type SelectiveCarPrepared struct {
	SelectiveCar
	// contains filtered or unexported fields
}

SelectiveCarPrepared is a SelectiveCar that has already been traversed, such that it can be written quicker with Dump. It also contains metadata already collection about the Car file like size and number of blocks that go into it

func (SelectiveCarPrepared) Cids 

func (sc SelectiveCarPrepared) Cids() []cid.Cid

Cids returns the list of unique block cids that will be written to the car file

func (SelectiveCarPrepared) Dump 

func (sc SelectiveCarPrepared) Dump(ctx context.Context, w io.Writer) error

Dump writes the car file as quickly as possible based on information already collected

func (SelectiveCarPrepared) Header 

func (sc SelectiveCarPrepared) Header() CarHeader

Header returns the header for the car file that will be written

func (SelectiveCarPrepared) Size 

func (sc SelectiveCarPrepared) Size() uint64

Size returns the total size in bytes of the car file that will be written

type Store 

type Store interface {
	Put(context.Context, blocks.Block) error
}

type WalkFunc 

type WalkFunc func(format.Node) ([]*format.Link, error)

Source Files

View all Source files

Directories

Path Synopsis
car module
cmd module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.