mg11

package

v1.0.2 Latest Latest Go to latest Published: Feb 17, 2022 License: MIT Imports: 32 Imported by: 1

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/ipfs/fs-repo-migrations

Links

Open Source Insights

Documentation ¶

Overview ¶

package mg11 contains the code to perform 11-12 repository migration in go-ipfs. This performs a switch to raw multihashes for all keys in the go-ipfs datastore (https://github.com/ipfs/go-ipfs/issues/6815).

Index ¶

Variables
func IsBasicFlatFSBlockstore(dstore ds.Datastore) (dsPath string, v1 *flatfs.ShardIdV1, err error)
type CidSwapper
type Migration
type Swap

Constants ¶

This section is empty.

Variables ¶

View Source

var EnableFlatFSFastPath bool = true

View Source

var NWorkers int = 1

NWorkers sets the number of swapping threads to run when applying a migration.

View Source

var SyncSize uint64 = 100 * 1024 * 1024 // 100MiB

SyncSize specifies how much we batch data before committing and syncing.

Functions ¶

func IsBasicFlatFSBlockstore ¶ added in v1.0.2

func IsBasicFlatFSBlockstore(dstore ds.Datastore) (dsPath string, v1 *flatfs.ShardIdV1, err error)

Types ¶

type CidSwapper ¶

type CidSwapper struct {
	Prefix ds.Key      // A prefix/namespace to limit the query.
	Store  ds.Batching // the datastore to migrate.
	SwapCh chan Swap   // a channel that gets notified for every swap
}

CidSwapper reads all the keys in a datastore and replaces them with their raw multihash.

func (*CidSwapper) Prepare ¶ added in v1.0.2

func (cswap *CidSwapper) Prepare() (uint64, error)

Prepare performs a dry run without copying anything but notifying SwapCh as it runs.

Retruns the total number of keys swapped.

func (*CidSwapper) Revert ¶

func (cswap *CidSwapper) Revert(unswapCh <-chan Swap) (uint64, error)

Revert allows to undo any operations made by Run(). The given channel should receive Swap objects as they were sent by Run. It returns the number of swap operations performed.

func (*CidSwapper) Run ¶

func (cswap *CidSwapper) Run(swapCh <-chan Swap) (uint64, error)

Run performs a migration reading the Swaps that need to be performed from the given Swap channel. The swaps can be obtained with Prepare().

Run returns the total number of keys swapped.

type Migration ¶

type Migration struct {
	// contains filtered or unexported fields
}

Migration implements the migration described above.

func (*Migration) Apply ¶

func (m *Migration) Apply(opts migrate.Options) error

Apply runs the migration and writes a log file that can be used by Revert. Steps:

Open a raw datastore using go-ipfs settings
Simulate the migration and save a backup log
Run the migration by storing all CIDv1 addressed logs as raw-multihash addressed.

func (*Migration) Reversible ¶

func (m *Migration) Reversible() bool

Reversible returns true.

func (*Migration) Revert ¶

func (m *Migration) Revert(opts migrate.Options) error

Revert attempts to undo the migration using the log file written by Apply. Steps: - Read the backup log and write all entries as a CIDv1-addressed block - Do the same with the MFS root - Do the same with all the CidV1 blocks recursively referenced in the pinset

Revert does not delete blocks that are reverted so cover some corner cases.

func (*Migration) Versions ¶

func (m *Migration) Versions() string

Versions returns the current version string for this migration.

type Swap ¶

type Swap struct {
	Old ds.Key
	New ds.Key
}

Swap holds the datastore keys for the original CID and for the destination Multihash.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL