local

package
v0.0.0-...-89def8d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 24, 2021 License: MIT Imports: 47 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// Skip skips the path merge.
	Skip MergeStrategy = "Skip"
	// Merge attempts to merge the paths (directories only).
	Merge = "Merge"
	// Replace replaces the old path with the new path.
	Replace = "Replace"
)

Variables

View Source 
var (
	// ErrUpToDate indicates there are no locally staged changes.
	ErrUpToDate = errors.New("everything up-to-date")
	// ErrAborted indicates the caller aborted the operation via a confirm function.
	ErrAborted = errors.New("operation aborted by caller")
)
View Source 
var (
	// ErrNotABucket indicates the given path is not within a bucket.
	ErrNotABucket = errors.New("not a bucket (or any of the parent directories): .textile")
	// ErrBucketExists is used during initialization to indicate the path already contains a bucket.
	ErrBucketExists = errors.New("bucket is already initialized")
	// ErrThreadRequired indicates the operation requires a thread ID but none was given.
	ErrThreadRequired = errors.New("thread ID is required")
)
View Source 
var MaxPullConcurrency = 10

MaxPullConcurrency is the maximum number of files that can be pulled concurrently.

Functions

func ChangeColor 

func ChangeColor(t du.ChangeType) func(arg interface{}) aurora2.Value

ChangeColor returns an appropriate color for the given change type.

func ChangeType 

func ChangeType(t du.ChangeType) string

ChangeType returns a string representation of a change type.

func DefaultConfConfig 

func DefaultConfConfig() cmd.ConfConfig

DefaultConfConfig returns the default ConfConfig.

func Diff 

func Diff(ctx context.Context, ds ipld.DAGService, a, b ipld.Node) ([]*du.Change, error)

Diff returns a set of changes that transform node 'a' into node 'b'. Modified from https://github.com/ipfs/go-merkledag/blob/master/dagutils/diff.go#L104 It only traverses links in the following cases: 1. two node's links number are greater than 0. 2. both of two nodes are ProtoNode. 3. neither of the nodes is a file node, which contains only unnamed raw blocks Otherwise, it compares the cid and emits a Mod change object.

func Ignore 

func Ignore(pth string) bool

Ignore returns true if the path contains an ignored file.

Types

type AddOption 

type AddOption func(*addOptions)

AddOption is used when staging a remote Unixfs dag cid in a local bucket.

func WithAddEvents 

func WithAddEvents(ch chan<- Event) AddOption

WithAddEvents allows the caller to receive events when staging files from a remote Unixfs dag.

func WithSelectMerge 

func WithSelectMerge(f SelectMergeFunc) AddOption

WithSelectMerge allows the caller to select the path merge strategy.

type Bucket 

type Bucket struct {
	sync.Mutex
	// contains filtered or unexported fields
}

Bucket is a local-first object storage and synchronization model built on ThreadDB, IPFS, and Filecoin. A bucket represents a dynamic Unixfs directory with auto-updating IPNS resolution, website rendering, and Filecoin archivng.

Private buckets are fully encrypted using AES-CTR + AES-512 HMAC (see https://github.com/textileio/dcrypto for more). Both Unixfs node metadata (size, links, etc.) and node data (files) are obfuscated by encryption. The AES and HMAC keys used for bucket encryption are stored in the ThreadDB collection instance. This setup allows for bucket access to inherit from thread ACL rules.

Additionally, files can be further protected by password-based encryption before they are added to the bucket. See EncryptLocalPath and DecryptLocalPath for more.

func (*Bucket) AddRemoteCid 

func (b *Bucket) AddRemoteCid(ctx context.Context, c cid.Cid, dest string, opts ...AddOption) error

AddRemoteCid stages the Unixfs dag at cid in the local bucket, optionally allowing the caller to select a merge strategy.

func (*Bucket) CatRemotePath 

func (b *Bucket) CatRemotePath(ctx context.Context, pth string, w io.Writer) error

CatRemotePath writes the content of the remote path to writer.

func (*Bucket) DecryptLocalPath 

func (b *Bucket) DecryptLocalPath(pth string, key []byte, w io.Writer) error

DecryptLocalPath decrypts the file at path with key, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC.

func (*Bucket) DecryptLocalPathWithPassword 

func (b *Bucket) DecryptLocalPathWithPassword(pth, password string, w io.Writer) error

DecryptLocalPathWithPassword decrypts the file at path with password, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC. https://godoc.org/golang.org/x/crypto/scrypt is used to derive the keys from the password.

func (*Bucket) DecryptRemotePath 

func (b *Bucket) DecryptRemotePath(ctx context.Context, pth string, key []byte, w io.Writer) error

DecryptRemotePath decrypts the file at the remote path with password, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC.

func (*Bucket) DecryptRemotePathWithPassword 

func (b *Bucket) DecryptRemotePathWithPassword(ctx context.Context, pth, password string, w io.Writer) error

DecryptRemotePathWithPassword decrypts the file at the remote path with password, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC. https://godoc.org/golang.org/x/crypto/scrypt is used to derive the keys from the password.

func (*Bucket) Destroy 

func (b *Bucket) Destroy(ctx context.Context) (err error)

Destroy completely deletes the local and remote bucket.

func (*Bucket) DiffLocal 

func (b *Bucket) DiffLocal() ([]Change, error)

DiffLocal returns a list of locally staged bucket file changes.

func (*Bucket) EncryptLocalPath 

func (b *Bucket) EncryptLocalPath(pth string, key []byte, w io.Writer) error

EncryptLocalPath encrypts the file at path with key, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC.

func (*Bucket) EncryptLocalPathWithPassword 

func (b *Bucket) EncryptLocalPathWithPassword(pth, password string, w io.Writer) error

EncryptLocalPathWithPassword encrypts the file at path with password, writing the result to the writer. Encryption is AES-CTR + AES-512 HMAC. https://godoc.org/golang.org/x/crypto/scrypt is used to derive the keys from the password.

func (*Bucket) Get 

func (b *Bucket) Get(ctx context.Context) (bucket buckets.Bucket, err error)

Get returns info about a bucket from the remote.

func (*Bucket) GetIdentityToken 

func (b *Bucket) GetIdentityToken(dur time.Duration) (did.Token, error)

GetIdentityToken returns a did.Token for the bucket identity valid for the given duration.

func (*Bucket) Identity 

func (b *Bucket) Identity() (thread.Identity, error)

Identity returns the identity associated with the bucket.

func (*Bucket) Key 

func (b *Bucket) Key() string

Key returns the bucket's unique key identifier, which is also an IPNS public key.

func (*Bucket) ListRemotePath 

func (b *Bucket) ListRemotePath(ctx context.Context, pth string) (items []BucketItem, err error)

ListRemotePath returns a list of all bucket items under path.

func (*Bucket) LocalSize 

func (b *Bucket) LocalSize() (int64, error)

LocalSize returns the cumalative size of the bucket's local files.

func (*Bucket) Path 

func (b *Bucket) Path() (string, error)

Path returns the bucket's top-level local filesystem path.

func (*Bucket) PullPathAccessRoles 

func (b *Bucket) PullPathAccessRoles(ctx context.Context, pth string) (roles map[did.DID]collection.Role, err error)

PullPathAccessRoles returns access roles for a path.

func (*Bucket) PullRemote 

func (b *Bucket) PullRemote(ctx context.Context, opts ...PathOption) (roots Roots, err error)

PullRemote pulls remote files. By default, only missing files are pulled. See PathOption for more info.

func (*Bucket) PushLocal 

func (b *Bucket) PushLocal(ctx context.Context, opts ...PathOption) (roots Roots, err error)

PushLocal pushes local files. By default, only staged changes are pushed. See PathOption for more info.

func (*Bucket) PushPathAccessRoles 

func (b *Bucket) PushPathAccessRoles(
	ctx context.Context,
	pth string,
	roles map[did.DID]collection.Role,
) (merged map[did.DID]collection.Role, err error)

PushPathAccessRoles updates path access roles by merging the pushed roles with existing roles and returns the merged roles. roles is a map of string marshaled public keys to path roles. A non-nil error is returned if the map keys are not unmarshalable to public keys. To delete a role for a public key, set its value to buckets.None. 

func (b *Bucket) RemoteLinks(ctx context.Context, pth string) (links buckets.Links, err error)

RemoteLinks returns the remote links for the bucket.

func (*Bucket) Roots 

func (b *Bucket) Roots(ctx context.Context) (roots Roots, err error)

Roots returns the bucket's current local and remote root cids.

func (*Bucket) Thread 

func (b *Bucket) Thread() (id thread.ID, err error)

Thread returns the bucket's thread ID.

type BucketItem 

type BucketItem struct {
	Cid        cid.Cid      `json:"cid"`
	Name       string       `json:"name"`
	Path       string       `json:"path"`
	Size       int64        `json:"size"`
	IsDir      bool         `json:"is_dir"`
	Items      []BucketItem `json:"items"`
	ItemsCount int          `json:"items_count"`
}

BucketItem describes an item (file/directory) in a bucket.

type Buckets 

type Buckets struct {
	// contains filtered or unexported fields
}

Buckets is used to create new individual buckets with the provided client and config.

func NewBuckets 

func NewBuckets(c *client.Client, config cmd.ConfConfig) *Buckets

NewBuckets creates Buckets with client and config.

func (*Buckets) Client 

func (b *Buckets) Client() *client.Client

Client returns the underlying client object.

func (*Buckets) GetLocalBucket 

func (b *Buckets) GetLocalBucket(ctx context.Context, conf Config) (*Bucket, error)

GetLocalBucket loads and returns the bucket at path if it exists.

func (*Buckets) NewBucket 

func (b *Buckets) NewBucket(ctx context.Context, conf Config, opts ...NewOption) (buck *Bucket, err error)

NewBucket initializes a new bucket from the config. A local blockstore is created that's used to sync local changes with the remote. By default, this will be an unencrypted, unnamed, empty bucket. The remote bucket will also be created if it doesn't already exist. See NewOption for more info.

func (*Buckets) NewConfigFromCmd 

func (b *Buckets) NewConfigFromCmd(c *cobra.Command, pth string) (conf Config, err error)

NewConfigFromCmd returns a config by inflating values from the given cobra command and path. First, flags for "key" and "thread" are used if they exist. If still unset, the env vars {EnvPrefix}_KEY and {EnvPrefix}_THREAD are used.

func (*Buckets) RemoteBuckets 

func (b *Buckets) RemoteBuckets(
	ctx context.Context,
	id thread.ID,
	identity thread.Identity,
) (list []buckets.Bucket, err error)

RemoteBuckets lists all existing remote buckets in the thread.

type Change 

type Change struct {
	Type du.ChangeType
	Name string // Absolute file name
	Path string // File name relative to the bucket root
	Rel  string // File name relative to the bucket current working directory
}

Change describes a local bucket change.

type Config 

type Config struct {
	// Path is the path in which the new bucket should be created (required).
	Path string
	// Key is a key of an existing bucket (optional).
	// It's value may be inflated from a --key flag or {EnvPrefix}_KEY env variable.
	Key string
	// Thread is the thread ID of the target thread (required).
	// It's value may be inflated from a --thread flag or {EnvPrefix}_THREAD env variable.
	Thread thread.ID
	// Identity is an identity to use with the target thread.
	// It's value may be inflated from an --identity flag or {EnvPrefix}_IDENTITY env variable.
	// @todo: Handle more identities
	// @todo: Pull this from a global config of identites, i.e., ~/.threads:
	//   identities:
	//     default: clyde
	//     clyde: <priv_key_base_64>
	//     eddy: <priv_key_base_64>
	Identity thread.Identity
}

Config contains details for a new local bucket.

type ConfirmDiffFunc 

type ConfirmDiffFunc func([]Change) bool

ConfirmDiffFunc is a caller-provided function which presents a list of bucket changes.

type Event 

type Event struct {
	// Type of event.
	Type EventType
	// Path relative to the bucket's cwd.
	Path string
	// Cid of associated Path.
	Cid cid.Cid
	// Size of the total operation or completed file.
	Size int64
	// Complete is the amount of Size that is complete (useful for upload/download progress).
	Complete int64
}

Event describes a path event that occurred. These events can be used to display live info during path uploads/downloads.

type EventType 

type EventType int

EventType is the type of path event. 

const (
	// EventProgress indicates a file has made some progress uploading/downloading.
	EventProgress EventType = iota
	// EventFileComplete indicates a file has completed uploading/downloading.
	EventFileComplete
	// EventFileRemoved indicates a file has been removed.
	EventFileRemoved
)

type InitStrategy 

type InitStrategy int

InitStrategy describes the type of init strategy. 

const (
	// Hybrid indicates locally staged changes should be accepted, excluding deletions.
	Hybrid InitStrategy = iota
	// Soft indicates locally staged changes should be accepted, including deletions.
	Soft
	// Hard indicates locally staged changes should be discarded.
	Hard
)

type MergeStrategy 

type MergeStrategy string

MergeStrategy describes the type of path merge strategy.

type NewOption 

type NewOption func(*newOptions)

NewOption is used when creating a new bucket.

func WithCid 

func WithCid(c cid.Cid) NewOption

WithCid indicates an inited bucket should be boostraped with a particular UnixFS DAG.

func WithInitEvents 

func WithInitEvents(ch chan<- Event) NewOption

WithInitEvents allows the caller to receive events when pulling files from an existing bucket on initialization.

func WithName 

func WithName(name string) NewOption

WithName sets a name for the bucket.

func WithPrivate 

func WithPrivate(private bool) NewOption

WithPrivate specifies that an encryption key will be used for the bucket.

func WithStrategy 

func WithStrategy(strategy InitStrategy) NewOption

WithStrategy allows for selecting the init strategy. Hybrid is the default.

type PathOption 

type PathOption func(*pathOptions)

PathOption is used when pushing or pulling bucket paths.

func WithConfirm 

func WithConfirm(f ConfirmDiffFunc) PathOption

WithConfirm allows the caller to confirm a list of bucket changes.

func WithEvents 

func WithEvents(ch chan<- Event) PathOption

WithEvents allows the caller to receive events when pushing or pulling files.

func WithForce 

func WithForce(b bool) PathOption

WithForce indicates all remote files should be pulled even if they already exist.

func WithHard 

func WithHard(b bool) PathOption

WithHard indicates locally staged changes should be discarded.

type Repo 

type Repo struct {
	// contains filtered or unexported fields
}

Repo tracks a local bucket tree structure.

func NewRepo 

func NewRepo(pth, name string, layout options.Layout) (*Repo, error)

NewRepo creates a new bucket with the given path.

func (*Repo) CidVersion 

func (b *Repo) CidVersion() int

CidVersion gets the repo cid version (0 or 1).

func (*Repo) Close 

func (b *Repo) Close() error

Close closes the store and blocks service.

func (*Repo) Diff 

func (b *Repo) Diff(ctx context.Context, pth string) (diff []*du.Change, err error)

Diff returns a list of changes that are present in path compared to the bucket.

func (*Repo) GetNode 

func (b *Repo) GetNode(ctx context.Context, c cid.Cid) (ipld.Node, error)

GetNode returns the node at cid from the bucket.

func (*Repo) GetPathMap 

func (b *Repo) GetPathMap(pth string) (local, remote cid.Cid, err error)

GetPathMap returns details about a local path.

func (*Repo) HashFile 

func (b *Repo) HashFile(pth string) (cid.Cid, error)

HashFile returns the cid of the file at path. This method does not alter the bucket.

func (*Repo) MatchPath 

func (b *Repo) MatchPath(pth string, local, remote cid.Cid) (bool, error)

MatchPath returns whether or not the path exists and has matching local and remote cids.

func (*Repo) Path 

func (b *Repo) Path() string

Path returns the repo path.

func (*Repo) RemovePath 

func (b *Repo) RemovePath(ctx context.Context, pth string) error

RemovePath removes a local path map from the store.

func (*Repo) Root 

func (b *Repo) Root() (local, remote cid.Cid, err error)

Root returns the local and remote root cids.

func (*Repo) Save 

func (b *Repo) Save(ctx context.Context) error

Save saves the bucket as a node describing the file tree at the current path.

func (*Repo) SaveFile 

func (b *Repo) SaveFile(ctx context.Context, pth string, name string) error

SaveFile saves the file at path to the repo.

func (*Repo) SetCidVersion 

func (b *Repo) SetCidVersion(v int)

SetCidVersion set the repo cid version (0 or 1). The default version is 1.

func (*Repo) SetRemotePath 

func (b *Repo) SetRemotePath(pth string, remote cid.Cid) error

SetRemotePath sets or creates a mapping from a local path to a remote cid.

type Roots 

type Roots struct {
	Local  cid.Cid `json:"local"`
	Remote cid.Cid `json:"remote"`
}

Roots wraps local and remote root cids. If the bucket is not private (encrypted), these will be the same.

type SelectMergeFunc 

type SelectMergeFunc func(description string, isDir bool) (MergeStrategy, error)

SelectMergeFunc is a caller-provided function which is used to select a merge strategy.

type WatchOption 

type WatchOption func(*watchOptions)

WatchOption is used when watching a bucket for changes.

func WithOffline 

func WithOffline(offline bool) WatchOption

WithOffline will keep watching for bucket changes while the local network is offline.

func WithWatchEvents 

func WithWatchEvents(ch chan<- Event) WatchOption

WithWatchEvents allows the caller to receive events when watching a bucket for changes.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.