pod

package
v0.0.0-...-a3f3dab Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 4, 2017 License: AGPL-3.0 Imports: 12 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var IpfsApiServerUrl = ""

Functions

func NewAddCatalog

func NewAddCatalog() tasks.Taskable

Types

type AddCatalog

type AddCatalog struct {
	// title for collection if no collection present
	CollectionTitle string
	// url that points to catalog
	Url string `json:"url"`
	// paginate into dataset list, zero is no pagination / offset
	Limit int
	// offset to start archiving at
	Offset int
	// how many fetching goroutines to spin up. max 5
	Parallelism int
	// skip items that already have a hash value
	SkipArchived bool
	// how long to sleep between requests in seconds(inside of parallel routines)
	CrawDelay time.Duration
	// contains filtered or unexported fields
}

AddCatalog injests a a collection to IPFS, it iterates through each setting hashes on collection urls and, eventually, generates a cdxj index of the archive

func (*AddCatalog) Do

func (t *AddCatalog) Do(pch chan tasks.Progress)

func (*AddCatalog) SetDatastore

func (t *AddCatalog) SetDatastore(store datastore.Datastore)

AddCatalog task needs to talk to an underlying database it's expected that the task executor will call this method before calling Do

func (*AddCatalog) Valid

func (t *AddCatalog) Valid() error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL