warcindexer

package module

v0.0.0-...-075ebfc Latest Latest Go to latest Published: Sep 11, 2022 License: MIT Imports: 26 Imported by: 2

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/crossedbot/go-warc-indexer

Links

Open Source Insights

Documentation ¶

Index ¶

Constants
Variables
type Indexer
- func New(ctx context.Context, addr string) (Indexer, error)
type JsonBlock

Constants ¶

View Source

const (
	// Encryption Types
	NoneEncryption   = "none"
	AesGcmEncryption = "aes-gcm"
)

Variables ¶

View Source

var (
	// CDXJ Versioning
	VersionKey   = "WARC-CDXJ"
	VersionValue = "1.0"
)

Functions ¶

This section is empty.

Types ¶

type Indexer ¶

type Indexer interface {
	// Index indexes the WARC file at the given file path and returns a CDXJ
	Index(name string) (simplecdxj.CDXJ, error)

	// SetEncryptionKey sets the encryption key and salt used to encrypt data
	// before pushing to IPFS
	SetEncryptionKey(key, salt []byte)
}

Indexer represents an interface to a WARC file to IPFS indexer

func New ¶

func New(ctx context.Context, addr string) (Indexer, error)

New returns a new Indexer, using the given Multiaddr formatted address to interact with the IPFS node

type JsonBlock ¶

type JsonBlock struct {
	// Defined Fields
	Uri string `json:"uri"`
	Ref string `json:"ref"`
	Sha string `json:"sha"`
	Hsc int    `json:"hsc"`
	Mct string `json:"mct"`
	Rid string `json:"rid"`

	// Custom Fields
	Locator          string `json:"x_locator"`
	Title            string `json:"x_title,omitempty"`
	EncryptionKeyID  string `json:"x_encryption_key_id,omitempty"`
	EncryptionMethod string `json:"x_encryption_method,omitempty"`
	EncryptionNonce  string `json:"x_encryption_nonce,omitempty"`
}

JsonBlock represents the content block (AKA payload) of the CDXJ record

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL