index

package
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 17, 2022 License: BSD-3-Clause Imports: 23 Imported by: 1

Documentation

Overview

package index provides methods for bulk indexing collections of Who's On First documents in Elasticsearch.

Index

Constants

View Source
const FLAG_APPEND_SPELUNKER_V1 string = "append-spelunker-v1-properties"
View Source
const FLAG_ES_ENDPOINT string = "elasticsearch-endpoint"
View Source
const FLAG_ES_INDEX string = "elasticsearch-index"
View Source
const FLAG_INDEX_ALT string = "index-alt-files"
View Source
const FLAG_INDEX_PROPS string = "index-only-properties"
View Source
const FLAG_INDEX_SPELUNKER_V1 string = "index-spelunker-v1"
View Source
const FLAG_ITERATOR_URI string = "iterator-uri"
View Source
const FLAG_WORKERS string = "workers"

Variables

This section is empty.

Functions

func BulkIndexerFromFlagSet added in v0.5.0

func BulkIndexerFromFlagSet(ctx context.Context, fs *flag.FlagSet) (esutil.BulkIndexer, error)

BulkIndexerFromFlagSet returns a esutil.BulkIndexer instance derived from the values in 'fs'.

func NewBulkIndexerFlagSet

func NewBulkIndexerFlagSet(ctx context.Context) (*flag.FlagSet, error)

NewBulkIndexerFlagSet creates a new `flag.FlagSet` instance with command-line flags required by the `es-whosonfirst-index` tool.

func PrepareFuncsFromFlagSet added in v0.5.0

func PrepareFuncsFromFlagSet(ctx context.Context, fs *flag.FlagSet) ([]document.PrepareDocumentFunc, error)

PrepareFuncsFromFlagSet returns a list of zero or more known `document.PrepareDocumentFunc` functions based on the values in 'fs'.

func RunBulkIndexer added in v0.5.0

func RunBulkIndexer(ctx context.Context, opts *RunBulkIndexerOptions) (*esutil.BulkIndexerStats, error)

RunBulkIndexer will "bulk" index a set of Who's On First documents with configuration details defined in 'opts'.

func RunBulkIndexerWithFlagSet

func RunBulkIndexerWithFlagSet(ctx context.Context, fs *flag.FlagSet) (*esutil.BulkIndexerStats, error)

RunBulkIndexerWithFlagSet will "bulk" index a set of Who's On First documents with configuration details defined in 'fs'.

func RunES2BulkIndexerWithFlagSet added in v0.4.0

func RunES2BulkIndexerWithFlagSet(ctx context.Context, fs *flag.FlagSet) (*es.BulkProcessorStats, error)

RunBulkIndexerWithFlagSet will "bulk" index a set of Who's On First documents with configuration details defined by `fs`.

Types

type RunBulkIndexerOptions added in v0.5.0

type RunBulkIndexerOptions struct {
	// BulkIndexer is a `esutil.BulkIndexer` instance
	BulkIndexer esutil.BulkIndexer
	// PrepareFuncs are one or more `document.PrepareDocumentFunc` used to transform a document before indexing
	PrepareFuncs []document.PrepareDocumentFunc
	// IteratorURI is a valid `whosonfirst/go-whosonfirst-iterate/v2` URI string.
	IteratorURI string
	// IteratorPaths are one or more valid `whosonfirst/go-whosonfirst-iterate/v2` paths to iterate over
	IteratorPaths []string
	// IndexAltFiles is a boolean value indicating whether or not to index "alternate geometry" files
	IndexAltFiles bool
}

type RunBulkIndexerOptions contains runtime configurations for bulk indexing

func RunBulkIndexerOptionsFromFlagSet added in v0.5.0

func RunBulkIndexerOptionsFromFlagSet(ctx context.Context, fs *flag.FlagSet) (*RunBulkIndexerOptions, error)

RunBulkIndexerOptionsFromFlagSet returns a `RunBulkIndexerOptions` instance derived from the values in 'fs'.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL