bleve: github.com/blevesearch/bleve/mapping Index | Examples | Files

package mapping

import "github.com/blevesearch/bleve/mapping"

Index

Examples

Package Files

analysis.go document.go field.go index.go mapping.go reflect.go

Variables

var (
    IndexDynamic = true
    StoreDynamic = true
)

control the default behavior for dynamic fields (those not explicitly mapped)

var MappingJSONStrict = false

func SetLog Uses

func SetLog(l *log.Logger)

SetLog sets the logger used for logging by default log messages are sent to ioutil.Discard

type Classifier Uses

type Classifier interface {
    Type() string
}

A Classifier is an interface describing any object which knows how to identify its own type. Alternatively, if a struct already has a Type field or method in conflict, one can use BleveType instead.

type DocumentMapping Uses

type DocumentMapping struct {
    Enabled         bool                        `json:"enabled"`
    Dynamic         bool                        `json:"dynamic"`
    Properties      map[string]*DocumentMapping `json:"properties,omitempty"`
    Fields          []*FieldMapping             `json:"fields,omitempty"`
    DefaultAnalyzer string                      `json:"default_analyzer"`

    // StructTagKey overrides "json" when looking for field names in struct tags
    StructTagKey string `json:"struct_tag_key,omitempty"`
}

A DocumentMapping describes how a type of document should be indexed. As documents can be hierarchical, named sub-sections of documents are mapped using the same structure in the Properties field. Each value inside a document can be indexed 0 or more ways. These index entries are called fields and are stored in the Fields field. Entire sections of a document can be ignored or excluded by setting Enabled to false. If not explicitly mapped, default mapping operations are used. To disable this automatic handling, set Dynamic to false.

func NewDocumentDisabledMapping Uses

func NewDocumentDisabledMapping() *DocumentMapping

NewDocumentDisabledMapping returns a new document mapping that will not perform any indexing.

func NewDocumentMapping Uses

func NewDocumentMapping() *DocumentMapping

NewDocumentMapping returns a new document mapping with all the default values.

func NewDocumentStaticMapping Uses

func NewDocumentStaticMapping() *DocumentMapping

NewDocumentStaticMapping returns a new document mapping that will not automatically index parts of a document without an explicit mapping.

func (*DocumentMapping) AddFieldMapping Uses

func (dm *DocumentMapping) AddFieldMapping(fm *FieldMapping)

AddFieldMapping adds the provided FieldMapping for this section of the document.

Code:

// you can only add field mapping to those properties which already have a document mapping
documentMapping := NewDocumentMapping()
subDocumentMapping := NewDocumentMapping()
documentMapping.AddSubDocumentMapping("Property", subDocumentMapping)

fieldMapping := NewTextFieldMapping()
fieldMapping.Analyzer = "en"
subDocumentMapping.AddFieldMapping(fieldMapping)

fmt.Println(len(documentMapping.Properties["Property"].Fields))

Output:

1

func (*DocumentMapping) AddFieldMappingsAt Uses

func (dm *DocumentMapping) AddFieldMappingsAt(property string, fms ...*FieldMapping)

AddFieldMappingsAt adds one or more FieldMappings at the named sub-document. If the named sub-document doesn't yet exist it is created for you. This is a convenience function to make most common mappings more concise. Otherwise, you would:

subMapping := NewDocumentMapping()
subMapping.AddFieldMapping(fieldMapping)
parentMapping.AddSubDocumentMapping(property, subMapping)

Code:

// you can only add field mapping to those properties which already have a document mapping
documentMapping := NewDocumentMapping()
subDocumentMapping := NewDocumentMapping()
documentMapping.AddSubDocumentMapping("NestedProperty", subDocumentMapping)

fieldMapping := NewTextFieldMapping()
fieldMapping.Analyzer = "en"
documentMapping.AddFieldMappingsAt("NestedProperty", fieldMapping)

fmt.Println(len(documentMapping.Properties["NestedProperty"].Fields))

Output:

1

func (*DocumentMapping) AddSubDocumentMapping Uses

func (dm *DocumentMapping) AddSubDocumentMapping(property string, sdm *DocumentMapping)

AddSubDocumentMapping adds the provided DocumentMapping as a sub-mapping for the specified named subsection.

Code:

// adds a document mapping for a property in a document
// useful for mapping nested documents
documentMapping := NewDocumentMapping()
subDocumentMapping := NewDocumentMapping()
documentMapping.AddSubDocumentMapping("Property", subDocumentMapping)

fmt.Println(len(documentMapping.Properties))

Output:

1

func (*DocumentMapping) UnmarshalJSON Uses

func (dm *DocumentMapping) UnmarshalJSON(data []byte) error

UnmarshalJSON offers custom unmarshaling with optional strict validation

func (*DocumentMapping) Validate Uses

func (dm *DocumentMapping) Validate(cache *registry.Cache) error

type FieldMapping Uses

type FieldMapping struct {
    Name string `json:"name,omitempty"`
    Type string `json:"type,omitempty"`

    // Analyzer specifies the name of the analyzer to use for this field.  If
    // Analyzer is empty, traverse the DocumentMapping tree toward the root and
    // pick the first non-empty DefaultAnalyzer found. If there is none, use
    // the IndexMapping.DefaultAnalyzer.
    Analyzer string `json:"analyzer,omitempty"`

    // Store indicates whether to store field values in the index. Stored
    // values can be retrieved from search results using SearchRequest.Fields.
    Store bool `json:"store,omitempty"`
    Index bool `json:"index,omitempty"`

    // IncludeTermVectors, if true, makes terms occurrences to be recorded for
    // this field. It includes the term position within the terms sequence and
    // the term offsets in the source document field. Term vectors are required
    // to perform phrase queries or terms highlighting in source documents.
    IncludeTermVectors bool   `json:"include_term_vectors,omitempty"`
    IncludeInAll       bool   `json:"include_in_all,omitempty"`
    DateFormat         string `json:"date_format,omitempty"`
}

A FieldMapping describes how a specific item should be put into the index.

func NewBooleanFieldMapping Uses

func NewBooleanFieldMapping() *FieldMapping

NewBooleanFieldMapping returns a default field mapping for booleans

func NewDateTimeFieldMapping Uses

func NewDateTimeFieldMapping() *FieldMapping

NewDateTimeFieldMapping returns a default field mapping for dates

func NewGeoPointFieldMapping Uses

func NewGeoPointFieldMapping() *FieldMapping

NewGeoPointFieldMapping returns a default field mapping for geo points

func NewNumericFieldMapping Uses

func NewNumericFieldMapping() *FieldMapping

NewNumericFieldMapping returns a default field mapping for numbers

func NewTextFieldMapping Uses

func NewTextFieldMapping() *FieldMapping

NewTextFieldMapping returns a default field mapping for text

func (*FieldMapping) Options Uses

func (fm *FieldMapping) Options() document.IndexingOptions

Options returns the indexing options for this field.

func (*FieldMapping) UnmarshalJSON Uses

func (fm *FieldMapping) UnmarshalJSON(data []byte) error

UnmarshalJSON offers custom unmarshaling with optional strict validation

type IndexMapping Uses

type IndexMapping interface {
    MapDocument(doc *document.Document, data interface{}) error
    Validate() error

    DateTimeParserNamed(name string) analysis.DateTimeParser

    DefaultSearchField() string

    AnalyzerNameForPath(path string) string
    AnalyzerNamed(name string) *analysis.Analyzer
}

type IndexMappingImpl Uses

type IndexMappingImpl struct {
    TypeMapping           map[string]*DocumentMapping `json:"types,omitempty"`
    DefaultMapping        *DocumentMapping            `json:"default_mapping"`
    TypeField             string                      `json:"type_field"`
    DefaultType           string                      `json:"default_type"`
    DefaultAnalyzer       string                      `json:"default_analyzer"`
    DefaultDateTimeParser string                      `json:"default_datetime_parser"`
    DefaultField          string                      `json:"default_field"`
    StoreDynamic          bool                        `json:"store_dynamic"`
    IndexDynamic          bool                        `json:"index_dynamic"`
    CustomAnalysis        *customAnalysis             `json:"analysis,omitempty"`
    // contains filtered or unexported fields
}

An IndexMappingImpl controls how objects are placed into an index. First the type of the object is determined. Once the type is know, the appropriate DocumentMapping is selected by the type. If no mapping was determined for that type, a DefaultMapping will be used.

func NewIndexMapping Uses

func NewIndexMapping() *IndexMappingImpl

NewIndexMapping creates a new IndexMapping that will use all the default indexing rules

func (*IndexMappingImpl) AddCustomAnalyzer Uses

func (im *IndexMappingImpl) AddCustomAnalyzer(name string, config map[string]interface{}) error

AddCustomAnalyzer defines a custom analyzer for use in this mapping. The config map must have a "type" string entry to resolve the analyzer constructor. The constructor is invoked with the remaining entries and returned analyzer is registered in the IndexMapping.

bleve comes with predefined analyzers, like github.com/blevesearch/bleve/analysis/analyzers/custom_analyzer. They are available only if their package is imported by client code. To achieve this, use their metadata to fill configuration entries:

import (
    "github.com/blevesearch/bleve/analysis/analyzers/custom_analyzer"
    "github.com/blevesearch/bleve/analysis/char_filters/html_char_filter"
    "github.com/blevesearch/bleve/analysis/token_filters/lower_case_filter"
    "github.com/blevesearch/bleve/analysis/tokenizers/unicode"
)

m := bleve.NewIndexMapping()
err := m.AddCustomAnalyzer("html", map[string]interface{}{
    "type": custom_analyzer.Name,
    "char_filters": []string{
        html_char_filter.Name,
    },
    "tokenizer":     unicode.Name,
    "token_filters": []string{
        lower_case_filter.Name,
        ...
    },
})

func (*IndexMappingImpl) AddCustomCharFilter Uses

func (im *IndexMappingImpl) AddCustomCharFilter(name string, config map[string]interface{}) error

AddCustomCharFilter defines a custom char filter for use in this mapping

func (*IndexMappingImpl) AddCustomDateTimeParser Uses

func (im *IndexMappingImpl) AddCustomDateTimeParser(name string, config map[string]interface{}) error

AddCustomDateTimeParser defines a custom date time parser for use in this mapping

func (*IndexMappingImpl) AddCustomTokenFilter Uses

func (im *IndexMappingImpl) AddCustomTokenFilter(name string, config map[string]interface{}) error

AddCustomTokenFilter defines a custom token filter for use in this mapping

func (*IndexMappingImpl) AddCustomTokenMap Uses

func (im *IndexMappingImpl) AddCustomTokenMap(name string, config map[string]interface{}) error

AddCustomTokenMap defines a custom token map for use in this mapping

func (*IndexMappingImpl) AddCustomTokenizer Uses

func (im *IndexMappingImpl) AddCustomTokenizer(name string, config map[string]interface{}) error

AddCustomTokenizer defines a custom tokenizer for use in this mapping

func (*IndexMappingImpl) AddDocumentMapping Uses

func (im *IndexMappingImpl) AddDocumentMapping(doctype string, dm *DocumentMapping)

AddDocumentMapping sets a custom document mapping for the specified type

func (*IndexMappingImpl) AnalyzeText Uses

func (im *IndexMappingImpl) AnalyzeText(analyzerName string, text []byte) (analysis.TokenStream, error)

func (*IndexMappingImpl) AnalyzerNameForPath Uses

func (im *IndexMappingImpl) AnalyzerNameForPath(path string) string

AnalyzerNameForPath attempts to find the best analyzer to use with only a field name will walk all the document types, look for field mappings at the provided path, if one exists and it has an explicit analyzer that is returned.

func (*IndexMappingImpl) AnalyzerNamed Uses

func (im *IndexMappingImpl) AnalyzerNamed(name string) *analysis.Analyzer

func (*IndexMappingImpl) DateTimeParserNamed Uses

func (im *IndexMappingImpl) DateTimeParserNamed(name string) analysis.DateTimeParser

func (*IndexMappingImpl) DefaultSearchField Uses

func (im *IndexMappingImpl) DefaultSearchField() string

func (*IndexMappingImpl) FieldAnalyzer Uses

func (im *IndexMappingImpl) FieldAnalyzer(field string) string

FieldAnalyzer returns the name of the analyzer used on a field.

func (*IndexMappingImpl) MapDocument Uses

func (im *IndexMappingImpl) MapDocument(doc *document.Document, data interface{}) error

func (*IndexMappingImpl) UnmarshalJSON Uses

func (im *IndexMappingImpl) UnmarshalJSON(data []byte) error

UnmarshalJSON offers custom unmarshaling with optional strict validation

func (*IndexMappingImpl) Validate Uses

func (im *IndexMappingImpl) Validate() error

Validate will walk the entire structure ensuring the following explicitly named and default analyzers can be built

Package mapping imports 14 packages (graph) and is imported by 30 packages. Updated 2017-12-14. Refresh now. Tools for package owners.