sajari

package module
v0.0.0-...-9a18aee Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 27, 2017 License: MIT Imports: 16 Imported by: 8

README

Sajari APIs Go Client · Build Status

This repository provides functionality for interacting with Sajari APIs.

Installation

If you haven't setup Go before, you need to first set a GOPATH (see https://golang.org/doc/code.html#GOPATH).

To fetch and build the code:

$ go get code.sajari.com/sajari-sdk-go/...

This will also build the command line tools (in particular query, csv-importer, schema and pipeline which can be used to interaction with Sajari collections) into $GOPATH/bin (assumed to be in your PATH already).

Getting Started

package main

import (
	"log"

	"golang.org/x/net/context"

	"code.sajari.com/sajari-sdk-go"
)

func main() {
	creds := sajari.KeyCredentials("<key-id>", "<key-secret>")
	client, err := sajari.New("<project>", "<collection>", sajari.WithCredentials(creds))
	if err != nil {
		log.Fatalf("error creating client: %v", err)
	}
	defer client.Close()

	// Initialise the "website" pipeline. Ideal starting point for querying
	// website collections.
	pipeline := client.Pipeline("website")

	// Setup parameters for the search.
	values := map[string]string{
		"q":              "search terms",
		"resultsPerPage": "10",
	}

	// Setup tracking for the search results.
	t := sajari.Tracking{
		Type:  sajari.TrackingClick, // Enable click tracking.
		Field: "url",         // Use the url field for identifying records.
	}

	// Perform the search.
	resp, _, err := pipeline.Search(context.Background(), values, t)
	if err != nil {
		log.Fatalf("error performing search: %v", err)
	}

	for _, r := range resp.Results {
		log.Printf("Values: %v", r.Values)
		log.Println("Tokens: %v", r.Tokens)
	}
}

Documentation

Overview

Package sajari provides functionality for interacting with Sajari APIs.

Index

Constants

View Source
const (
	// BodyField is the name of the internal field which is used to represent the
	// record body.
	BodyField = "_body"

	// IDField is the name of the internal identifier field which is added to
	// each record.
	IDField = "_id"
)

Field constants for internal fields. NB: All field names prefixed with _ (underscore) are reserved for internal use.

Variables

View Source
var DefaultAddTransforms = []Transform{
	SplitStopStemIndexedFieldsTranform,
}

DefaultAddTransforms is the default list of transforms which are used when adding records.

View Source
var ErrNoSuchRecord = errors.New("sajari: no such record")

ErrNoSuchRecord is returned when a requested record cannot be found.

Functions

This section is empty.

Types

type Aggregate

type Aggregate interface {
	// contains filtered or unexported methods
}

Aggregate is an interface which is implemented by all aggregate types in this package.

func AvgAggregate

func AvgAggregate(field string) Aggregate

AvgAggregate computes the average value of a numeric field over a result set.

func BucketAggregate

func BucketAggregate(bs ...Bucket) Aggregate

BucketAggregate is an aggregate which counts records which fall into a defined set of buckets.

func CountAggregate

func CountAggregate(field string) Aggregate

CountAggregate is an aggregate which counts unique field values.

func MaxAggregate

func MaxAggregate(field string) Aggregate

MaxAggregate computes the maximum value of a numeric field over a result set.

func MinAggregate

func MinAggregate(field string) Aggregate

MinAggregate computes the minimum value of a numeric field over a result set.

func SumAggregate

func SumAggregate(field string) Aggregate

SumAggregate computes the sum of a numeric field over a result set.

type Body

type Body struct {
	// Text to search for.
	Text string

	// Weight (significance) of text.
	Weight float64
}

Body is weighted free text.

type Bucket

type Bucket struct {
	// Name of the bucket.
	Name string

	// Filter that records must satisfy to be included in the bucket.
	Filter Filter
}

Bucket is a container used to classify records in a result set (see BucketAggregate). A record is included in the bucket if it satisfies the filter.

type BucketResponse

type BucketResponse struct {
	// Name of the bucket.
	Name string

	// Number of records.
	Count int
}

type BucketsResponse

type BucketsResponse map[string]BucketResponse

BucketsResponse is a type returned from a query performing bucket aggregate.

type Client

type Client struct {
	Project    string
	Collection string

	ClientConn *grpc.ClientConn
	// contains filtered or unexported fields
}

Client is a type which makes requests to the Sajari Engine.

func New

func New(project, collection string, opts ...Opt) (*Client, error)

New creates a new Client which can be used to make requests to Sajari services.

func (*Client) Add

func (c *Client) Add(ctx context.Context, r Record, ts ...Transform) (*Key, error)

Add adds a record to a collection, returning a key which can be used to retrieve the record. If no transforms are specified then DefaultAddTransforms is used.

func (*Client) AddMulti

func (c *Client) AddMulti(ctx context.Context, rs []Record, ts ...Transform) ([]*Key, error)

AddMulti adds records to the underlying collection, returning a list of Keys which can be used to retrieve the respective record. If any of the adds fail then a MultiError will be returned with errors set in the respective indexes. If no transforms are specified then DefaultAddTransforms is used.

func (*Client) Close

func (c *Client) Close() error

Close releases all resources held by the Client.

func (*Client) Delete

func (c *Client) Delete(ctx context.Context, k *Key) error

Delete removes the record identified by key k.

func (*Client) DeleteMulti

func (c *Client) DeleteMulti(ctx context.Context, ks []*Key) error

DeleteMulti removes the records identified by the keys k. Returns non-nil error if there was a communication problem, but fails silently if any key doesn't have a corresponding record.

func (*Client) Exists

func (c *Client) Exists(ctx context.Context, k *Key) (bool, error)

Exists returns true iff a record identified by k exists or non-nil error if there was a problem completing the request.

func (*Client) ExistsMulti

func (c *Client) ExistsMulti(ctx context.Context, k []*Key) ([]bool, error)

ExistsMulti checks whether records identified by keys exist. Returns a slice of bool values indiciating the existence of each.

func (*Client) Get

func (c *Client) Get(ctx context.Context, k *Key) (Record, error)

Get returns the record identified by the Key.

func (*Client) GetMulti

func (c *Client) GetMulti(ctx context.Context, k []*Key) ([]Record, error)

GetMulti retrieves the records identified by the keys k.

func (*Client) Learn

func (c *Client) Learn(ctx context.Context, k *Key, r Request, count int, score float32) error

Learn takes a record identified by k and a query request r and applies pos and neg weighting to the intersections of r and the record.

func (*Client) LearnMulti

func (c *Client) LearnMulti(ctx context.Context, ks []*Key, r Request, counts []int, scores []float32) error

Learn takes a list of records identified by keys ks and a query request r and applies pos and neg weighting to the intersections of r and the record.

func (*Client) Mutate

func (c *Client) Mutate(ctx context.Context, k *Key, m ...FieldMutation) error

Patch makes changes to a record identified by k in-place (where possible). To remove a value set the corresponding field name to nil in values map.

func (*Client) MutateMulti

func (c *Client) MutateMulti(ctx context.Context, rms ...RecordMutation) error

func (*Client) Pipeline

func (c *Client) Pipeline(name string) *Pipeline

Pipeline returns a Pipeline for querying a collection.

func (*Client) Query

func (c *Client) Query() *Query

Query returns a handler for running queries using the Client.

func (*Client) Schema

func (c *Client) Schema() *Schema

Schema returns the schema (list of fields) for the collection.

type CountResponse

type CountResponse map[string]int

CountResponse is a type returned from a query which has performed a count aggregate.

type Credentials

type Credentials interface {
	// contains filtered or unexported methods
}

Credentials is an interface which is implemented by types providing credential information used in requests.

func KeyCredentials

func KeyCredentials(keyID, keySecret string) Credentials

KeyCredentials defines a Credential which uses a Key ID-Secret pair.

type FeatureFieldBoost

type FeatureFieldBoost struct {
	// contains filtered or unexported fields
}

FeatureFieldBoost is a wrapper which turns a FieldBoost into a feature. See NewFeatureFieldBoost.

func NewFeatureFieldBoost

func NewFeatureFieldBoost(b FieldBoost, value float64) FeatureFieldBoost

FeatureFieldBoost uses the normalised form of the FieldBoost b (computed internally) to count for a portion (value between 0 and 1) of the overall record score.

type FeatureQuery

type FeatureQuery struct {
	// FieldBoosts is a list of FeatureFieldBoosts which contribute
	// to the feature score of a record in a query.
	FieldBoosts []FeatureFieldBoost
}

FeatureQuery is a feature-based query which contributes to the scoring of records.

type Field

type Field struct {
	// Name is the name used to identify the field.
	Name string

	// Description is a description of the field.
	Description string

	// Type defines the type of the field.
	Type Type

	// Repeated indicates that this field can hold a list of values.
	Repeated bool

	// Required indicates that this field should always be set on all records.
	Required bool

	// Indexed indicates that the field should be indexed.  This is only valid for
	// String or StringArray fields (see TypeString, TypeStringArray).
	Indexed bool

	// Unique indicates that the field is unique (and this will
	// be encoforced when new records are added).  Unique fields can
	// be used to retrieve/delete records.
	Unique bool
}

Field represents a meta field which can be assigned in a collection record.

type FieldBoost

type FieldBoost interface {
	// contains filtered or unexported methods
}

FieldBoost is an interface satisfied by field-based boosting types.

FieldBoosts are a way to influence the scoring of a record during a search based on the record's field data. All boosts are all normalised internally (converted to a number between 0 and 1).

func ElementFieldBoost

func ElementFieldBoost(field string, elts []string) FieldBoost

ElementFieldBoost represents an element-based boosting for repeated field values.

The resulting boost is the proportion of elements in elts that are also in the field value.

func FilterFieldBoost

func FilterFieldBoost(f Filter, value float64) FieldBoost

FilterFieldBoost is a boost which is applied to records which satisfy the filter. Value must be greater than 0. Records which match the filter will receive a boost of Value

func IntervalFieldBoost

func IntervalFieldBoost(field string, values ...IntervalPoint) FieldBoost

IntervalFieldBoost represents an interval-based boost for numeric field values.

An interval field boost is defined by a list of points with corresponding boost values. When a field value falls between between two IntervalPoints.Point values is computed linearly.

func TextFieldBoost

func TextFieldBoost(field string, text string) FieldBoost

TextFieldBoost represents a text-based boosting for string fields.

It compares the text gainst the record field using a bag-of-words model.

type FieldMutation

type FieldMutation interface {
	// contains filtered or unexported methods
}

FieldMutation is an interface satisfied by all field mutations defined in this package.

func SetField

func SetField(field string, value interface{}) FieldMutation

SetField is a FieldMutation which sets field to value. If value is nil then this unsets field.

func SetFields

func SetFields(m map[string]interface{}) []FieldMutation

SetFields converts the map of field-value pairs into field mutations for use in Mutate.

type Filter

type Filter interface {
	// contains filtered or unexported methods
}

Filter is an interface satisified by filter types that can defined in this package.

func AllFilters

func AllFilters(filters ...Filter) Filter

AllFilters returns a filter which matches documensts that satisfy all of the supplied filters. Equivalent to AND.

func AnyFilter

func AnyFilter(filters ...Filter) Filter

AnyFilter returns a filter which matches records that satisfy any of the given filters. Equivalent to OR.

func FieldFilter

func FieldFilter(fieldOp string, value interface{}) Filter

FieldFilter creates a field-based filter. The fieldOp argument must be a field name followed by optional space and then one of "=", "!=", ">", ">=" "<", "<=", "~" (contains), "!~" (does not contain), "^" (prefix) or "$" (suffix).

Filter which matches records where the field 'url' begins with "https://www.sajari.com":

FieldFilter("url ^", "https://www.sajari.com")

Filter which matches records where the field 'name' contains "Sajari":

FieldFilter("name ~", "Sajari")

Filter which matches records where the field 'count' is greater than or equal to 10:

FieldFilter("count >=", 10)

func GeoFilter

func GeoFilter(fieldLat, fieldLng string, lat, lng, radius float64, region GeoFilterRegion) Filter

GeoFilter is a geo-based boost for records with numeric fields containing latitude/longitude.

// Construct a geo-filter on fields "lat" and "lng" which define a location
// within 10km of Sydney (33.8688° S, 151.2093° E).
GeoFilter("lat", "lng", -33.8688, 151.2093, 10.00, GeoFilterInside)

func NoneOfFilters

func NoneOfFilters(filters ...Filter) Filter

NoneOfFilters returns a filter which matches records that do not satisfy any of the given filters. Equivalent to NAND.

func OneOfFilters

func OneOfFilters(filters ...Filter) Filter

OneOfFilters returns a filter which matches records that satisfy only one of the given filters. Equivalent to XOR.

type GeoFilterRegion

type GeoFilterRegion int

GeoFilterRegion is an enumeration of region values for specifying regions in GeoFilters

const (
	// GeoFilterInside is used to configure a geo filter to be
	// applied to all points within the radius.
	GeoFilterInside GeoFilterRegion = iota

	// GeoFilterOutside is used to contigure a geo boost to be
	// applied to all points outside the radius.
	GeoFilterOutside
)

Constants for use with GeoFilter.Region.

type IndexQuery

type IndexQuery struct {
	// Text is the default body (free-text) of the query.
	// If set this has weight 1.0.
	Text string

	// Body is a list of weighted free-text.
	Body []Body

	// Terms is a list of pre-split terms.
	Terms []Term

	// FieldBoosts to be applied to the index score.
	FieldBoosts []FieldBoost

	// InstanceBoosts to be applied (change scoring of records).
	InstanceBoosts []InstanceBoost
}

IndexQuery is a query run against

type InstanceBoost

type InstanceBoost interface {
	// contains filtered or unexported methods
}

InstanceBoost is an interface satisfied by instance-based boosting types defined in this package.

InstanceBoosts are a way to influence the scoring of a record during a search by changing the importance of term instances in indexed records.

func FieldInstanceBoost

func FieldInstanceBoost(field string, value float64) InstanceBoost

FieldInstanceBoost is a boost applied to index terms which originate in a particular field.

func ScoreInstanceBoost

func ScoreInstanceBoost(minCount int, threshold float64) InstanceBoost

ScoreInstanceBoost is a boost applied to index terms which have interaction data set. For an instance score boost to take effect, the instance must have received at least minCount score updates (i.e. count). If an item is performing as it should then its score will be 1. If the score is below threshold (0 < threshold < 1) then the score will be applied.

type IntervalPoint

type IntervalPoint struct {
	// Point is a field value.
	Point float64

	// Value of boost to assign at this point.
	Value float64
}

IntervalPoint is point-value pair used to construct an IntervalFieldBoost.

It defines the boost value at a particular point in an interval.

type Key

type Key struct {
	// contains filtered or unexported fields
}

Key is a unique identifier for a stored record.

func NewKey

func NewKey(field string, value interface{}) *Key

NewKey creates a new Key with a field and value. Field must be marked as unique in the collection schema.

func (*Key) String

func (k *Key) String() string

String implements Stringer.

type MultiError

type MultiError []error

MultiError can be returned from calls which make multiple requests (see AddMulti, GetMulti, etc).

func (MultiError) Error

func (me MultiError) Error() string

Error implements error.

type Mutation

type Mutation interface {
	// contains filtered or unexported methods
}

Mutation is an interface which is satisfied by schema field mutations.

func IndexedMutation

func IndexedMutation(indexed bool) Mutation

IndexedMutation creates a schema field mutation which changes the indexed property on a field.

func NameMutation

func NameMutation(name string) Mutation

NameMutation creates a schema field mutation which changes the name of a field.

func RepeatedMutation

func RepeatedMutation(repeated bool) Mutation

RepeatedMutation creates a schema field mutation which changes the repeated property on a field.

func RequiredMutation

func RequiredMutation(required bool) Mutation

RequiredMutation creates a schema field mutation which changes the required constraint on a field.

func TypeMutation

func TypeMutation(ty Type) Mutation

TypeMutation creates a schema field mutation which changes the type of a field.

func UniqueMutation

func UniqueMutation(unique bool) Mutation

UniqueMutation creates a schema field mutation which changes the unique constraint on a field.

type Opt

type Opt func(c *Client)

Opt is a type which defines Client options.

func WithCredentials

func WithCredentials(c Credentials) Opt

WithCredentials sets the client credentials used in each request.

func WithEndpoint

func WithEndpoint(endpoint string) Opt

WithEndpoint configures the client to use a custom endpoint.

func WithGRPCDialOption

func WithGRPCDialOption(opt grpc.DialOption) Opt

WithGRPCDialOption returns an Opt which appends a new grpc.DialOption to an underlying gRPC dial.

type Pipeline

type Pipeline struct {
	// contains filtered or unexported fields
}

Pipeline is a handler for a named pipeline.

func (*Pipeline) Search

func (p *Pipeline) Search(ctx context.Context, values map[string]string, tracking Tracking) (*Results, map[string]string, error)

Search runs a search query defined by a pipline with the given values and tracking configuration. Returns the query results and returned values (which could have been modified in the pipeline).

type Query

type Query struct {
	// contains filtered or unexported fields
}

Query is a handler which runs queries on a collection.

func (*Query) Analyse

func (q *Query) Analyse(ctx context.Context, k *Key, r Request) ([]string, error)

Analyse returns the list of overlapping terms between the record identified by k and the search search request r.

func (*Query) AnalyseMulti

func (q *Query) AnalyseMulti(ctx context.Context, ks []*Key, r Request) ([][]string, error)

AnalyseMulti performs Analysis on multiple records against the same query request.

func (*Query) Search

func (q *Query) Search(ctx context.Context, r *Request) (*Results, error)

Search performs an engine search with the Request r, returning a set of Results and non-nil error if there was a problem.

type Record

type Record map[string]interface{}

Record is a set of key-value pairs.

func NewRecord

func NewRecord(body string, values map[string]interface{}) Record

NewRecord creates a new Record with the given body and field values.

type RecordMutation

type RecordMutation struct {
	// Key identifies the record to mutate.
	Key *Key

	// FieldMutations to apply to the record.
	FieldMutations []FieldMutation
}

RecordMutation is a mutation to apply to a Record.

type Request

type Request struct {
	// Tracking configuration.
	Tracking Tracking

	// Filter to be applied (exclude records from results).
	Filter Filter

	// IndexQuery defines a query to be run against the search index.
	IndexQuery IndexQuery

	// FeatureQuery
	FeatureQuery FeatureQuery

	// Offset of results to return.
	Offset int

	// Limit is the number of results to return.
	Limit int

	// Ordering applied to results.
	Sort []Sort

	// Fields returned in results, if empty will return all fields.
	Fields []string

	// Aggregates is a set of Aggregates to run against a result set.
	Aggregates map[string]Aggregate

	// Transforms is a list of transforms to be applied to the query before it is run.
	Transforms []Transform
}

Request is an API representation of a search request.

type Result

type Result struct {
	// Values are field values of records.
	Values map[string]interface{}

	// Tokens contains any tokens associated with this Result.
	Tokens map[string]interface{}

	// Score is the overall score of this Result.
	Score float64

	// IndexScore is the index-matched score of this Result.
	IndexScore float64
}

Result is an individual query result.

type Results

type Results struct {
	// Reads is the total number of index values read.
	Reads int

	// TotalResults is the total number of results for the query.
	TotalResults int

	// Time taken to perform the query.
	Time time.Duration

	// Aggregates computed on the query results (see Aggregate).
	Aggregates map[string]interface{}

	// Results of the query.
	Results []Result
}

Results is a collection of results from a Search.

type Schema

type Schema struct {
	// contains filtered or unexported fields
}

Schema provides methods for managing collection schemas. Use Client.Schema to create one for a collection.

func (*Schema) Add

func (s *Schema) Add(ctx context.Context, fs ...Field) error

Add adds Fields to the collection schema.

func (*Schema) Fields

func (s *Schema) Fields(ctx context.Context) ([]Field, error)

Fields returns the fields in the collection.

func (*Schema) MutateField

func (s *Schema) MutateField(ctx context.Context, name string, muts ...Mutation) error

MutateField mutates a field identifier by name. Each mutation is performed in the order in which it is specified. If any fail, then the rest are ignored.

type Sort

type Sort interface {
	// contains filtered or unexported methods
}

Sort is an interface satisfied by all types which produce sort config.

type SortByField

type SortByField string

SortByField defines a sort order using a field.

type Term

type Term struct {
	Value  string  // String representation of the field
	Field  string  // Meta field
	Pos    uint16  // Number of positive interactions
	Neg    uint16  // Number of negative interactions
	Weight float64 // Significance of term
	WOff   uint16  // Word offset
	POff   uint16  // Paragraph offset
}

Term is a scored term.

type Tracking

type Tracking struct {
	// Tracking specifies which kind (if any) tokens should be generated and returned
	// with the query results.
	Type TrackingType

	// QueryID is a unique identifier for a single search query.  In the
	// case of live querying this is defined to be multiple individual queries
	// (i.e. as a user types the query is re-run).
	QueryID string

	// Sequence (i.e. sequential identifier) of this  in the context of a
	// sequence of queries.
	Sequence int

	// Field is the field to be used for adding identifier information to
	// generated tokens (see TrackingType).
	Field string

	// Data are values which will be recorded along with tracking data produced
	// for the request.
	Data map[string]string
}

Tracking configures tracking for a search query.

type TrackingType

type TrackingType string

TrackingType defines different modes of tracking which can be applied to search requests.

const (
	TrackingNone   TrackingType = ""        // No tracking is enabled.
	TrackingClick  TrackingType = "CLICK"   // Click tracking is enabled, Click tokens will be returned with results.
	TrackingPosNeg TrackingType = "POS_NEG" // Positive/negative interaction tokens should be returned with results.
)

TrackingType constants.

type Transform

type Transform string

Transform is a definition of a transformation applied to a Request which is applied before the Request is executed.

const (
	// SplitStopStemIndexedFieldsTransform splits indexed fields into terms,
	// removes stop words and stems what remains.
	// This is the default setting for transforms.
	SplitStopStemIndexedFieldsTranform Transform = "split-stop-stem-indexed-fields"

	// StopStemmerTransform removes stop terms and stems terms.
	StopStemTransform Transform = "stop-stem"

	// SplitIndexFields splits index fields into terms.
	SplitIndexedFieldsTransform Transform = "split-indexed-fields"
)

Transforms commonly used when adding records.

type Type

type Type string

Type defines field data types.

const (
	TypeString    Type = "STRING"
	TypeInteger   Type = "INTEGER"
	TypeFloat     Type = "FLOAT"
	TypeBoolean   Type = "BOOLEAN"
	TypeTimestamp Type = "TIMESTAMP"
)

Directories

Path Synopsis
Package autocomplete provides methods for interacting directly with auto complete models.
Package autocomplete provides methods for interacting directly with auto complete models.
Package bayes provides methods for creating, training and querying bayes models.
Package bayes provides methods for creating, training and querying bayes models.
cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL