kafka

package
v1.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 9, 2019 License: MIT Imports: 11 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Concurrency

func Concurrency(j int) func(*Client)

Concurrency is used to set the size of the search worker pool

func WithDecoder

func WithDecoder(d Decoder) func(*Client)

WithDecoder is used to insert a Decoder plugin

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client fetches from kafka

func New

func New(addrs []string, opts ...Opt) (*Client, error)

New returns a kafka Client.

func (*Client) Close

func (c *Client) Close()

Close disconnects from kafka

func (*Client) Fetch

func (c *Client) Fetch(info Partition, end int64, cb func(string)) error

Fetch gets all messages in a partition up intil the 'end' offset.

func (*Client) GetPartition

func (c *Client) GetPartition(part Partition, end int, f func([]byte) bool) ([]Message, error)

GetPartition fetches a kafka partition. It includes a callback func so that the caller can tell it when to stop consuming.

func (*Client) GetTopic

func (c *Client) GetTopic(topic string) ([]Partition, error)

GetTopic gets a single kafka topic

func (*Client) GetTopics

func (c *Client) GetTopics() ([]string, error)

GetTopics gets topics (duh)

func (*Client) Search

func (c *Client) Search(info Partition, s string, cb func(i, j int64)) (int64, error)

Search is for searching for a string in a single kafka partition. It stops at the first match.

func (*Client) SearchTopic

func (c *Client) SearchTopic(partitions []Partition, s string, firstResult bool, cb func(int64, int64)) ([]Partition, error)

SearchTopic allows the caller to search across all partitions in a topic.

type Decoder

type Decoder interface {
	Decode(topic string, data []byte) ([]byte, error)
}

Decoder is the interface that is required of plugins

type Message

type Message struct {
	Partition Partition `json:"partition"`
	Value     []byte    `json:"msg"`
	Offset    int64     `json:"offset"`
}

Message holds information about a single kafka message

type Opt

type Opt func(*Client)

Opt is a func that sets an attribute on Client

type Partition

type Partition struct {
	Topic     string `json:"topic"`
	Partition int32  `json:"partition"`
	Start     int64  `json:"start"`
	End       int64  `json:"end"`
	Offset    int64  `json:"offset"`
	Filter    string `json:"filter"`
}

Partition holds information about a kafka partition

func (*Partition) String

func (p *Partition) String() string

String turns a partition into a string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL