gizmo: github.com/NYTimes/gizmo/pubsub Index | Files | Directories

package pubsub

import "github.com/NYTimes/gizmo/pubsub"

Package pubsub contains two generic interfaces for publishing data to queues and subscribing and consuming data from those queues.

// Publisher is a generic interface to encapsulate how we want our publishers
// to behave. Until we find reason to change, we're forcing all publishers
// to emit protobufs.
type Publisher interface {
    // Publish will publish a message.
    Publish(ctx context.Context, key string, msg proto.Message) error
    // Publish will publish a []byte message.
    PublishRaw(ctx context.Context, key string, msg []byte) error
}

// Subscriber is a generic interface to encapsulate how we want our subscribers
// to behave. For now the system will auto stop if it encounters any errors. If
// a user encounters a closed channel, they should check the Err() method to see
// what happened.
type Subscriber interface {
    // Start will return a channel of raw messages
    Start() <-chan SubscriberMessage
    // Err will contain any errors returned from the consumer connection.
    Err() error
    // Stop will initiate a graceful shutdown of the subscriber connection
    Stop() error
}

Where a `SubscriberMessage` is an interface that gives implementations a hook for acknowledging/delete messages. Take a look at the docs for each implementation in `pubsub` to see how they behave.

There are currently 3 implementations of each type of `pubsub` interfaces:

For pubsub via Amazon's SNS/SQS, you can use the `pubsub/aws` package.

For pubsub via Google's Pubsub, you can use the `pubsub/gcp` package.

For pubsub via Kafka topics, you can use the `pubsub/kafka` package.

For publishing via HTTP, you can use the `pubsub/http` package.

Index

Package Files

doc.go pubsub.go

Variables

var Log = logrus.New()

Log is the structured logger used throughout the package.

type MultiPublisher Uses

type MultiPublisher interface {
    Publisher

    // PublishMulti will publish multiple messages with a context.
    PublishMulti(context.Context, []string, []proto.Message) error
    // PublishMultiRaw will publish multiple raw byte array messages with a context.
    PublishMultiRaw(context.Context, []string, [][]byte) error
}

MultiPublisher is an interface for publishers who support sending multiple messages in a single request, in addition to individual messages.

type Publisher Uses

type Publisher interface {
    // Publish will publish a message with context.
    Publish(context.Context, string, proto.Message) error
    // Publish will publish a raw byte array as a message with context.
    PublishRaw(context.Context, string, []byte) error
}

Publisher is a generic interface to encapsulate how we want our publishers to behave. Until we find reason to change, we're forcing all pubslishers to emit protobufs.

type Subscriber Uses

type Subscriber interface {
    // Start will return a channel of raw messages.
    Start() <-chan SubscriberMessage
    // Err will contain any errors returned from the consumer connection.
    Err() error
    // Stop will initiate a graceful shutdown of the subscriber connection.
    Stop() error
}

Subscriber is a generic interface to encapsulate how we want our subscribers to behave. For now the system will auto stop if it encounters any errors. If a user encounters a closed channel, they should check the Err() method to see what happened.

type SubscriberMessage Uses

type SubscriberMessage interface {
    Message() []byte
    ExtendDoneDeadline(time.Duration) error
    Done() error
}

SubscriberMessage is a struct to encapsulate subscriber messages and provide a mechanism for acknowledging messages _after_ they've been processed.

Directories

PathSynopsis
aws
gcp
http
kafka
pubsubtestPackage pubsubtest provides a publisher and a subscriber intended for use in tests.

Package pubsub imports 4 packages (graph) and is imported by 41 packages. Updated 2019-03-12. Refresh now. Tools for package owners.