storage

package

v0.0.1 Latest Latest Go to latest Published: Jul 15, 2021 License: Apache-2.0 Imports: 12 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/newity/crawler

Documentation ¶

Overview ¶

Google cloud Pub/Sub implementation of Storage

Index ¶

type Badger
- func NewBadger(path string) (*Badger, error)
type Nats
- func NewNats(clusterID, clientID, natsURL string, maxPubAcksInflight int) (*Nats, error)
type PubSub
- func NewPubSub(project string, opts ...option.ClientOption) (*PubSub, error)
type Storage

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Badger ¶

type Badger struct {
	// contains filtered or unexported fields
}

func NewBadger ¶

func NewBadger(path string) (*Badger, error)

func (*Badger) Close ¶

func (b *Badger) Close() error

func (*Badger) Delete ¶

func (b *Badger) Delete(key string) error

func (*Badger) Get ¶

func (b *Badger) Get(key string) ([]byte, error)

Get retrieves data from BadgerDB using key

func (*Badger) GetStream ¶

func (b *Badger) GetStream(key string) (<-chan []byte, <-chan error)

func (*Badger) InitChannelsStorage ¶

func (b *Badger) InitChannelsStorage(channels []string) error

func (*Badger) Put ¶

func (b *Badger) Put(key string, value []byte) error

Put saves value by key

type Nats ¶

type Nats struct {
	Connection stan.Conn
	// contains filtered or unexported fields
}

func NewNats ¶

func NewNats(clusterID, clientID, natsURL string, maxPubAcksInflight int) (*Nats, error)

func (*Nats) Close ¶

func (n *Nats) Close() error

Close stops all running goroutines related to topics.

func (*Nats) Delete ¶

func (n *Nats) Delete(key string) error

Detele does not work for Nats.

func (*Nats) Get ¶

func (n *Nats) Get(topic string) ([]byte, error)

Get reads one message from the topic and closes channel.

func (*Nats) GetStream ¶

func (n *Nats) GetStream(topic string) (<-chan []byte, <-chan error)

GetStream reads a stream of messages from topic and writes them to the channel.

func (*Nats) InitChannelsStorage ¶

func (n *Nats) InitChannelsStorage(channels []string) error

func (*Nats) Put ¶

func (n *Nats) Put(topic string, msg []byte) error

Put stores message to topic.

type PubSub ¶

type PubSub struct {
	// contains filtered or unexported fields
}

func NewPubSub ¶

func NewPubSub(project string, opts ...option.ClientOption) (*PubSub, error)

func (*PubSub) Close ¶

func (p *PubSub) Close() error

Close stops all running goroutines related to topics.

func (*PubSub) Delete ¶

func (p *PubSub) Delete(key string) error

Detele deletes topic and subscription specified by key.

func (*PubSub) Get ¶

func (p *PubSub) Get(topic string) ([]byte, error)

Get reads one message from the topic and closes channel.

func (*PubSub) GetStream ¶

func (p *PubSub) GetStream(topic string) (<-chan []byte, <-chan error)

GetStream reads a stream of messages from topic and writes them to the channel.

func (*PubSub) InitChannelsStorage ¶

func (p *PubSub) InitChannelsStorage(channels []string) error

func (*PubSub) Put ¶

func (p *PubSub) Put(topic string, msg []byte) error

Put stores message to topic.

type Storage ¶

type Storage interface {
	// init storage (initial setup of storage and connection create operations)
	InitChannelsStorage(channels []string) error
	// put value by key
	Put(key string, value []byte) error
	// get data from storage by specified key
	Get(key string) ([]byte, error)
	// get channel with some data from storage (for message broker storage implementations)
	GetStream(key string) (<-chan []byte, <-chan error)
	// remove parser.Data from storage
	Delete(key string) error
	// close connection to storage (network connections, file descriptors, goroutines)
	Close() error
}

Storage interface is a contract for storage implementations

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL