README
¶
hydra-go
Golang implementation of hydra which is a library for producing and consuming
messages from multiple subscribed topics on IPFS.
Motivation
There exists many publish-subscribe/message queue system but IPFS has a built in pubsub system. This is a decentralized system that we can exploit to build more complex systems similar to Kafka, RabbitMQ. This project attempts to build some standards around producing and consuming messages from multiple topics hosted on IPFS.
Those standards include:
- Application specific messages:
- JSON
- Protobuf
- Encrypted messages
- AES
- Asymmetric Encryption
- Symmetric Encryption
Testing
In your terminal you can run all tests by using the following:
$ go test
Example usage
In the examples folder there is a simple go program that creates a producer and
consumer with the same topics. This shows how you can use this package to make a
simple project to read and write messages without having to set up any infrastructure
to provide the messaging service.
When you run the program you should see output similar to:
$ go run examples/simple_producer_consumer.go
Consuming messages...
Produced message: 1522692256: [0=test_message_0]
Consumed message: test_message_0
Consumed message: test_message_1
Produced message: 1522692256: [1=test_message_1]
Produced message: 1522692257: [2=test_message_2]
Consumed message: test_message_2
Produced message: 1522692257: [3=test_message_3]
Consumed message: test_message_3
Produced message: 1522692258: [4=test_message_4]
Consumed message: test_message_4
Produced message: 1522692258: [5=test_message_5]
Consumed message: test_message_5
Produced message: 1522692259: [6=test_message_6]
Consumed message: test_message_6
Produced message: 1522692259: [7=test_message_7]
Consumed message: test_message_7
Produced message: 1522692260: [8=test_message_8]
Consumed message: test_message_8
Produced message: 1522692260: [9=test_message_9]
Consumed message: test_message_9
Contributing
Feel free to check out the CONTRIBUTING.md for the guidelines on contributing
to the project.
TODO
- Create standards for message headers (In Progress)
- Create standards for encrypted messages
LICENSE
GPL
Documentation
¶
Index ¶
- Constants
- func NewHeader(key string, value []byte) *message.Header
- func NewMessage(key, value []byte, headers []*message.Header) *message.Message
- type Config
- type Consumer
- func (c *Consumer) Poll() Event
- func (c *Consumer) ReadMessage() (*message.Message, error)
- func (c *Consumer) Start()
- func (c *Consumer) Stop()
- func (c *Consumer) Subscribe(topic string) error
- func (c *Consumer) SubscribeTopics(topics []string) error
- func (c *Consumer) Topics() []string
- func (c *Consumer) Unsubscribe(topic string) error
- func (c *Consumer) UnsubscribeTopics(topics []string) error
- type Error
- type Event
- type IPFSClient
- type Producer
- func (p *Producer) AddTopic(topic string)
- func (p *Producer) AddTopics(topics []string)
- func (p *Producer) Produce(topic string, msg *message.Message) error
- func (p *Producer) ProduceAll(msg *message.Message) error
- func (p *Producer) RemoveTopic(topic string)
- func (p *Producer) RemoveTopics(topics []string)
- func (p *Producer) Topics() []string
Constants ¶
const ( HYDRA_IPFS_READ_RECORD_ERROR = iota + 1 HYDRA_RECORD_UNMARSHAL_ERROR = iota HYDRA_RESPONSE_TIMEOUT_ERROR = iota HYDRA_ZERO_BYTES_RECIEVED_ERROR = iota )
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Config ¶
Config holds the needed values for either a producer or consumer to connect to an IPFS client and a initial set of topics that the producer or consumer are subscribed on creation.
func DefaultConfig ¶
func DefaultConfig() *Config
DefaultConfig returns a hydra config that points to localhost:5001 and has no topics configured to pull from.
type Consumer ¶
type Consumer struct {
// contains filtered or unexported fields
}
Consumer "consumes" all topic messages that it is subscribed to. It provides a high level interface for topic management and pulling messages from IPFS pubsub.
func NewConsumer ¶
func NewConsumer(client IPFSClient, config *Config) (*Consumer, error)
NewConsumer creates a new consumer that is connected to a IPFS client via the configuration passed and also is subscribed to any topics set in the config.
func (*Consumer) ReadMessage ¶
ReadMessage will wait until there is a message from any one of the subscribed topics and return that message.
func (*Consumer) Start ¶
func (c *Consumer) Start()
Start will start the consumption of all messages from the topics that the consumer is subscribed to. This needs to be called before attempting to read any messages.
func (*Consumer) Stop ¶
func (c *Consumer) Stop()
Stop will stop the consumer from reading any more messages from the topics that it is subscribed to.
func (*Consumer) Subscribe ¶
Subscribe add a topic to the consumers list of topics and will allow messages to be consumed from those topics.
func (*Consumer) SubscribeTopics ¶
SubscribeTopics adds a list of topics that the consumer will consume messages from.
func (*Consumer) Topics ¶
Topics returns the current list of topics that the consumer is subscribed to.
func (*Consumer) Unsubscribe ¶
Unsubscribe removes a topic from the list of topics that the conumser is consuming.
func (*Consumer) UnsubscribeTopics ¶
UnsubscribeTopics removes a list of topics from the list of topvs that the consumer is consuming.
type Error ¶
type Error struct {
// contains filtered or unexported fields
}
Error represent a general error returned from the reading or writing of a message to IPFS pubsub.
type Event ¶
type Event interface {
String() string
}
Event defines either an error or message passed from IPFS pubsub.
type IPFSClient ¶
type Producer ¶
type Producer struct {
// contains filtered or unexported fields
}
Producer is a high level message producer that publishes messages to a single or multiple topics on IPFS pubsub.
func NewProducer ¶
func NewProducer(client IPFSClient, config *Config) *Producer
NewProducer creates a new producer connected to a IPFS client specified in the configuration.
func (*Producer) AddTopic ¶
AddTopic will add a topic to the list of topics the producer will publish messages to.
func (*Producer) AddTopics ¶
AddTopics will add a list of topics to the list of topics the producer will publish messages to.
func (*Producer) Produce ¶
Produce will publish a message to a specific topic on IPFS. If the topic provided is not in producers list of subscribed topics it will throw an error.
func (*Producer) ProduceAll ¶
ProduceAll will publish a message to all of the topics that the producer is subscribed to.
func (*Producer) RemoveTopic ¶
RemoveTopic will will remove a topic from the producers list of topics.
func (*Producer) RemoveTopics ¶
RemoveTopics removes a list of topics from the producers list of topics.
Source Files
Directories