kafka

Kafka Pubsub using Watermill Library

An example of implementing an event-driven application using pubsub pattern with kafka as the message broker. We leverage the Watermill framework to set up our connection to kafka.

Overview

The code in this example is based on the Kafka example found in the Watermill project with the following differences:

• Our example uses bitnami Kafka and Zookeeper docker images instead of Confluence docker images.
• Our go program runs outside of Docker network, hence the broker URL points to localhost.
• The code for sending and receiving Kafka messages are implemented as goroutines.

Setup

1. Run Kafka and Zookeeper.

   $ docker-compose up
   

2. Run the go program.

   $ go run main.go
   

3. Terminate the programs.

   $ # Press CTRL-C to terminate the go program
   $ # Next shutdown kafka and zookeeper
   $ docker-compose down
   

4. Alternatively, you can run everything with just 2 commands.

   $ make run
   $ make teardown    # Run this to remove the containers
   

Reference and Credits

• Bitnami Kafka Docker Image
• Bitnami Zookeeper Docker Image
• Confluence Kafka Docker Image
• Confluence Zookeeper Docker Image
• Watermill Homepage
• Watermill Github Project