event-gateway

⚠ Still under development, it didn't reach v1.0.0 and therefore is not suitable for production use yet.

Overview

AsyncAPI Event Gateway (name is subject to change) is the Event Gateway solution by excellence.

Based on traditional API Gateways, it intercepts all incoming messages moving them into a pipeline of middlewares and handlers such as:

• Message validation
• Message manipulation
• Message aggregation
• Message filtering
• Authentication
• Throttling
• Routing
• Monitoring (including tracing)

It supports all the protocols AsyncAPI supports through bindings.

This Event Gateway is also compatible with the HTTP protocol, natively or through an external provider like Krakend.io.

Goals

1. Performance-first while ensuring message delivery

The Event Gateway it's a stateless solution that ensures messages are delivered as fast as possible using a minimal resource footprint. Delivering messages as a top priority means no data loss should happen.

2. Transparent usage.

No change in the user's code is needed. The service acts as a proxy between the client and the final broker(s). Messages infer the protocol based on the shape of the input network packet.

3. Fully configurable.

The service is entirely configurable, and the user can specify the settings for all protocols as well. For example, consumers' and producers' settings.

4. API-first

The service provides an API for uploading AsyncAPI specs, allowing the user to update their message validation, among others, very quickly. It could even be an automated task whenever you update your specs.

5. Extensible

The Event Gateway can extend its functionality via middlewares written by the community. A catalog of middlewares made by the community is also available.

Roadmap

The idea is to keep iterating and support all the protocols AsyncAPI supports through bindings.
However, we reduced the scope for the first versions, so we can give support to the most used protocols.

For the first version, only Kafka protocol will be supported.

Demo

A demo of the Event Gateway has been deployed and it is completely available to the users. It is a very limited demo environment, but it is a good starting point for the users to familiarize on the concept.

Note that as per today, only the Kafka protocol is supported. More info at docs/config/kafka.md. The AsyncAPI file used for the demo is available here. You can also open it with Studio.

This video introduces this demo application:

Main things you can do now:

• Produce Kafka messages to the Kafka broker event-gateway-demo.asyncapi.com:20472 and topic event-gateway-demo. The expected message payload (lightMeasured) is:

  type: object
  properties:
    lumens:
      type: integer
      minimum: 0
      description: Light intensity measured in lumens.
    sentAt:
      type: string
      format: date-time
      description: Date and time when the message was sent.
  

  You can use kcat as the Kafka producer (headers are totally optional):

  echo '{"lumens": 100}' | kcat -H test=true -H origin=readme -H time=(date) -b event-gateway-demo.asyncapi.com:20472 -t event-gateway-demo -P
  

• Consume Kafka messages from the Kafka broker event-gateway-demo.asyncapi.com:20472 and topic event-gateway-demo. You can use kcat as the Kafka consumer (headers are totally optional):

  kcat -b event-gateway-demo.asyncapi.com:20472 -t event-gateway-demo -C
  

• Watch for the messages via the websocket endpoint at ws://event-gateway-demo.asyncapi.com:5000/ws. You can use Websocat to connect to the websocket:

  websocat -v ws://event-gateway-demo.asyncapi.com:5000/ws
  

  This websocket endpoint consumes from another Kafka topic where the original produced messages are forwarded.

  In the case of producing messages with an invalid payload (payloads that don't validate against the schema above), an extra metadata field (Kafka header under the hood) named _asyncapi_eg_validation_error will be included in the message. For example:

  "_asyncapi_eg_validation_error": "{\"ts\":\"2021-12-20T11:33:26.583143572Z\",\"errors\":[\"lumens: Invalid type. Expected: integer, given: boolean\"]}",
  

Getting Started

Install from Docker

TBD

Install from pre-compiled binaries

TBD

Install from source

This project is built with Go, and it uses Go Modules for managing dependencies.
The Minimum required version of Go is set in go.mod file.

1. Clone this repository.
2. Run make build. The binary will be placed at bin/out/event-gateway.

Configuration

Please refer to config reference.

Contributing

Read CONTRIBUTING guide.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Sergio Moya 💬 🐛 💻 📖 🤔 🚇 🚧 📆 🔬 👀 ⚠️

Fran Méndez 🤔 👀

Maciej Urbańczyk 👀

Lukasz Gornicki 👀

Paul B. 👀

Jonas Lagoni 👀

This project follows the all-contributors specification. Contributions of any kind welcome!