kubemq-bridges

KubeMQ Bridges

KubeMQ Bridges bridge, replicate, aggregate, and transform messages between KubeMQ clusters no matter where they are, allowing to build a true cloud-native messaging single network running globally.

Key Features:

• Runs anywhere - Kubernetes, Cloud, on-prem , anywhere
• Stand-alone - small docker container / binary
• Build Any Topology - connects KubeMQ clusters in 1:1, 1:n , n:1, n:n
• Middleware Supports - Logs, Metrics, Retries and Rate Limiters
• Easy Configuration - simple yaml file builds your topology

An example of a use case:

Concept

KubeMQ Bridges' concept is bridging between sources and targets, thus Bindings.

Binding can be any source kinds to any target kinds, as shown below:

KubeMQ Bridges can support any binding topology :

Bridge

See an example

Replicate

See an example

Aggregate

See an example

Transform

See an example

Installation

Kubernetes

An example of kubernetes deployment can be find below:

apiVersion: core.k8s.kubemq.io/v1alpha1
kind: KubemqConnector
metadata:
  name: kubemq-bridges
spec:
  type: bridges
  replicas: 1
  config: |-
    bindings:
      - name: bridges-example-binding
        properties:
          log_level: "info"
        sources:
          kind: source.events
          name: cluster-sources
          connections:
            - address: "kubemq-cluster-grpc:50000"
              client_id: "cluster-events-source"
              channel: "events.source"
              group:   ""
              concurrency: "1"
              auto_reconnect: "true"
              reconnect_interval_seconds: "1"
              max_reconnects: "0"
        targets:
          kind: target.events
          name: cluster-targets
          connections:
            - address: "kubemq-cluster-grpc:50000"
              client_id: "cluster-events-target"
              channels: "events.target"

Binary (Cross-platform)

Download the appropriate version for your platform from KubeMQ Bridges Releases. Once downloaded, the binary can be run from anywhere.

Ideally, you should install it somewhere in your PATH for easy use. /usr/local/bin is the most probable location.

Running KubeMQ Bridges

kubemq-bridges --config config.yaml

Windows Service

1. Download the Windows version from KubeMQ Bridges Releases. Once downloaded, the binary can be installed from anywhere.
2. Create config.yaml configuration file and save it to the same location of the Windows binary.

Service Installation

Run:

kubemq-bridges.exe --service install

Service Installation With Username and Password

Run:

kubemq-bridges.exe --service install --username {your-username} --password {your-password}

Service UnInstall

Run:

kubemq-bridges.exe --service uninstall

Service Start

Run:

kubemq-bridges.exe --service start

Service Stop

Run:

kubemq-bridges.exe --service stop

Service Restart

Run:

kubemq-bridges.exe --service restart

NOTE: When running under Windows service, all logs will be emitted to Windows Events Logs.

Configuration

KubeMQ Bridges loads configuration file on startup. The configuration file is a yaml file that contains definitions for bindings of Sources and Targets.

The default config file name is config.yaml, and KubeMQ bridges search for this file on loading.

Structure

Config file structure:

apiPort: 8080 # kubemq bridges api and health end-point port
bindings:
  - name: clusters-sources # unique binding name
    properties: # Bindings properties such middleware configurations
      log_level: error
      retry_attempts: 3
      retry_delay_milliseconds: 1000
      retry_max_jitter_milliseconds: 100
      retry_delay_type: "back-off"
      rate_per_second: 100
    sources:
      kind: source.query # Sources kind
      name: name-of-sources # sources name 
      connections: # Array of connections settings per each source kind
        - .....
    targets:
      kind: target.query # Targets kind
      name: name-of-targets # targets name
      connections: # Array of connections settings per each target kind
        - .....

Build Wizard

KubeMQ Bridges configuration can be build with --build flag

./kubemq-bridges --build

Properties

In bindings configuration, KubeMQ Bridges supports properties setting for each pair of source and target bindings.

These properties contain middleware information settings as follows:

Logs Middleware

KubeMQ Bridges supports level based logging to console according to as follows:

An example for only error level log to console:

bindings:
  - name: sample-binding 
    properties: 
      log_level: error
    sources:
    ......  

Retry Middleware

KubeMQ Bridges supports Retries' target execution before reporting of error back to the source on failed execution.

Retry middleware settings values:

An example for 3 retries with back-off strategy:

bindings:
  - name: sample-binding 
    properties: 
      retry_attempts: 3
      retry_delay_milliseconds: 1000
      retry_max_jitter_milliseconds: 100
      retry_delay_type: "back-off"
    sources:
    ......  

Rate Limiter Middleware

KubeMQ Bridges supports Rate Limiting of target executions.

Rate Limiter middleware settings values:

An example for 100 executions per second:

bindings:
  - name: sample-binding 
    properties: 
      rate_per_second: 100
    sources:
    ......  

Sources

Sources section contains sources configuration for binding as follows:

Targets

Targets section contains target configuration for binding as follows: