binlogo

Binlogo

中文 | English

Binlogo is the distributed, visualized application based on MySQL binlog. In short, binlogo is to process the data changes of MySQL into easily understand messages and output to different systems according to the user's configuration. Here are part of advantages:

• Distributed, multi node improves availability of the whole system.
• Visualization, can complete common operations and observe the status of the whole cluster in the control website
• It can be output to multiple queues or other applications, and new outputs are constantly added

Get Started

• Install etcd. Binlogo relies on etcd, so you must install etcd first.

• Install binlogo. Binlogo's download address: Download Address

• Message Format： Data format of binlogo output

• Start binlogo.

  • Edit config. ${binlogo-path}/etc/binlogo.yaml

    

  • $ ./binlogo server --config ./etc/binlogo.yaml

• Open up your browser to http://127.0.0.1:9999/console to view the console website

• Create Pipeline: config.go

Follow the steps.

• Run pipeline.

Click button to run the pipeline instance.

• Operation condition.

You can see the operation condition of pipeline.

• See the output

Insert some into mysql, watch the ouput on stdout.

• Configuration output to Kafka

• High performance, possible data loss.
  • acks=1
  • enable.idempotence=false
  • compression.type=snappy
  • retries=0
• For reliability performance:
  • acks=-1
  • enable.idempotence=true
  • retries=3 or larger one

Docker

• Docker Hub

$ docker pull jin06/binlogo
$ docker run -e "ETCD_ENDPOINTS=172.17.0.3:2379" --name BinlogoNode -it -d -p 9999:9999 jin06/binlogo:latest
Open browser access http://127.0.0.1:9999/console
I started five nodes with docker. The following is a screenshot

Kubernetes

• doc

Other outputs

• HTTP
• RabbitMQ
• Kafka
• Redis
• AliCloud RocketMQ

Docs

• docs link

Questions

• To Report bug: GitHub Issue
• Contact author: jlonmyway@gmail.com