ws-collector

ws-collector

Simply put, ws-collector is a WebSocket-based ingestion bridge for configurable message queue back-end. It supports ingestion for any Websockets compatable publisher.

Supports:

• Any JSON-based message format
• Token-based client authentication
• Supports Apache Kafka, RabbitMQ, MQTT

Installation

You can install ws-collector by either cloning this repo (below) or by downloading the latest binary distribution.

git clone git@github.com:mchmarny/ws-collector.git
cd ./ws-collector

To start the server simply invoke the ws-collector executable

./ws-collector

Configuration

The ws-collector comes with a configuration file located in the root of the application (config.json). You can use that file to thange the server and backend configuration parameters:

{
   "id": "c1",
   "trace": true,
   "server": {
    "root": "/ws",
    "host": "127.0.0.1",
    "port": 8080,
    "token":""
    },
   "backend": {
    "type":"kafka",
    "uri":"localhost:9092",
    "args": {
        ...
    }
   }
}

Note, when runtime is Cloud Foundry the following configuration attributes are going to be set to CF environment variables:

id = VCAP_APPLICATION.instance_id + VCAP_APPLICATION.instance_index
server.port = VCAP_APPLICATION.port
server.host = VCAP_APPLICATION.host
backend.uri = VCAP_SERVICES[x].credentials.uri

Backends

Currently ws-collector supports two MQ backends:

• RabbitMQ
• Apache Kafka
• MQTT

RabbitMQ

"service_name": "queue",
"exchange": "messages",
"exchange_type": "direct",
"route": "messages"

A sample of the RabbitMQ backend arguments can be foound in config-amqp-sample.json file

Apache Kafka

"topic": "jobs",
"partition": 0,
"buffer_time_in_sec": 3,
"timeout_in_sec": 3

A sample of the Kafka backend arguments can be foound in config-kafka-sample.json file

Apache Kafka

"topic": "jobs",
"qos": 0

A sample of the Kafka backend arguments can be foound in config-mqtt-sample.json file

License

This project is under the MIT License. See the LICENSE file for the full license text.