README
¶
AMQP to Kafka forwarder 
Fetches messages from AMQP and forwards them to Kafka.
Build
Given a properly setup go develop environment, just run
go get -d -t .
go build
The resulting binary should be named amqp-to-kafka (or amqp-to-kafka.exe
on Windows). For further detail, e.g. cross-platform builds and custom build
parameters, please refer to the go-documentation.
Usage
amqp-to-kafka takes the following commandline parameters:
--amqp-uri: URI of the AMQP broker--amqp-queue: name of the queue to forward--amqp-exchange: name of the AMQP exchange to forward messages from--amqp-binding-key: binding key to select forwarded messages--amqp-consumer-tag: consumer tag, see AMQP documentation--kafka-brokers=STRING: list of Kafka brokers used for bootstrapping--kafka-topic=STRING: Kafka topic for outgoing messages--metrics-address=HOST:PORT: Listening address to serve metrics
amqp-uri, amqp-exchange, kafka-brokers and kafka-topic are mandatory,
everything else is optional. When metrics-address is given amqp-to-kafka
binds that address to export prometheus compatible metrics. Parameters may set
via environment variables as well
AMQP_URI: URI of the AMQP brokerAMQP_QUEUE: name of the queue to forwardAMQP_EXCHANGE: name of the AMQP exchange to forward messages fromAMQP_BINDING_KEY: binding key to select forwarded messagesAMQP_CONSUMER_TAG: consumer tag, see AMQP documentationKAFKA_BROKERS: list of Kafka brokers used for bootstrappingKAFKA_TOPIC: Kafka topic for outgoing messagesMETRICS_ADDRESS: Listening address to serve metrics
When both are specified, commandline parameters take precedence over environment variables.
Either amqp-queue or amqp-exchange and amqp-binding-key may be specified.
In case amqp-queue is used all messages to the given queue are forwarded.
In case amqp-exchange and amqp-binding-key are specified a temporary queue
is declared and bound to the exchange using the binding key to select messages
forwarded.
Docker
A docker-ized variant is available
meteogroup/amqp-to-kafka.
Metrics are exposed on port 8080. To run use
docker run -P meteogroup/amqp-to-kafka <additional commandline arguments>
License
Copyright © 2017 MeteoGroup Deutschland GmbH, all the files in this repository are released under the terms of Apache License 2.0.
Documentation
¶
Overview ¶
Copyright © 2017 MeteoGroup Deutschland GmbH
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright © 2017 MeteoGroup Deutschland GmbH ¶
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright © 2017 MeteoGroup Deutschland GmbH ¶
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright © 2017 MeteoGroup Deutschland GmbH ¶
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright © 2017 MeteoGroup Deutschland GmbH ¶
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Source Files