logtalez
Problem Statement
We want to tail logs from remote servers as conveniently as if they were local, in a safe and secure manner.
Solution
logtalez - a library and command line client for subscribing to log streams from rsyslog using the omczmq output plugin..
- Create dynamic topics using rsyslog's parsing and template features.
- Subscribe to topics to receive the logs you want.
- Publisher side filtering keeps bandwidth usage low.
- Brokerless design keeps operation simple.
- Ephemeral streaming keeps things light weight.
- CurveZMQ authentication and encryption keeps things secure.
Installation
Dependencies
Version: 1.0.11 (or newer)
Sodium is a "new, easy-to-use software library for encryption, decryption, signatures, password hashing and more". ZeroMQ uses sodium for the basis of the CurveZMQ security protocol.
Version: 4.2.0 (or newer)
ZeroMQ is an embeddable ZMTP protocol library.
Version: 4.0.1 (or newer)
CZMQ is a high-level C binding for ZeroMQ. It provides an API for various services on top of ZeroMQ such as authentication, actors, service discovery, etc.
GoCZMQ is a Go interface to the CZMQ API.
Version: 8.9.0 or newer
Rsyslog is the "rocket fast system for log processing".
You will need to use the "--enable-omczmq" configure flag to build zeromq + curve support.
Generating Certificates
logtalez uses CURVE security certificates generated by the zcert API. They are stored in ZPL format. Logtalez includes a simple cert generation tool (curvecertgen) for convenience.
To generate a public / private key pair:
$ ./curvecertgen bogus_cert
Name: Brian
Email: bogus@whatever.com
Organization: Bogus Org
Version: 1
The above would generate a bogus_cert and bogus_cert_secret file.
Configuring Your Rsyslog Server
The following rsyslog configuration snippet consists of:
- A template that dynamically sets a "topic" on a message consisting of hostname.syslogtag + an "@cee" cookie and JSON message payload
- A rule snippet that attempts to parse a syslog message as JSON, then outputs it over a zeromq publish socket using the template
module(load="mmjsonparse")
module(load="omczmq")
template(name="pubsub_host_tag" type="list") {
property(name="hostname")
constant(value=".")
property(name="syslogtag" position.from="1" position.to="32")
constant(value="@cee:")
constant(value="{")
constant(value="\"@timestamp\":\"")
property(name="timereported" dateFormat="rfc3339" format="json")
constant(value="\",\"host\":\"")
property(name="hostname")
constant(value="\",\"severity\":\"")
property(name="syslogseverity-text")
constant(value="\",\"facility\":\"")
property(name="syslogfacility-text")
constant(value="\",\"syslogtag\":\"")
property(name="syslogtag" format="json")
constant(value="\",")
property(name="$!all-json" position.from="2")
}
ruleset(name="zmq_pubsub_out") {
action(
name="zmq_pubsub"
template="pubsub_host_tag"
type="omczmq"
endpoints="tcp://*:24444"
socktype="PUB"
authtype="CURVESERVER"
clientcertpath="/etc/curve.d/"
servercertpath="/etc/curve.d/my_server_cert"
)
}
action(type="mmjsonprase")
if $parsesuccess == "OK" then {
call zmq_pubsub_out
}
Usage
import "github.com/digitalocean/logtalez"
func main() {
endpoints := []string{"tcp://127.0.0.1:24444,tcp://example.com:24444"}
topics = []string{"host1.nginx","host2.nginx","host3.nginx"}
serverCert := "/home/me/.curve/server_public_cert"
clientCert := "/home/me/.curve/client_public_cert"
lt, err := logtalez.New(endpoints, topics, serverCert, clientCert)
if err != nil {
panic(err)
}
buf := make([]byte, 65560)
for {
n, err := lt.Read(buf)
if err != nil && err != io.EOF {
panic(err)
}
fmt.Println(string(buf[:n]))
}
}
- jq JSON processor
- humanlog "Logs for humans to read."
- Anything that can read stdout!
GoDoc
godoc
License
This project uses the MPL v2 license, see LICENSE