relp

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 12, 2023 License: LGPL-3.0 Imports: 12 Imported by: 0

README

GoDoc

RELP Server Library for the GO programming language.

This package provides RELP server implementation based on https://github.com/rsyslog/librelp specifications. RELP is great protocol used to transfer log without loss. It is easy to understand and embed some security consideration: confidentiality, integrity and authentication using TLS, loss-proof using RELP.

This library doesn't have any other dependencies than standard Go library.

Library support version 0 (experimental) and 1 of the protocol. Note RHEL8-like distribution proposes Rsyslog with version 0 of the RELP protocol.

Known limits

The "starttls" feature is not supported. The RELP C library documentation talk about this command, but it is not implemented in librelp. In other way, it is not implemented in any rsyslog. To use TLS, just use RELP over TLS connection. The Go RELP library embed necessary to handle TLS.

Only server side is implemented. If it have some needs about the client side, we can consider implement it.

Network managment overview

The RELP library propose two ways for the network management:

The developper implements full network stack including TLS if it is useful, he gives only an io.ReadWriter to the library. The caller is responssible to close all connections.

The developper implement only TCP server, it gives net.Conn To the library The library manages TLS is required. The caller is reponsible to close tcp connection.

Consume logs methods overview

The RELP library propose four ways for the output logs management

The caller implements negociation, he receive all messages and decide responses. It also do something on negociation. This mode is too complicated and useless.

The caller receive only logs message, he should sent acquirements. This mode allow to ensure log processing before sending acquirement. for example, the developper can ensure the log line was writen on disk before aquire the log line. By this way if the process crash before writing, the log line keep in client storage.

The caller receive logs as string. The library sent automatically acquirements. This is usefull to write quick code to receive logs, but you cannot deal with acquirements.

The library implement io.ReadCloser interface. you can receive logs as stream. Be prudent with '\n', some logs can contains '\n' and other control character, but '\n' is the log separator in []byte stream. The option StreamEscapeLf replace the control char '\n' by two ascii chars '\\', 'n'.

The library doesn't not implement timeouts because the server never wait for client.

Documentation

Overview

Example (Read)

This is full example using io.ReadCloser interface with RELP library. 

var ln net.Listener
var conn net.Conn
var err error
var opts *relp.Options
var r *relp.Relp

// Create and validate options
opts, err = relp.ValidateOptions(&relp.Options{
	Tls: relp.Opt_tls_disabled,
})
if err != nil {
	log.Fatalf(err.Error())
}

// Create listen connexion
ln, err = net.Listen("tcp", "0.0.0.0:4150")
if err != nil {
	log.Fatalf(err.Error())
}

// accept connection
conn, err = ln.Accept()
if err != nil {
	log.Fatalf(err.Error())
}

defer conn.Close()

// Create RELP from net.Conn
r, err = relp.NewTcp(conn, opts)
if err != nil {
	log.Fatal(err.Error())
}

defer r.Close()

// Use io.ReadCloser interface to copy log from RELP stream to os.Stdout
_, err = io.Copy(os.Stdout, r)
if err != nil {
	log.Fatal(err.Error())
}

Output:
Example (ReceiveLog)

This example show how configure TLS and how to use RELP library making the most of the RELP protocol. 

var ln net.Listener
var conn net.Conn
var err error
var opts *relp.Options
var r *relp.Relp
var msg *relp.Message

// Create and validate options
opts, err = relp.ValidateOptions(&relp.Options{
	Tls:           relp.Opt_tls_connection,
	Certificate:   "ca/certs/server1.crt",
	PrivateKey:    "ca/private/server1.key",
	CACertificate: "ca/certs/myCA.crt",
	Crl:           "ca/rootca.crl",
	CnAcl: []relp.Acl{
		relp.Acl{
			Value:  "client*",
			Action: relp.Acl_accept,
		},
		relp.Acl{
			Value:  "*",
			Action: relp.Acl_reject,
		},
	},
})
if err != nil {
	log.Fatalf(err.Error())
}

// Create listen connexion
ln, err = net.Listen("tcp", "0.0.0.0:4150")
if err != nil {
	log.Fatalf(err.Error())
}

// accept connection
conn, err = ln.Accept()
if err != nil {
	log.Fatalf(err.Error())
}

defer conn.Close()

// Create RELP from net.Conn
r, err = relp.NewTcp(conn, opts)
if err != nil {
	log.Fatal(err.Error())
	return
}

defer r.Close()

// Receive log message loop
for {

	// Receive log message
	msg, err = r.ReceiveLog()
	if err != nil {
		log.Fatal(err.Error())
	}

	// Process log message. Here, just display it
	println(string(msg.Data))

	// Acknoledge message
	err = r.AnswerOk(msg)
	if err != nil {
		log.Fatal(err.Error())
	}
}

Output:
Example (ReceiveLogAndAck)

Show how using a loop of log message with acknoledgments offloaded by the RELP library 

var ln net.Listener
var conn net.Conn
var err error
var opts *relp.Options
var r *relp.Relp
var msg string

// Create and validate options
opts, err = relp.ValidateOptions(&relp.Options{
	Tls: relp.Opt_tls_disabled,
})
if err != nil {
	log.Fatalf(err.Error())
}

// Create listen connexion
ln, err = net.Listen("tcp", "0.0.0.0:4150")
if err != nil {
	log.Fatalf(err.Error())
}

// accept connection
conn, err = ln.Accept()
if err != nil {
	log.Fatalf(err.Error())
}

defer conn.Close()

// Create RELP from net.Conn
r, err = relp.NewTcp(conn, opts)
if err != nil {
	log.Fatal(err.Error())
}

defer r.Close()

// Receive log message loop
for {

	// Receive and acknoledge log message
	msg, err = r.ReceiveLogAndAck()
	if err != nil {
		log.Fatal(err.Error())
	}

	// Process log message. Here, just display it
	println(msg)
}

Output:
Example (ReceiveMessage)

This example show full protocol handler. In most cases, this way is not interesting. 

var ln net.Listener
var conn net.Conn
var err error
var opts *relp.Options
var r *relp.Relp
var msg *relp.Message
var out map[string][]interface{}
var status bool
var offer []string
var message string

// Create and validate options
opts, err = relp.ValidateOptions(&relp.Options{
	Tls: relp.Opt_tls_disabled,
})
if err != nil {
	log.Fatalf(err.Error())
}

// Create listen connexion
ln, err = net.Listen("tcp", "0.0.0.0:4150")
if err != nil {
	log.Fatalf(err.Error())
}

// accept connection
conn, err = ln.Accept()
if err != nil {
	log.Fatalf(err.Error())
	return
}

defer conn.Close()

// Create RELP from net.Conn
r, err = relp.NewTcp(conn, opts)
if err != nil {
	log.Fatal(err.Error())
}

defer r.Close()

// receive first message, wich must it "open"
msg, err = r.ReceiveMessage()
if err != nil {
	log.Fatal(err.Error())
}
if msg == nil {
	log.Fatal("client close connection")
}
if msg.Command != "open" {
	log.Fatal("expect open message")
}

// decode offer
out, err = r.DecodeOffer(msg)
if err != nil {
	log.Fatal(err.Error())
}

// negociate offer
status, offer, message = r.NegociateOffer(msg, out)

// Answer offer
if !status {
	r.AnswerError(msg, message)
	log.Fatal("negociation fail")
}
r.AnswerOffer(msg, message, offer)

// Receive log message loop
for {

	// receive next message
	msg, err = r.ReceiveMessage()
	if msg == nil {
		log.Fatal("client close connection")
		return
	}

	// process message according with the received command
	switch msg.Command {

	case "syslog":

		// Process log message. Here, just display it
		println(string(msg.Data))

		// Acknoledge message
		err = r.AnswerOk(msg)
		if err != nil {
			log.Fatal(err.Error())
		}

	case "close":

		// process connection close
		r.AnswerOk(msg)
		log.Fatal("client close connection")

	default:
		// unknown message
		log.Fatalf("unknown command %q", msg.Command)
	}
}

Output:

Index

Examples

Constants

View Source 
const Acl_accept = 0

ACL configuration: accept connexion

View Source 
const Acl_reject = 1

ACL configuration: reject connexion

View Source 
const Opt_tls_allowed = 1

TLS configuraiton option: Allow starttls command. Note: starttls is not supported. This option will be implemented when RELP reference library will support starttls. Even "startls" is not supported, the negociation of "startls" command it is. Using this option allow the client to use "starttls" command. This doesn't work. Do not use.

View Source 
const Opt_tls_connection = 3

TLS configuration option: Enable RELP over TLS at client connection. This option disable "starttls". When this connecion mode is activated, the configuraiton require setr of certificates throught options "CACertificate", "Certificate" and "PrivateKey". At this time only private pki is supported. certification chain is not supported. A CRL file can be used with option "Crl" and a lot of ACL on certificate Common Name could be configured. Note the server require client certificate.

View Source 
const Opt_tls_disabled = 0

TLS configuraiton option: Disable TLS support trought starttls function. Note: starttls is not supported. This option will be implemented when RELP reference library will support starttls. Even "startls" is not supported, the negociation of "startls" command it is. Using this option, "starttls" is not announced.

View Source 
const Opt_tls_offloading = 4

TLS configuration option: This option assume the io.ReadWriter given ensure transport security over TLS and disbale "starttls" command.

View Source 
const Opt_tls_required = 2

TLS configuraiton option: Require starttls command. Note: starttls is not supported. This option will be implemented when RELP reference library will support starttls. Even "startls" is not supported, the negociation of "startls" command it is.

Variables

This section is empty.

Functions

This section is empty.

Types

type Acl 

type Acl struct {
	Value  string
	Action int
}

This struct define ACL. ACL is part of a list which define condition and action. Condition is a wildcard match between Value field and client certificate CN. If the wildcard match, Action reject or accept is applied and the Acl evalution stop.

type Message 

type Message struct {
	Txnr    int
	Command string
	Datalen int
	Data    []byte
}

This struct contains RELP message. Txnr is the unique number od the message in the stream. Command is a string containing RELP command. today commands could be "open", "syslog" and "close". Datalen is the length of Data buffer. the Data content is according with command. Data associated with command "open" shold be decoded with function DecodeOffer. Data associated with "syslog" is a syslog message and could be processed as string. Command "close" as no data associated.

type Options 

type Options struct {
	Tls            int    // Opt_tls_*
	CACertificate  string // Path to CA file at PEM format
	Certificate    string // Path to certificate file at PEM format
	PrivateKey     string // Path to private key at PEM format
	Crl            string // Path to CRL file at PEM format
	CnAcl          []Acl
	RecvSize       int
	StreamEscapeLf bool
	// contains filtered or unexported fields
}

This struct contains configuration option for the RELP server. CACertificate, Certificate and PrivateKey are mandatory if Tls option is Opt_tls_connection. The Certificate must be validated by the CACertificate. Each incoming TLS client connection must provides certificate signed by CA.

CnAcl is a list of Acl which validates cleint Common Name certificate.

RecvSize is the size of read buffer. Default size is 16384.

StreamEscapeLf is an option which replace all '\n' characters by '\', 'n'. this is usefull because in stream mode, the log separator is '\n', but some component could log multiline messages, for example backtraces. This escape is not strict, because we cannot differentiate modified sequence '\', 'n' with original sequence, so the original log cannot be recovered with high confidence.

func ValidateOptions 

func ValidateOptions(opts *Options) (*Options, error)

This function must be called to validate options. Options could be nil, in this case default value are used and OPtion struct is returned. If Option are provided as argument, the Option returned is the same struct.

Once the function ValidateOptions is called, the Options struct should not modified.

The function prepare TLS connection opening all cryptographic elements (CA, certificate, private key, CRL). Validate wildcard format of ACLs, and set defaults for some not specified values.

type Relp 

type Relp struct {
	// contains filtered or unexported fields
}

This struct is used to maintain state of established RELP connection. This struct implements interface io.ReadCloser

func NewReadWriter 

func NewReadWriter(fd io.ReadWriter, options *Options) (*Relp, error)

Handle new connection on the stream defined by "fd". The caller must ensure all the network processing as it want. (bind, listen, accept, goroutine, TLS, ...). "options" contains set of options to apply on this RELP session. The TLS options and common name ACL are not applicable in this mode. If the function retuns non nil "error" the connection could be close because. The function return RELP struct which is used to follow RELP stream.

func NewTcp 

func NewTcp(tcp_fd net.Conn, options *Options) (*Relp, error)

Handle new connection on the TCP stream defined by "fd". The caller just ensure networks server part (bind, listen, accept). The RELP library process TLS protocol if activated. If the function retuns non nil "error" the connection could be close because. The function return RELP struct which is used to follow RELP stream.

func (*Relp) AnswerError 

func (r *Relp) AnswerError(msg *Message, message string) error

This function just answer ko message. It is used to indicate an error while processing command. Commonly, sending an error is just below closing connexion, anyway, if the function return error, the connection should be closed.

func (*Relp) AnswerMulmtipleOk 

func (r *Relp) AnswerMulmtipleOk(msgs []*Message) error

This function answer ok message for a lot of incoming messages. It is used to acknolegde a batch of commands "syslog". This function is designed to deal with batch of log messages. The server program can receive a lot of log message, store this lot of message with one write on disk, ans then acknoledge all the message to the client. If the function return error, the connection should be closed.

func (*Relp) AnswerOffer 

func (r *Relp) AnswerOffer(msg *Message, message string, commands []string) error

This function send answer to the client in response to "offer" command. The function require a msg to exploit Txnr, a short human readable message acknoledging "offer" command. Often this message is "Ok". And the list of commands proposed by the server. this list of command could be obtained by the function NegociateOffer which produce this list if communication between client and server is possible. If the function return error, the connection should be closed.

func (*Relp) AnswerOk 

func (r *Relp) AnswerOk(msg *Message) error

This function just answer ok message. It is used to acknolegde commands "syslog" and "close". If the function return error, the connection should be closed.

func (*Relp) Close 

func (r *Relp) Close() error

This function close the connection where the RELP library is responsible. If the caller gives a tcp.Conn, the library just close RELP and TLS connection. If the caller give an io.ReadWriter the library just close RELP. The caller is responsible closing connections it establish or accept.

func (*Relp) DecodeOffer 

func (r *Relp) DecodeOffer(msg *Message) (map[string][]interface{}, error)

This function decode "open" command. The message associated with "open" command is called "offer" which announce client supported options. If the function returns error the connection should be closed. In succes case, the fucntion returns map with offer options as string index. In version 0 and 1 options are "relp_version", "commands" and "relp_software". "relp_version" contains client version. If the version sent is different from 0 or 1, the negociation should fail because the version is not supported by the library. "relp_software" is indicative and contains client version. "commands" contains the list of commands supported by client. Specification indicates "startls" and "syslog", in practice "startls" is never used because its not implemented in librelp. "syslog" is mandatory because its the only one format supported today.

func (*Relp) NegociateOffer 

func (r *Relp) NegociateOffer(msg *Message, offer map[string][]interface{}) (bool, []string, string)

This function check protocol version and supported options. It must take "open" command message as input. The function returns negociation status, true to accept connection, false to reject with 500. the function return an array of strings which contains supported commands on server side according with commands announced by the client. Last string returned is a human readable short message for the RELP reponse. These value could be modified before sending response, but in most cases its useless. The result is send to the client throught the function AnswerOffer. If the negociation result is reject, the connexion should be closed.

func (*Relp) Read 

func (r *Relp) Read(p []byte) (int, error)

Implements io.ReadCloser Read function. Negociation and acknoledgement are transparent for the caller. Log messages are separated by '\n', see the option StreamEscapeLf for more information. This function doesn't permit to control acknoledgements, so use it with caution, logs could be loss during program processing.

func (*Relp) ReceiveLog 

func (r *Relp) ReceiveLog() (*Message, error)

Just return log message, negociation is transparent for the caller. Note the message must be acknoledged using function AnswerOk or AnswerMulmtipleOk. This is the most valuable way to use RELP library, because the caller could send acknoledgements according with its processing. If the function return error, the connection should be closed.

func (*Relp) ReceiveLogAndAck 

func (r *Relp) ReceiveLogAndAck() (string, error)

Just return log message as string, negociation and acknoledgement are transparent for the caller. If the function returns error, the connection should be closed. This function doesn't permit to control acknoledgements, so / use it with caution, logs could be loss during program processing.

func (*Relp) ReceiveMessage 

func (r *Relp) ReceiveMessage() (*Message, error)

Low level receive function. It returns raw message which need parsing. Message "open" need to be parsed by DecodeOffer function. Message "syslog" just need string conversion. If the function returns error, the connection should be closed.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.