Foo Protocol Proxy
_____ ____ _ _ ____
| ___|__ ___ | _ \ _ __ ___ | |_ ___ ___ ___ | | | _ \ _ __ _____ ___ _
| |_ / _ \ / _ \ | |_) | '__/ _ \| __/ _ \ / __/ _ \| | | |_) | '__/ _ \ \/ / | | |
| _| (_) | (_) | | __/| | | (_) | || (_) | (_| (_) | | | __/| | | (_) > <| |_| |
|_| \___/ \___/ |_| |_| \___/ \__\___/ \___\___/|_| |_| |_| \___/_/\_\\__, |
|___/
is a Golang implementation for proxy receiver over Foo protocol.
Table of Contents
Overview
Foo is a simple, powerful, extensible, cloud-native request/response protocol.
Foo protocol messages are UTF-8 encoded strings with the following BNF-ish definition:
Msg := Type <whitespace> Seq [<whitespace> Data] '\n'
Type := "REQ" | "ACK" | "NAK"
Seq := <integer>
Data := <string without newline>
When the Foo server receives a TCP connection initiated by a Foo client,
A session is started and a set of messages are exchanged between the two parties,
till the client terminates the connection eventually.
The clients sign their messages to the server with "REQ" along with the required action,
and the server might respond with acknowledgement "ACK" or denial "NAK" signature.
Example
A->B: <connect>
A<-B: REQ 1 Hey\n
A<-B: ACK 1 Hello\n
A->B: REQ 2 Hey there\n
A<-B: ACK 2\n
A->B: REQ 3 Hey\n
A->B: REQ 4 Hey\n
A->B: REQ 5 Hey\n
A<-B: ACK 3 What\n
A<-B: ACK 4 What do you want\n
A->B: REQ 6 Hey\n
A<-B: NAK 5 Stop it\n
A<-B: NAK 6 Stop doing that\n
A->B: <disconnect>
The proxy has a reporting features like:
- Reporting stats in JSON format to stdout when sending
SIGUSR2
signal to the process.
- Reporting stats in JSON format over HTTP "/stats" or "/metrics.
- Health check over HTTP "/health" or "/status".
- Data recovery after failure using a sliding window of
10s
time frame.
JSON Sample Response
{"msg_total":10,"msg_req":10,"msg_ack":8,"msg_nak":2,"request_rate_1s":0.005,"request_rate_10s":0.004,"response_rate_1s":0.004,"response_rate_10s":0.003}
Getting Started
Prerequisites
- Golang installation, having $GOPATH properly set.
- Optional, required only if using Docker approach.
Installation
To install Foo Protocol Proxy
$ go get github.com/ahmedkamals/foo-protocol-proxy
Test Driver
You can use the following steps as a testing procedure
-
Server
$ bin/server-linux -listen ":8001"
-
Proxy
-
Normal Approach
Running proxy on the host directly.
$ make build
$ bin/foo-protocol-proxy-{OS}-{ARCH} -forward "{FORWARDING_PORT}" -listen "{LISTENING_PORT}" -http "{HTTP_ADDRESS}" -recovery-path "{RECOVERY_PATH}"
Environment
:
OS
- the current operating system, e.g. (linux, darwin, ...etc.)
ARCH
- the current system architecture, e.g. (amd64, 386)
Params
:
FORWARDING_PORT
- e.g. ":8081"
LISTENING_PORT
- e.g. ":8082"
HTTP_ADDRESS
- e.g. "0.0.0.0:8088"
RECOVERY_PATH
- e.g. "data/recovery.json"
Sending SIGUSR2
Signal
# Process name: foo-protocol-proxy-{OS}-{ARCH}
# For linux, and amd64, it would be as following:
$ kill -SIGUSR2 $(pidof foo-protocol-proxy-linux-amd64) > /dev/null 2>&1
-
Docker Approach
Running proxy through docker container.
$ bash deploy.sh -f "{FORWARDING_PORT}" -l "{LISTENING_PORT}" -h "{HTTP_ADDRESS}" -r "{RECOVERY_PATH}"
Params
:
f
- e.g. 8081
l
- e.g. 8082
h
- e.g. 8088
r
- e.g. "data/recovery.json"
Sending SIGUSR2
Signal
$ docker exec -it foo-proxy-0.0.1 pkill -SIGUSR2 foo-protocol-proxy > /dev/null 2>&1
$ docker logs -f foo-proxy-0.0.1
-
Multiple Client Connections
$ for i in {0..1000..1}
do
bin/client-linux -connect "localhost:8002";
done
Tests
Not all items covered, just made one example.
$ make unit
Coding - Structure & Design
Item |
Description |
Dispatcher |
parses configuration, builds configuration object, and passes it to the proxy. |
Proxy |
orchestrates the interactions between the components. |
Listner |
awaits for client connections, and on every new connection, a BridgeConnection instance is created. |
BridgeConnection |
acts as Bi-directional communication object, that passes data forward and backward to the server. |
Analyzer |
perform analysis by sniffing all the data read and written from the server. |
Stats |
wraps stats data the would be flushed to stdout upon request. |
TimeTable |
contains snapshot of aggregated number of requests/responses at specific timestamp. |
Saver |
handles reading/writing data. |
Recovery |
handles storing/retrieval of backup data. |
HTTPServer |
reports metrics/stats over HTTP using the path /metrics or /stats , also used for health check using /health or /status . |
Todo
- Resource pooling for connections, to enable reuse of a limited number of open connections with the server,
and to requeue unused ones.
- Performance and memory optimization.
- More unit tests coverage.
- Refactoring.
Enjoy!