README
¶
tcp-info
The tcp-info tool executes a polling loop that tracks the measurement statistics of every open TCP socket on a system. Data is written, in JSONL format (refered to internally as ArchivedRecord), to files compressed using zstd. This tool forms the basis of a lot of measurements on the Kubernetes-based Measurement Lab platform.
We expect most people will run this tool using a docker container. To invoke, with data written to ~/data, and prometheus metrics published on port 7070:
docker run --network=host -v ~/data:/home/ -it measurementlab/tcp-info -prom=7070
Fast tcp-info collector in Go
This repository uses the netlink API to collect inet_diag messages, partially parses them, and caches the intermediate representation. It then detects differences from one scan to the next, and queues connections that have changed for logging. It logs the intermediate representation through external zstd processes to one file per connection.
The previous version uses protobufs, but we have discontinued that largely because of the increased maintenance overhead, and risk of losing unparsed data. Instead, we are now using ArchivedRecord which is partially parsed netlink messages, mostly in base64 encoded blobs, marshaled to JSONL format, with one JSON object per line.
To run the tests or the collection tool, you will also require zstd, which can be installed with:
bash <(curl -fsSL https://raw.githubusercontent.com/horta/zstd.install/master/install)
OR
sudo apt-get update && sudo apt-get install -y zstd
Example sidecar
The tcp-info eventsocket interface allows sidecar services to receive "open" and
"close" events on a unix domain socket connection. A simple reference
implementation cmd/example-eventsocket-client can be started using
docker-compose.
docker-compose up
New TCP events are processed by the example-eventsocket-client sidecar and
logged to stderr. You may trigger a TCP connection from within the TCPINFO
container using a command like:
docker exec -it tcp-info_tcpinfo_1 wget www.google.com
Parse library and command line tools
CSV tool
The cmd/csvtool directory contains a tool for parsing ArchivedRecord and producing CSV files. Currently reads netlink-jSONL from stdin and writes CSV to stdout.
Code Layout
- inetdiag - code related to include/uapi/linux/inet_diag.h. All structs will be in structs.go
- tcp - Should include ONLY the code related to include/uapi/linux/tcp.h
- parse - code related to parsing the messages in inetdiag and tcp.
- zstd - zstd reader and writer.
- saver - code related to writing ParsedMessages to files.
- cache - code to cache netlink messages and detect changes.
- collector - code related to collecting netlink messages from the kernel.
Dependencies (as of March 2019)
- saver: inetdiag, cache, parse, tcp, zstd
- collector: parse, saver, inetdiag, tcp
- main.go: collector, saver, parse (just for sanity check)
- cache: parse
- parse: inetdiag
And (almost) all package use metrics.
Layers for main.go (each layer depends only on items to right, or lower layers)
- main.go
- collector > saver > cache
- netlink > inetdiag
- tcp, zstd, metrics
Layers for parse package
- parse (used by command line tools, etl)
- netlink > inetdiag
- tcp, zstd, metrics
Documentation
¶
There is no documentation for this package.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cache keeps a cache of connection info records.
|
Package cache keeps a cache of connection info records. |
|
cmd
|
|
|
csvtool
Main package in csvtool implements a command line tool for converting ArchiveRecord files to CSV files.
|
Main package in csvtool implements a command line tool for converting ArchiveRecord files to CSV files. |
|
example-eventsocket-client
example-eventsocket-client is a minimal reference implementation of a tcpinfo eventsocket client.
|
example-eventsocket-client is a minimal reference implementation of a tcpinfo eventsocket client. |
|
Package collector repeatedly queries the netlink socket to discover measurement data about open TCP connections and sends that data down a channel.
|
Package collector repeatedly queries the netlink socket to discover measurement data about open TCP connections and sends that data down a channel. |
|
Package inetdiag provides basic structs and utilities for INET_DIAG messaages.
|
Package inetdiag provides basic structs and utilities for INET_DIAG messaages. |
|
Package metrics defines prometheus metric types and provides convenience methods to add accounting to various parts of the pipeline.
|
Package metrics defines prometheus metric types and provides convenience methods to add accounting to various parts of the pipeline. |
|
Package netlink contains the bare minimum needed to partially parse netlink messages.
|
Package netlink contains the bare minimum needed to partially parse netlink messages. |
|
Package saver contains all logic for writing records to files.
|
Package saver contains all logic for writing records to files. |
|
Package snapshot contains code to generate Snapshots from ArchiveRecords, and utilities to load them from files.
|
Package snapshot contains code to generate Snapshots from ArchiveRecords, and utilities to load them from files. |
|
Package tcp provides TCP state constants and string coversions for those constants.
|
Package tcp provides TCP state constants and string coversions for those constants. |
|
Package zstd provides utilities for connecting to external zStandard compression tasks.
|
Package zstd provides utilities for connecting to external zStandard compression tasks. |