grafaman

📈 grafaman

Metrics coverage reporter for Graphite and Grafana.

💡 Idea

$ grafaman coverage \
    --grafana https://grafana.api/ -d DTknF4rik \
    --graphite https://graphite.api/ \
    --metrics apps.services.awesome-service
# +--------------------------------------------------------------------+--------+
# | Metric                                                             | Hits   |
# +--------------------------------------------------------------------+--------+
# | apps.services.awesome-service.jaeger.finished_spans_sampled_n      |      0 |
# | apps.services.awesome-service.rpc.client.success.ok.percentile.75  |      1 |
# | apps.services.awesome-service.rpc.client.success.ok.percentile.95  |      1 |
# | apps.services.awesome-service.rpc.client.success.ok.percentile.99  |      2 |
# | apps.services.awesome-service.rpc.client.success.ok.percentile.999 |      1 |
# | ...                                                                |    ... |
# | apps.services.awesome-service.go.pod-5dbdcd5dbb-6z58f.threads      |      0 |
# +--------------------------------------------------------------------+--------+
# |                                                              Total | 65.77% |
# +--------------------------------------------------------------------+--------+

A full description of the idea is available here.

🏆 Motivation

At Avito, we develop many services built on top of our excellent PaaS and internal modules. These services send a lot of metrics about their internal state that are then output to Grafana dashboards.

I need a tool that helps me to understand what metrics are published by services and how many of them are presented at Grafana dashboards.

🤼‍♂️ How to

Metrics coverage report

$ grafaman coverage \
    --grafana https://grafana.api/ -d DTknF4rik \
    --graphite https://graphite.api/ \
    -m apps.services.awesome-service \
    --last 24h \
    --trim='complex.$env.' --trim='env.$env.' \
    --exclude='*.count' --exclude='*.max' --exclude='*.min' --exclude='*.sum'

Supported environment variables:

• GRAFANA_URL
• GRAFANA_DASHBOARD
• GRAPHITE_URL
• GRAPHITE_METRICS

Supported config files by default:

• .env.paas
• app.toml

located at current working directory.

Supported output formats:

• table view
  • default
  • compact
  • compact-lite
  • markdown
  • rounded
  • unicode
• json

$ grafaman coverage ... -f json | jq
# [
#   {
#     "name": "apps.services.awesome-service.jaeger.finished_spans_sampled_n",
#     "hits": 0
#   },
#   ...
#   {
#     "name": "apps.services.awesome-service.go.pod-5dbdcd5dbb-6z58f.threads",
#     "hits": 0
#   }
# ]

• tsv

$ grafaman coverage ... -f tsv | column -t
# apps.services.awesome-service.jaeger.finished_spans_sampled_n         0
# apps.services.awesome-service.rpc.client.success.ok.percentile.75     1
# apps.services.awesome-service.rpc.client.success.ok.percentile.95     1
# apps.services.awesome-service.rpc.client.success.ok.percentile.99     2
# apps.services.awesome-service.rpc.client.success.ok.percentile.999    1
# ...                                                                 ...
# apps.services.awesome-service.go.pod-5dbdcd5dbb-6z58f.threads         0

Fetch metrics from Graphite

$ grafaman metrics -e https://graphite.api/ -m apps.services.awesome-service --last 24h

Fetch queries from Grafana

$ grafaman queries -e https://grafana.api/ -d DTknF4rik \
    -m apps.services.awesome-service \
    --trim='complex.$env.' --trim='env.$env.' \
    --sort

🧩 Installation

Homebrew

$ brew install kamilsk/tap/grafaman

Binary

$ curl -sSfL https://raw.githubusercontent.com/kamilsk/grafaman/master/bin/install | sh
# or
$ wget -qO-  https://raw.githubusercontent.com/kamilsk/grafaman/master/bin/install | sh

Don't forget about security.

Source

# use standard go tools
$ go get github.com/kamilsk/grafaman@latest
# or use egg tool
$ egg tools add github.com/kamilsk/grafaman@latest

egg is an extended go get.

Bash and Zsh completions

$ grafaman completion bash > /path/to/bash_completion.d/grafaman.sh
$ grafaman completion zsh  > /path/to/zsh-completions/_grafaman.zsh
# or autodetect
$ source <(grafaman completion)

See kubectl documentation.

made with ❤️ for everyone