promalert

What is PromAlert?

Yet another Prometheus Alertmanager webhook processor inspired by qvl/promplot

How it works

Receive webhook from Alertmanager, draw images from alert expression, upload pictures to S3 bucket, generate public links, send a notification to Slack.

Why pormalert?

Standard prometheus slack receiver cant use threads and draw graphs from alert.

Main features:

• Threads to keep alert channel informative
• Updating init firing message, mark it as resolved
• Graph rendering with metric and alert level

Message logic

Post the new message in case when we can find a reference in the history example.

Otherwise, based on the status of the message, we apply the following logic:

Status = Firing

Post full message to thread with broadcasting. Set refiring footer to last known firing message. Store link to this message in memory (next resolving updates will edit this message) example.

Status = Resolved

Post short message (header + images) to thread example. Update footer of last fired message example.

Installation

Helm chart

Coming soon

Docker

Use env file to store required params

docker run -p 8080:8080 --env-file env.list kuzaxak/promalert

Configuration

The following tables list the configurable parameters of the PromAlert and their default values.

You can use a YAML configuration file or env variable. Package viper used to parse them. Environment variables prefix: PROMALERT_

Required params:

Additional params:

AWS

AWS credentials parsed by aws-go-client in the following order:

1. Environment variables.
2. Shared credentials file.
3. If your application is running on an Amazon EC2 instance, IAM role for Amazon EC2.

Message templating

Template applies per alert in group. Data in the template . = Alert

Available functions:

Default message template:

  :chart_with_upwards_trend: *<{{ .GeneratorURL }}|Graph>*
  {{- if .Labels.runbook }} :notebook: *<{{ .Labels.runbook }}|Runbook>*{{ end }}
  {{- if .Annotations.runbook_url }} :notebook: *<{{ .Annotations.runbook_url }}|Runbook>*{{ end }}

  *Alert:* {{ if .Annotations.title }}{{ .Annotations.title }}{{ end }}{{ if .Annotations.summary }}{{ .Annotations.summary }}{{ end }}
  {{ if .Labels.severity }}*Severity:*  `{{ .Labels.severity }}`{{ end }}
  {{ if .Annotations.message }}*Message:*  {{ .Annotations.message }}{{ end }}
  {{ if .Annotations.description }}*Description:* {{ .Annotations.description }}{{ end }}
  *Details:*
    {{ range $key, $value := .Labels }} • {{ $key }}: {{ $value }}
    {{ end }}

Header template:

  *{{ .Labels.alertname }}*
  [Status]: {{if eq .Status "firing" }}:fire::fire::fire:{{else}}:white_check_mark::white_check_mark::white_check_mark:{{ end }}

Footer template:

  {{if eq .Status "firing" }}:fire: Refired{{else}}:white_check_mark: Resolved{{ end }} {{ dateFormat "15:04:05" now }}

Rendered message:

Build

docker build --rm -t promalert .

Bug Reporting

To submit a bug report use the GitHub bug tracker for the project:

Github Issues

License

GNU Lesser General Public License v3.0

See LICENSE to see the full text.