webhookd

package module
v0.1.4-0...-f970457 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 17, 2019 License: BSD-3-Clause Imports: 2 Imported by: 0

README

go-webhookd

What is the simplest webhook-wrangling server-daemon-thing.

In many ways this is nothing more than a fancy bucket-brigade. By design.

Receivers handle the actual webhook side of things, doing auth and basic sanity checking and validation. Assuming everything is as it should be receivers return a bag of bytes (the actual webhook message that may or may not be massaged depending the receiver). That bag is then handed to one or more dispatchers which do something with those bytes. Those details, including security considerations are left as an exercise to the reader.

In between (receivers and dispatchers) are an optional chain of transformations which accept bytes as their input, do something with those bytes, and then return bytes.

Install

You will need to have both Go (specifically version 1.12 or higher) and the make programs installed on your computer. Assuming you do just type:

make tools

All of this package's dependencies are bundled with the code in the vendor directory.

Usage

webhookd
./bin/webhookd -h
Usage of ./bin/webhookd:
  -config string
    	Path to a valid webhookd config file

webhookd is an HTTP daemon for handling webhook requests. Individual webhook endpoints (and how they are processed) are defined in a config file that is read at start-up time.

Example

This is a deliberately juvenile example, just to keep things simple.

Let's assume an insecure receiver with debugging enabled that reads input, transforms it using the go-chicken clucking method and drops the results on the floor.

Here are the relevant settings in the config file:

{
	"daemon": {
		"protocol": "http",
		"host": "localhost",
		"port": 8080,
		"allow_debug": false
	},
	...
	"webhooks": [
		{
			"endpoint": "/insecure-test",
			"receiver": "insecure",
			"transformations": [ "clucking" ],
			"dispatchers": [ "null" ]
		}
	]
}

First we start webhookd:

./bin/webhookd -config ./config.json
2018/07/21 08:43:37 webhookd listening for requests on http://localhost:8080

Then we pass webhookd a file along with a debug=1 query parameter so that we can see the output:

curl -v 'http://localhost:8080/insecure-test?debug=1' -d @README.md
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /insecure-test?debug=1 HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Length: 12790
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Access-Control-Allow-Origin: *
< Content-Type: text/plain
< X-Webhookd-Time-To-Dispatch: 16.907µs
< X-Webhookd-Time-To-Process: 13.033089ms
< X-Webhookd-Time-To-Receive: 209.332µs
< X-Webhookd-Time-To-Transform: 12.802186ms
< Date: Sat, 21 Jul 2018 15:43:40 GMT
< Transfer-Encoding: chunked
<
# bok bok b'gawk-cluck cluck![](bok bok b'gawk/bok bok b'gawk-bok bok bok.cluck
cluck)bok bok bok bok bok b'gawk bok bok bok cluck cluck bok bok b'gawk-bok bok
bok cluck cluck-bok bok b'gawk-bok bok bok.bok bok b'gawk cluck cluck bok bok
b'gawk bok bok b'gawk bok bok b'gawk bok bok b'gawk cluck cluck bok bok b'gawk
bok bok bok cluck cluck bok bok bok-bok bok bok. bok bok
... and so on
Caveats
TLS

TLS is not supported yet so you should not run webhookd on the public internets without first putting a TLS-enabled proxy in front of it.

Dynamic endpoints

At some point there might be dynamic (or runtime) webhook endpoints but today there are not.

In the meantime you can gracefully restart webhookd by sending its PID a USR2 signal which will cause the config file (and all the endpoints it defines) to be re-read. It's not elegant but it works. For example:

$> ./bin/webhookd -config config.json
2016/10/16 00:19:47 Serving 127.0.0.1:8080 with pid 2723

$> kill -USR2 2723
2016/10/16 00:19:59 Graceful handoff of 127.0.0.1:8080 with new pid 2724 and old pid 2723
2016/10/16 00:19:59 Exiting pid 2723.
Setting up a webhookd server

While you can set up a webhookd server by hand it's probably easier to all that work with a config file and let code take care of all the details, including registering all the webhooks. Config files are discussed in detail below.

All error handling in the examples below have been removed for the sake of brevity.

Setting up a webhookd server with a handy config file
import (
	"github.com/whosonfirst/go-webhookd/config"
	"github.com/whosonfirst/go-webhookd/daemon"
)

wh_config, _ := config.NewConfigFromFile("config.json")

wh_daemon, _ := daemon.NewWebhookDaemonFromConfig(wh_config)
wh_daemon.Start()

You can also just grab the HTTP handler func with wh_daemon.HandlerFunc() if you need or want to start a webhookd daemon in your own way.

Setting up a webhookd server "by hand"
import (
	"github.com/whosonfirst/go-webhookd"		
	"github.com/whosonfirst/go-webhookd/daemon"	
	"github.com/whosonfirst/go-webhookd/dispatchers"
	"github.com/whosonfirst/go-webhookd/receivers"
	"github.com/whosonfirst/go-webhookd/transformations"
	"github.com/whosonfirst/go-webhookd/webhook"	
)

wh_receiver, _ := receivers.NewInsecureReceiver()

null, _ := transfromations.NewNullTransformation()
wh_transformations := []webhookd.WebhookTransformation{ null }

pubsub, _ := dispatchers.NewPubSubDispatcher("localhost", 6379, "websocketd")
wh_dispatchers, _ := []webhookd.WebhookDispatcher{ pubsub }

wh, _ := webhook.NewWebhook("/foo", wh_receiver, wh_transformations, wh_dispatchers)

wh_daemon, _ := daemon.NewWebhookDaemon("http", "localhost", 8080)
wh_daemon.AddWebhook(wh)
wh_daemon.Start()

See the way we're using an Insecure receiver and a PubSub dispatcher with a Null transformation? All are these are discussed in detail below.

Sending stuff to webhookd

curl -v http://localhost:8080/foo -d @README.md

* upload completely sent off: 703 out of 703 bytes
< HTTP/1.1 200 OK
< Date: Fri, 20 May 2016 01:31:05 GMT
< Content-Length: 0
< Content-Type: text/plain; charset=utf-8
< 
* Connection #0 to host localhost left intact

Where did it go...

./bin/subscribe webhookd
{'pattern': None, 'type': 'subscribe', 'channel': 'webhookd', 'data': 1L}
{'pattern': None, 'type': 'message', 'channel': 'webhookd', 'data': '# go-webhookd## ImportantYou should not try to use this, yet. No. No, really.## UsageIt _should_ work something like this. If you\'re reading this sentence that means it _doesn\'t_.```import (\t"github.com/whosonfirst/go-webhookd"\t"github.com/whosonfirst/go-webhookd/dispatchers"\t"github.com/whosonfirst/go-webhookd/receivers")dispatcher := dispatchers.NewPubSubDispatcher("localhost", 6379, "pubsub-channel")receiver := receivers.NewGitHubReceiver("github-webhook-s33kret")endpoint := "/wubwubwub"webhook := webhookd.NewWebhook(endpoint, receiver, dispatcher)daemon := webhookd.NewWebHookDaemon(webhook)daemon.AddWebhook(webhook)daemon.Start()```## See also'}

In this case, it went to Redis PubSub land! Where things go depend on how you've configured your dispatchers.

Config files

Config files for webhookd are JSON files consisting of five top-level sections. An example config file is included with this repository. The five top-level sections are:

daemon
	"daemon": {
		"host": "localhost",
		"port": 8080
	}

The daemon section is a dictionary defining configuration details for the webhookd daemon itself.

receivers
	"receivers": {
		"insecure": {
			"name": "Insecure"
		},
		"github": {
			"name": "GitHub",
			"secret": "s33kret"
		}
	}

The receivers section is a dictionary of "named" receiver configuations. This allows the actual webhook configurations (described below) to signal their respective receivers using the dictionary "name" as a simple short-hand.

transformations
	"transformations": {
		"chicken": {
			"name": "Chicken",
			"language": "zxx",
			"clucking": false
		}
	}

The transformations section is a dictionary of "named" tranformation configuations. This allows the actual webhook configurations (described below) to signal their respective transformations using the dictionary "name" as a simple short-hand.

dispatchers
	"dispatchers": {
		"pubsub": {
			"name": "PubSub",
			"host": "localhost",
			"port": 6379,
			"channel": "webhookd"
		}
	}

The dispatchers section is a dictionary of "named" dispatcher configuations. This allows the actual webhook configurations (described below) to signal their respective dispatchers using the dictionary "name" as a simple short-hand.

webhooks
	"webhooks": [
		{
			"endpoint": "/github-test",
			"receiver": "github",
			"dispatchers": [ "pubsub" ]
		},
		{
			"endpoint": "/insecure-test",
		 	"receiver": "insecure",
			"dispatchers": [ "pubsub" ]
		},
		{
			"endpoint": "/slack-test",
			"receiver": "slack",
			"transformations": [ "slack", "chicken" ],
			"dispatchers": [ "slack", "log"]
		}
	]

The webhooks section is a list of dictionaries. These are the actual webhook endpoints that clients (out there on the internet) will access.

  • endpoint This is the path that a client will access. It is the webhook URI that clients will send requests to.
  • receiver The named receiver (defined in the receivers section) that the webhook will use to process requests.
  • transformations An optional list of named transformations (defined in the transformations section) that the webhook process the message body with.
  • dispatchers The list of named dispatchers (defined in the dispatchers section) that the webhook will relay a successful request to.

Receivers

GitHub
	{
		"name": "GitHub",
		"secret": "s33kret"
	}

This receiver handles Webhooks sent from GitHub. It validates that the message sent is actually from GitHub (by way of the X-Hub-Signature header) but performs no other processing.

Properties
  • name string This is always GitHub.
  • secret string The secret used to generate the HMAC hex digest of the message payload.
  • ref string An optional Git ref to filter by. If present and a WebHook is sent with a different ref then the daemon will return a 666 error response.
Insecure
	{
		"name": "Insecure"
	}

As the name suggests this receiver is completely insecure. It will happily accept anything you send to it and relay it on to the dispatcher defined for that webhook.

This receiver exists primarily for debugging purposes and you should not deploy it in production.

Properties
  • name string This is always Insecure.
Slack
	{
		"name": "Slack"
	}

This receiver handles Webhooks sent from Slack. It does not process the message at all. This receiver has not been fully tested yet so proceed with caution.

Properties
  • name string This is always Slack.

Transformations

Chicken
	{
		"name": "Chicken",
		"language": "zxx",
		"clucking": false
	}

The Chicken transformation will convert every word in your message to 🐔 using the go-chicken package. If this seems silly that's because it is. It's also more fun that yet-another boring "make all the words upper-cased" example.

Properties
  • name string This is always Chicken.
  • language string A three-letter language code specifying which language go-chicken should use.
  • clucking bool A boolean flag indicating whether or not to cluck when generating results.
GitHubCommits
	{
		"name": "GitHubCommits",
		"exclude_additions":false,
		"exclude_modification":false,
		"exclude_deletions":true,
	}

The GitHubCommits transformation will extract all the commits (added, modified, removed) from a push event and return a JSON encoded list of paths.

Properties
  • name string This is always GitHubCommits.
  • exclude_additions bool A flag to indicate that new additions in a commit should be ignored. Optional; default false.
  • exclude_modification bool A flag to indicate that modifications in a commit should be ignored. Optional; default false.
  • exclude_deletions bool A flag to indicate that deletions in a commit should be ignored. Optional; default false.
GitHubRepo
	{
		"name": "GitHubRepo",
		"exclude_additions":false,
		"exclude_modification":false,
		"exclude_deletions":true,
	}

The GitHubRepo transformation will extract the reporsitory name for all the commits matching (added, modified, removed) criteria.

Properties
  • name string This is always GitHubRepo.
  • exclude_additions bool A flag to indicate that new additions in a commit should be ignored. Optional; default false.
  • exclude_modification bool A flag to indicate that modifications in a commit should be ignored. Optional; default false.
  • exclude_deletions bool A flag to indicate that deletions in a commit should be ignored. Optional; default false.
Null
	{
		"name": "Null"
	}

The Null transformation will not do anything. It's not clear why you would ever use this outside of debugging but that's your business.

Properties
  • name string This is always Null.
SlackText
	{
		"name": "SlackText"
	}

The SlackText transformation will extract and return the text property from a Webhook sent by Slack.

Properties
  • name string This is always SlackText.

Dispatchers

Lambda
	{
		"name": "Lambda",
		"dsn": "region=us-east-1",
		"function": "WebhookdLambdaFunction"
	}

The Lambda dispatcher will send messages to an AWS Lambda function.

Properties
  • name string This is always Lambda.
  • dsn string A valid go-whosonfirst-aws Lambda DSN string.
  • name string The name of your Lambda function.
Log
	{
		"name": "Log"
	}

The Log dispatcher will send messages to Go's logging facility. As of this writing that means everything is logged to STDOUT but eventually it will be more sophisticated.

Properties
  • name string This is always Null.
Null
	{
		"name": "Null"
	}

The Null dispatcher will send messages in to the vortex, never to be seen again. This can be useful for debugging.

Properties
  • name string This is always Null.
PubSub
	{
		"name": "PubSub",
		"host": "localhost",
		"port": 6379,
		"channel": "webhookd"
	}

The PubSub dispatcher will send messages to a Redis PubSub channel.

Properties
  • name string This is always PubSub.
  • host string The address of the Redis host you want to connect to.
  • port int The port number of the Redis host you want to connect to.
  • channel string The name of the Redis PubSub channel you want to send messages to.
Slack
	{
		"name": "Slack",
		"config": "/path/to/.slackcat.conf"			
	}

The Slack dispatcher will send messages to a Slack channel using the slackcat package.

Properties
  • name string This is always Slack.
  • config string The path to a valid slackcat config file. Eventually you will be able to specify a plain-vanilla Slack Webhook URL but not today.

To do

See also

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type WebhookDispatcher 

type WebhookDispatcher interface {
	Dispatch([]byte) *WebhookError
}

type WebhookError 

type WebhookError struct {
	Code    int
	Message string
}

func (WebhookError) Error 

func (e WebhookError) Error() string

type WebhookHandler 

type WebhookHandler interface {
	Endpoint() string // sudo make me a net.URI or something
	Receiver() WebhookReceiver
	Transformations() []WebhookTransformation
	Dispatchers() []WebhookDispatcher
}

type WebhookReceiver 

type WebhookReceiver interface {
	Receive(*http.Request) ([]byte, *WebhookError)
}

type WebhookTransformation 

type WebhookTransformation interface {
	Transform([]byte) ([]byte, *WebhookError)
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
webhookd
webhookd-generate-hook
webhookd-test-github

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.