vesper

Vesper - Middleware for AWS Lambda

The Golang Middleware engine for AWS Lambda.

• Vesper - Middleware for AWS Lambda
  • Introduction
  • Get started
  • API
    • Usage
    • Logging
  • Auto unmarshalling
  • Writing your own Middleware
  • Available Middleware
    • Warmup
    • Parser
    • JSONParser
    • SQSParser
    • JSONSQSParser
  • How it works
    • Execution order
    • Interrupt middleware execution early
  • TODO
  • Developer Documentation
    • Goals
    • Contributing
  • What's in a name?

Introduction

Vesper is a very simple middleware engine for Lamda functions. If you are used to HTTP Web frameworks like Gorilla Mux and Go Kit, then you will be familiar with the concepts adopted in Vesper.

Middleware allows developers to isolate common technical concerns - such as input/output validation, logging and error handling - into functions that decorate the main business logic. This enables you to reuse these focus on writing code that remains clean, readable and easy to test and maintain.

Get started

Create a new serverless project from the Vesper template:

serverless create -u https://github.com/mefellows/vesper/tree/master/template

API

Usage

// MyHandler implements the Lambda Handler interface
func MyHandler(ctx context.Context, u User) (interface{}, error) {
	log.Println("[MyHandler]: handler invoked with user: ", u.Username)

	return u.Username, nil
}

func main() {
  // Create a new vesper instance, passing in all the Middlewares
  v := vesper.New(MyHandler, vesper.WarmupMiddleware)

  // Replace the standard lambda.Start() with Vesper's wrapper
  v.Start()
}

Logging

You can set your own custom logger with vesper.Logger(l LogPrinter).

Auto unmarshalling

The default behavior for Vesper is to automatically JSON unmarshal the payload into the type specificed in the handler parameter. This is consistent with the behaviour of the AWS Go Lambda library. This is useful if your handler accepts an input parameter which can be directly JSON unmarshalled into the parameter type. An example of this is the event types found in github.com/aws/aws-lambda-go/events.

This behaviour can be turned off in which case Vesper will start the middleware chain with the payload in []byte form. If you are using a Parser middleware, you will likely want to turn this off to allow the parser to do the converting instead of Vesper.

Here is an example of how to turn off the auto unmarshalling:

// MyHandler implements the Lambda Handler interface
func MyHandler(ctx context.Context, input []byte) error {
  log.Println("[MyHandler]: handler invoked with payload: ", input)

	return nil
}

func main() {
  // Create a new vesper instance and disable auto unmarshalling
  v := vesper.New(MyHandler).
  			DisableAutoUnmarshal()

  // Replace the standard lambda.Start() with Vespers wrapper
	v.Start()
}

Writing your own Middleware

A middleware is a function that takes a LambdaFunc and returns another LambdaFunc. A LambdaFunc is simple a named type for the AWS Handler signature.

Most middleware's do three things:

1. Modify or perform some action on the incoming request (such as validating the request)
2. Call the next middleware in the chain
3. Modify or perform some action on the outgoing response (such as validating the response)

Example:

var dummyMiddleware = func(next vesper.LambdaFunc) vesper.LambdaFunc {
	// one time scope setup area for middleware - e.g. in-memory FIFO cache

	return func(ctx context.Context, in interface{}) (interface{}, error) {
		log.Println("[dummyMiddleware] START:")
		// (1) Modify the incoming request, or update the context before passing to the
		// next middleware in the chain
		// (2) You must call the next middleware in the chain if the request should proceed
		// and you want other middleware to execute
		res, err := next(ctx, in)
		// (3) Your last chance to modify the response before it is passed to any remaining
		// middleware in the chain
		log.Println("[dummyMiddleware] END:", in)
		return res, err
	}
}

Available Middleware

Warmup

Short circuits a request if the serverless warmup event is detected.

TIP: This middleware should be included early in the chain, before any validation or processing happens

Implements a warmup handler for https://www.npmjs.com/package/serverless-plugin-warmup

Example:

func main() {
	m := vesper.New(MyHandler, vesper.WarmupMiddleware, /* any other middlewares here */)
	m.Start()
}

Parser

Parses the input payload to the type specificed in the handler parameter. It accepts a decoder function so you can decide how it parses the payload.

NOTE: the auto unmarshaling needs to be turned off for this middleware to work correctly. See Auto unmarshalling.

Example of usage:

import (
	"encoding/json"

	"github.com/mefellows/vesper"
)

func main() {
	m := vesper.New(MyHandler).
		DisableAutoUnmarshal().
		Use(vesper.ParserMiddleware(json.Unmarshal))
	m.Start()
}

JSONParser

This is a shorthand for using the Parser middleware - vesper.ParserMiddleware(json.Unmarshal).

Example of usage:

import "github.com/mefellows/vesper"

func main() {
	m := vesper.New(MyHandler).
		DisableAutoUnmarshal().
		Use(vesper.JSONParserMiddleware())
	m.Start()
}

SQSParser

Parses the input payload as an SQS event and then extracts and unmarshals the body into the type specificed in the handler parameter. The handler parameter must be a slice as SQS events are always batched. It accepts a decoder function so you can decide how it parses the SQS record body.

NOTE: the auto unmarshaling needs to be turned off for this middleware to work correctly. See Auto unmarshalling.

Example of usage:

import (
	"encoding/json"

	"github.com/mefellows/vesper"
)

func MyHandler(ctx context.Context, users []users) error {
  log.Println("[MyHandler]: handler invoked with users: ", input)

	return nil
}

func main() {
	m := vesper.New(MyHandler).
		DisableAutoUnmarshal().
		Use(vesper.SQSParserMiddleware(json.Unmarshal))
	m.Start()
}

JSONSQSParser

This is a shorthand for using the SQSParser middleware - vesper.SQSParserMiddleware(json.Unmarshal).

Example of usage:

import "github.com/mefellows/vesper"

func main() {
	m := vesper.New(MyHandler).
				DisableAutoUnmarshal().
  			Use(vesper.JSONParserMiddleware())
	m.Start()
}

How it works

Vesper implements the classic onion-like middleware pattern, with some peculiar details.

When you attach a new middleware this will wrap the business logic contained in the handler in two separate steps.

When another middleware is attached this will wrap the handler again and it will be wrapped by all the previously added middlewares in order, creating multiple layers for interacting with the request (event) and the response.

This way the request-response cycle flows through all the middlewares, the handler and all the middlewares again, giving the opportunity within every step to modify or enrich the current request, context or the response.

Execution order

Middlewares have two phases: before and after.

The before phase, happens before the handler is executed. In this code the response is not created yet, so you will have access only to the request.

The after phase, happens after the handler is executed. In this code you will have access to both the request and the response.

If you have three middlewares attached as in the image above this is the expected order of execution:

• middleware1 (before)
• middleware2 (before)
• middleware3 (before)
• handler
• middleware3 (after)
• middleware2 (after)
• middleware1 (after)

Notice that in the after phase, middlewares are executed in reverse order, this way the first handler attached is the one with the highest priority as it will be the first able to change the request and last able to modify the response before it gets sent to the user.

Interrupt middleware execution early

Some middlewares might need to stop the whole execution flow and return a response immediately.

If you want to do this you cansimple omit invoking next middleware and return early.

Note: this will stop the execution of successive middlewares in any phase (before and after) and returns an early response (or an error) directly at the Lambda level. If your middlewares does a specific task on every request like output serialization or error handling, these won't be invoked in this case.

In this example we can use this capability for rejecting an unauthorised request:

var authMiddleware = func(next vesper.LambdaFunc) vesper.LambdaFunc {
	return func(ctx context.Context, in interface{}) (interface{}, error) {
		log.Println("[authMiddleware] START: ", in)
		user := in.(User)
		if user.Username == "fail" {
			error := map[string]string{
				"error": "unauthorised",
			}
			// NOTE: we do not call the "next" middleware, and completely prevent execution of subsequent middlewares
			return error, fmt.Errorf("user %v is unauthorised", in)
		}
		res, err := next(ctx, in)
		log.Printf("[authMiddleware] END: %+v \n", in)
		return res, err
	}
}

TODO

• Cleanup interface / write tests for Vesper
• Setup CI
• Implement HandlerSignatureMiddleware
• Implement Typed Record Handler Middleware for SQS
• Implement Typed Record Handler Middleware for Kinesis
• Implement Typed Record Handler Middleware for SNS
• Write / Publish documentation
• Integrate / demo with lambda starter kit (using Message structure proposal)

Developer Documentation

Goals

1. Vesper is a middleware library - it shall provide a small API for this purpose, along with common middlewares
2. Compatibility with the AWS Go SDK interface to ensure seamless integration with tools like SAM, Serverless, 1ocal testing and so on, and to reduce cognitive overload for users
3. Preserve type safety and encourage the use of types throughout the system
4. Allow user to controls message batch semantics (e.g. ability to control concurrency)
5. Be comprehensible / avoid magic
6. Enable/allow use of user-defined messages structures

Contributing

See CONTRIBUTING.

What's in a name?

Golang has a rich history of naming HTTP and middleware type libraries on classy gin-based beverages (think Gin, Martini and Negroni). Vesper is yet another gin-based beverage.

Vesper was also inspired by Middy JS, who's mascot is a Moped.

Vesper is a (not so clever) portmanteau of Vesper, the gin-based martini, and Vespa, a beautiful Italian scooter.