httpserver

Http Server Module

Http server module based on Echo.

• Installation
• Documentation
  • Usage
  • Add-ons
    • Logger
    • Error handler
    • Http Handlers
      • Debug handlers
      • Pprof handlers
      • Healthcheck handlers
    • Middlewares
      • Request id middleware
      • Request logger middleware
      • Request tracer middleware
      • Request metrics middleware
    • HTML Templates

Installation

go get github.com/ankorstore/yokai/httpserver

Documentation

Usage

This module provides a HttpServerFactory, allowing to build an echo.Echo instance.

package main

import (
	"github.com/ankorstore/yokai/httpserver"
	"github.com/labstack/echo/v4"
	"github.com/labstack/gommon/log"
)

var server, _ = httpserver.NewDefaultHttpServerFactory().Create()

// equivalent to:
var server, _ = httpserver.NewDefaultHttpServerFactory().Create(
	httpserver.WithDebug(false),                                  // debug disabled by default
	httpserver.WithBanner(false),                                 // banner disabled by default
	httpserver.WithLogger(log.New("default")),                    // echo default logger
	httpserver.WithBinder(&echo.DefaultBinder{}),                 // echo default binder
	httpserver.WithJsonSerializer(&echo.DefaultJSONSerializer{}), // echo default json serializer
	httpserver.WithHttpErrorHandler(nil),                         // echo default error handler
)

server.Start(...)

See Echo documentation for more details.

Add-ons

This module provides several add-ons ready to use to enrich your http server.

Logger

This module provides an EchoLogger, compatible with the log module:

package main

import (
	"github.com/ankorstore/yokai/httpserver"
	"github.com/ankorstore/yokai/log"
)

func main() {
	logger, _ := log.NewDefaultLoggerFactory().Create()

	server, _ := httpserver.NewDefaultHttpServerFactory().Create(
		httpserver.WithLogger(httpserver.NewEchoLogger(logger)),
	)
}

Error handler

This module provides a JsonErrorHandler, with configurable error obfuscation and call stack:

package main

import (
	"github.com/ankorstore/yokai/httpserver"
)

func main() {
	server, _ := httpserver.NewDefaultHttpServerFactory().Create(
		httpserver.WithHttpErrorHandler(httpserver.JsonErrorHandler(
			false, // without error details obfuscation
			false, // without error call stack
		)),
	)
}

You can set the parameters:

• obfuscate=true to obfuscate the error details from the response message, i.e. will use for example Internal Server Error for a response code 500 (recommended for production)
• stack=true to add the error call stack to the log and response (not suitable for production)

This will make a call to [GET] https://example.com and forward automatically the authorization, x-request-id and traceparent headers from the handler request.

Http Handlers

This module provides several debug handlers, compatible with the config module:

• DebugBuildHandler to dump current build information
• DebugConfigHandler to dump current config values
• DebugRoutesHandler to dump current registered routes on the server
• DebugVersionHandler to dump current version

package main

import (
	"github.com/ankorstore/yokai/config"
	"github.com/ankorstore/yokai/httpserver"
	"github.com/ankorstore/yokai/httpserver/handler"
)

func main() {
	cfg, _ := config.NewDefaultConfigFactory().Create()

	server, _ := httpserver.NewDefaultHttpServerFactory().Create(
		httpserver.WithDebug(true),
	)

	if server.Debug {
		server.GET("/debug/build", handler.DebugBuildHandler())
		server.GET("/debug/config", handler.DebugConfigHandler(cfg))
		server.GET("/debug/routes", handler.DebugRoutesHandler(server))
		server.GET("/debug/version", handler.DebugVersionHandler(cfg))
	}
}

This will expose [GET] /debug/* endpoints (not suitable for production).

This module provides pprof handlers, compatible with the net/http/pprof package:

• PprofIndexHandler to offer pprof index dashboard
• PprofAllocsHandler to provide a sampling of all past memory allocations
• PprofBlockHandler to provide stack traces that led to blocking on synchronization primitives
• PprofCmdlineHandler to provide the command line invocation of the current program
• PprofGoroutineHandler to provide the stack traces of all current goroutines
• PprofHeapHandler to provide a sampling of memory allocations of live objects
• PprofMutexHandler to provide stack traces of holders of contended mutexes
• PprofProfileHandler to provide CPU profile
• PprofSymbolHandler to look up the program counters listed in the request
• PprofThreadCreateHandler to provide stack traces that led to the creation of new OS threads
• PprofTraceHandler to provide a trace of execution of the current program

package main

import (
	"github.com/ankorstore/yokai/config"
	"github.com/ankorstore/yokai/httpserver"
	"github.com/ankorstore/yokai/httpserver/handler"
)

func main() {
	server, _ := httpserver.NewDefaultHttpServerFactory().Create()

	// index dashboard
	server.GET("/debug/pprof/", handler.PprofIndexHandler())

	// linked from index dashboard
	server.GET("/debug/pprof/allocs", handler.PprofAllocsHandler())
	server.GET("/debug/pprof/block", handler.PprofBlockHandler())
	server.GET("/debug/pprof/cmdline", handler.PprofCmdlineHandler())
	server.GET("/debug/pprof/goroutine", handler.PprofGoroutineHandler())
	server.GET("/debug/pprof/heap", handler.PprofHeapHandler())
	server.GET("/debug/pprof/mutex", handler.PprofMutexHandler())
	server.GET("/debug/pprof/profile", handler.PprofProfileHandler())
	server.GET("/debug/pprof/symbol", handler.PprofSymbolHandler())
	server.POST("/debug/pprof/symbol", handler.PprofSymbolHandler())
	server.GET("/debug/pprof/threadcreate", handler.PprofThreadCreateHandler())
	server.GET("/debug/pprof/trace", handler.PprofTraceHandler())
}

This will expose pprof index dashboard on [GET] /debug/pprof/, from where you'll be able to retrieve all pprof profiles types.

This module provides a HealthCheckHandler, compatible with the healthcheck module:

package main

import (
	"probes"

	"github.com/ankorstore/yokai/healthcheck"
	"github.com/ankorstore/yokai/httpserver"
	"github.com/ankorstore/yokai/httpserver/handler"
)

func main() {
	checker, _ := healthcheck.NewDefaultCheckerFactory().Create(
		healthcheck.WithProbe(probes.SomeProbe()),                            // register for startup, liveness and readiness checks           
		healthcheck.WithProbe(probes.SomeOtherProbe(), healthcheck.Liveness), // register liveness checks only
	)

	server, _ := httpserver.NewDefaultHttpServerFactory().Create()

	server.GET("/healthz", handler.HealthCheckHandler(checker, healthcheck.Startup))
	server.GET("/livez", handler.HealthCheckHandler(checker, healthcheck.Liveness))
	server.GET("/readyz", handler.HealthCheckHandler(checker, healthcheck.Readiness))
}

This will expose endpoints for k8s startup, readiness and liveness probes:

• [GET] /healthz: startup probes checks
• [GET] /livez: liveness probes checks
• [GET] /readyz: readiness probes checks

Middlewares

This module provides a RequestIdMiddleware, ensuring the request and response will always have a request id (coming by default from the X-Request-Id header or generated if missing) for correlation needs.

package main

import (
	"github.com/ankorstore/yokai/httpserver"
	"github.com/ankorstore/yokai/httpserver/middleware"
)

func main() {
	server, _ := httpserver.NewDefaultHttpServerFactory().Create()

	server.Use(middleware.RequestIdMiddleware())
}

If you need, you can configure the request header name it fetches the id from, or the generator used for missing id generation:

import (
	"github.com/ankorstore/yokai/generate/generatetest/uuid"
)

server.Use(middleware.RequestIdMiddlewareWithConfig(middleware.RequestIdMiddlewareConfig{
	RequestIdHeader: "custom-header",
	Generator: uuid.NewTestUuidGenerator("some-value"),
}))

This module provides a RequestLoggerMiddleware:

• compatible with the log module
• ensuring all log entries will contain the requests x-request-id header value by default, in the field requestID, for correlation
• ensuring a recap log entry will be emitted at request completion

You can then use the CtxLogger method to access the correlated logger from with your handlers:

package main

import (
	"github.com/ankorstore/yokai/httpserver"
	"github.com/ankorstore/yokai/httpserver/middleware"
	"github.com/ankorstore/yokai/log"
	"github.com/labstack/echo/v4"
)

func main() {
	logger, _ := log.NewDefaultLoggerFactory().Create()

	server, _ := httpserver.NewDefaultHttpServerFactory().Create(
		httpserver.WithLogger(httpserver.NewEchoLogger(logger)),
	)

	server.Use(middleware.RequestLoggerMiddleware())

	// handler
	server.GET("/test", func(c echo.Context) error {
		// emit correlated log
		httpserver.CtxLogger(c).Info().Msg("info")

		// equivalent to
		log.CtxLogger(c.Request().Context()).Info().Msg("info")
	})
}

By default, the middleware logs all requests with info level, even if failed. If needed, you can configure it to log with a level matching the response (or http error) code:

• code < 400: log level info
• 400 <= code < 500: log level warn
• code >= 500 or non http error: log level error

server.Use(middleware.RequestLoggerMiddlewareWithConfig(middleware.RequestLoggerMiddlewareConfig{
	LogLevelFromResponseOrErrorCode: true,
}))

You can configure additional request headers to log:

• the key is the header name to fetch
• the value is the log field name to fill

server.Use(middleware.RequestLoggerMiddlewareWithConfig(middleware.RequestLoggerMiddlewareConfig{
	RequestHeadersToLog: map[string]string{
		"x-header-foo": "foo",
		"x-header-bar": "bar",
	},
}))

You can also configure the request URI prefixes to exclude from logging:

server.Use(middleware.RequestLoggerMiddlewareWithConfig(middleware.RequestLoggerMiddlewareConfig{
	RequestUriPrefixesToExclude: []string{
		"/foo",
		"/bar",
	},
}))

Note: if a request to an excluded URI fails (error or http code >= 500), the middleware will still log for observability purposes.

This module provides a RequestTracerMiddleware:

• using the global tracer by default
• compatible with the trace module
• ensuring a recap trace span will be emitted at request completion

You can then use, from within your handlers the CtxTracer method to access the correlated tracer:

package main

import (
	"github.com/ankorstore/yokai/httpserver"
	"github.com/ankorstore/yokai/httpserver/middleware"
	"github.com/ankorstore/yokai/trace"
	"github.com/labstack/echo/v4"
)

func main() {
	server, _ := httpserver.NewDefaultHttpServerFactory().Create()

	server.Use(middleware.RequestTracerMiddleware("my-service"))

	// handler
	server.GET("/test", func(c echo.Context) error {
		// emit correlated span
		_, span := httpserver.CtxTracer(c).Start(c.Request().Context(), "my-span")
		defer span.End()

		// equivalent to
		_, span = trace.CtxTracerProvider(c.Request().Context()).Tracer("my-tracer").Start(c.Request().Context(), "my-span")
		defer span.End()
	})
}

If you need, you can configure the tracer provider and propagators:

import (
	"github.com/ankorstore/yokai/trace"
	"go.opentelemetry.io/otel/propagation"
)

tracerProvider, _ := trace.NewDefaultTracerProviderFactory().Create()

server.Use(middleware.RequestTracerMiddlewareWithConfig("my-service", middleware.RequestTracerMiddlewareConfig{
	TracerProvider: tracerProvider,
	TextMapPropagator: propagation.TraceContext{},
}))

And you can also configure the request URI prefixes to exclude from tracing:

server.Use(middleware.RequestTracerMiddlewareWithConfig("my-service", middleware.RequestTracerMiddlewareConfig{
	RequestUriPrefixesToExclude: []string{"/test"},
}))

This module provides a RequestMetricsMiddleware:

• ensuring requests processing count and duration are collected
• using the global promauto metrics registry by default

package main

import (
	"github.com/ankorstore/yokai/httpserver"
	"github.com/ankorstore/yokai/httpserver/middleware"
	"github.com/labstack/echo/v4"
)

func main() {
	server, _ := httpserver.NewDefaultHttpServerFactory().Create()

	server.Use(middleware.RequestMetricsMiddleware())

	// handler
	server.GET("/test", func(c echo.Context) error {
		// ...
	})
}

If you need, you can configure the metrics registry, namespace, subsystem, buckets and request path / response code normalization:

import (
	"github.com/prometheus/client_golang/prometheus"
)

registry := prometheus.NewPedanticRegistry()

server.Use(middleware.RequestMetricsMiddlewareWithConfig(middleware.RequestMetricsMiddlewareConfig{
	Registry:            registry,
	Namespace:           "foo",
	Subsystem:           "bar",
	Buckets:             []float64{0.01, 1, 10},
	NormalizeRequestPath: true,
	NormalizeResponseStatus: true,
}))

Regarding metrics normalization, if you create a handler like:

server.GET("/foo/bar/:id", func(c echo.Context) error {
	// returns a 200 response
	return c.String(http.StatusOK, c.Param("id"))
})

And receive requests on /foo/bar/baz?page=1:

• if NormalizeRequestPath=true, the metrics path label will be /foo/bar/:id, otherwise it'll be /foo/bar/baz?page=1
• if NormalizeResponseStatus=true, the metrics status label will be 2xx, otherwise it'll be 200

HTML Templates

This module provides a HtmlTemplateRenderer for rendering HTML templates.

Considering the following template:

<!-- path/to/templates/welcome.html -->
<html>
	<body>
		<h1>Welcome {{index . "name"}}!</h1>
	</body>
</html>

To render it:

package main

import (
	"net/http"

	"github.com/ankorstore/yokai/httpserver"
	"github.com/labstack/echo/v4"
)

func main() {
	server, _ := httpserver.NewDefaultHttpServerFactory().Create(
		httpserver.WithRenderer(httpserver.NewHtmlTemplateRenderer("path/to/templates/*.html")), // templates lookup pattern
	)

	// handler
	server.GET("/welcome", func(c echo.Context) error {
		return c.Render(http.StatusOK, "welcome.html", map[string]interface{}{
			"name": "some name",
		})
	})
}

See Echo templates documentation for more details.