telemetry

Telemetry

Module to assist in setting up OpenTelemetry integration for trace exporter.

Four exporter types are supported:

• none: no output generated
• stdout: output data directly to screen
• file: output data to a local file
• sumo: use grpc exporter to send data to sumo logic

Output to stdout and file are intended for development and testing use only, e.g. for testing output could be directed to /dev/null when testing code only and not concerned with actual outputs.

Configuration

When creating a new tracer, the type of exporter is specified by the Config.Mode.

• For the file exporter Config.File must be specified
• For the sumo exporter Config.Endpoint must be specified

Config.Attributes allows for a set of key value pairs to be set on the trace resource

Recommended attributes to set:

• application, eg 'CaaS Seller', 'Caas File', 'TAP'
• service.name
• environment

	tc := telemetry.Config{
		Mode: mode,
		Attributes: map[string]string{
			"service.name": s.env.Get(constants.VarAppName),
			"application":  telAppName,
			"environment":  s.env.Get(constants.VarAppEnv),
		},
	}

	switch mode {
	case telemetry.Sumo:
		tc.Endpoint = s.env.Get(constants.VarTelemetryEndpoint)
	case telemetry.File:
		fileName := s.env.Get(constants.VarTelemetryFile)
		fh, err := os.Create(fileName)
		if err != nil {
			l.Warn().Err(err).Str("file", fileName).Msg("Failed creating file")
		}
		s.traceFile = fh // track here, so we can close file on exit
		tc.File = fh
	default:
	}

	s.telemeter, err = telemetry.NewTracer(tc)
	if err != nil {
		l.Error().Err(err).Msg("Failed initialising telemetry tracer")
		return err
	}

Middleware

A middleware wrapper of the otlp gorilla/mux middleware to provide a urfave/negroni compatible middleware. If using gorilla/mux middleware directly, this should not be required.

// mux middleware:

import "go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux"

func main() {
	router := mux.NewRouter()
	router.Use(otelmux.Middleware("appName"))
	...
}

// wrapped mux middleware:

import (
	"github.com/urfave/negroni"

	"gitlab.com/msts-public/general/gomods/telemetry"
)

func main() {
	router := mux.NewRouter()
	router.HandleFunc("/", BaseHandler)

	n := negroni.New(telemetry.NewMiddleware("appName"))
	n..UseHandler(router)
	...
}

Logger Hook

A logger hook is available to assist in adding error messages to the trace when logging errors.

Example: adding hook to middleware logger

func (c *Logger) ServeHTTP(w http.ResponseWriter, r *http.Request, next http.HandlerFunc) {

	l := c.logger.With().Logger()

	span := trace.SpanFromContext(r.Context())
	if span != nil {
		th := telemetry.TraceHook{Span: span}
		l = l.Hook(th)
	}

	r = r.WithContext(l.WithContext(r.Context()))
	next(w, r)
}

References

• Sumo logic instrumentation setup: https://help.sumologic.com/Traces/Getting_Started_with_Transaction_Tracing/Instrument_your_application_with_OpenTelemetry/Go_OpenTelemetry_auto-instrumentation
• OpenTelemetry getting started: https://opentelemetry.io/docs/go/getting-started
• OpenTelemetry docs: https://opentelemetry.io/docs/
• OpenTelemetry - gorilla/mux integration: https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/instrumentation/github.com/gorilla/mux/otelmux/example