ctrace

ctrace-go

Canonical OpenTracing for Go

Why

OpenTracing is a young specification and for most (if not all) SDK implementations, output format and wire protocol are specific to the backend platform implementation. ctrace attempts to decouple the format and wire protocol from the backend tracer implementation.

What

ctrace specifies a canonical format for trace logs. By default the logs are output to stdout but you can configure them to go to any WritableStream.

Required Reading

To fully understand this platform API, it's helpful to be familiar with the OpenTracing project project, terminology, and ctrace specification more specifically.

Install

Install via glide as follows:

$ glide get github.com/Nordstrom/ctrace-go

Usage

Add instrumentation to the operations you want to track. Most of this is done by middleware.

Initialization

First import ctrace.

import (
	ctrace "github.com/Nordstrom/ctrace-go"
)

Instrumentation

The primary way to use ctrace-go is to hook in one or more of the auto-instrumentation decorators. These are the decorators we currently support.

• TracedHttpHandler - this wraps the default serve mux to trace Incoming HTTP Requests.
• TracedHttpClientTransport - this wraps the http.Transport for tracing Outgoing HTTP Requests.
• TracedAPIGwLambdaProxyHandler - this wraps AWS Gateway Lambda Proxy handler using AWS Lambda Go Shim to trace requests coming into an AWS Lambda from an API Gateway proxy event.

TracedHttpHandler

To automatically instrument incoming HTTP Requests use the TracedHttpHandler.

import (
	"net/http"

	ctrace "github.com/Nordstrom/ctrace-go"
)

func ok(w http.ResponseWriter, r *http.Request) {
	region := r.URL.Query().Get("region")
	msg := hello(r.Context(), region)
	w.Write([]byte(msg))
}

func main() {
	http.HandleFunc("/ok", ok)
	http.ListenAndServe(":8004", ctrace.TracedHTTPHandler(http.DefaultServeMux))
}

TracedHTTPClientTransport

To automatically instrument outgoing HTTP Requests use the TracedHttpClientTransport.

import (
	"context"
	"net/http"

	ctrace "github.com/Nordstrom/ctrace-go"
)

var httpClient = &http.Client{
	Transport: ctrace.TracedHTTPClientTransport(&http.Transport{}),
}

func send(ctx context.Context, method string, url string, body io.Reader) (*http.Response, error) {
	req, err := http.NewRequest(method, url, body)
	if err != nil {
		return nil, err
	}
	return httpClient.Do(req.WithContext(ctx))
}

TracedAPIGwLambdaProxyHandler

To automatically instrument incoming API Gateway Lambda proxy requests use TracedAPIGwLambdaProxyHandler.

import (
	"github.com/eawsy/aws-lambda-go-core/service/lambda/runtime"
	"github.com/eawsy/aws-lambda-go-event/service/lambda/runtime/event/apigatewayproxyevt"
	ctrace "github.com/Nordstrom/ctrace-go"
)

handler := func(
	ctx context.Context,
	evt *apigatewayproxyevt.Event,
	lambdaCtx *runtime.Context,
) (interface{}, error) {
  // ...
}

var TracedHandler = ctrace.TracedAPIGwLambdaProxyHandler(handler)

Log Events

To log events within the context of the current Span, use span.LogFields.

import (
	ctrace "github.com/Nordstrom/ctrace-go"
	log "github.com/Nordstrom/ctrace-go/log"
)

func hello(ctx context.Context, region string) string {
	msg := fmt.Sprintf("Hello %v!", region)
	ctrace.LogInfo(ctx, "generate-msg", log.Message(message))
	return msg
}

Advanced Usage

If middleware does not fully meet your needs, you can manually instrument spans operations of interest and adding log statements to capture useful data relevant to those operations.

func main() {
	ctrace.Init(TracerOptions{
		MultiEvent: true,
		Writer:     fileWriter,
	})
}

Creating a Span given an existing Go context.Context

If you use context.Context in your application, OpenTracing's Go library will happily rely on it for Span propagation. To start a new (blocking child) Span, you can use StartSpanFromContext.

func xyz(ctx context.Context, ...) {
    // ...
    span, ctx := opentracing.StartSpanFromContext(ctx, "operation_name")
    defer span.Finish()
    span.LogFields(
        log.String("event", "soft error"),
        log.String("type", "cache timeout"),
        log.Int("waited.millis", 1500))
    // ...
}

Starting an empty trace by creating a "root span"

It's always possible to create a "root" Span with no parent or other causal reference.

func xyz() {
   // ...
   sp := opentracing.StartSpan("operation_name")
   defer sp.Finish()
   // ...
}

Best Practices

The following are recommended practices for using opentracing and ctrace-go within a GoLang project.

Context Propagation

Require a context.Context argument as the first parameter of every(a) API call

func (h handler) HandleRequest(ctx context.Context, r MyRequest) error {
  // ...
}

Rule of thumb: because contexts are always request-scopeed, never hold a reference to them on a struct. Always pass as a function parameter.

a. Obviously not every function in your codebase. You'll get a feel for the balance when you start writing context-aware code.

Custom Context Types

At some point, you will be tempted to invent your own "custom" context type. For example to provide convenience methods, since Value() takes an interface{} key and returns an interface{}.

You will regret it. Use extractor functions instead.

Contributing

Please see the contributing guide for details on contributing to ctrace-go.

License

Apache License 2.0