protoslog

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 17, 2024 License: Apache-2.0 Imports: 11 Imported by: 0

README

protoslog Go Reference CI

protoslog provides utilities for using protocol buffer messages with the log/slog package introduced in Go 1.21.

Example

protoslog operates against Protocol Buffer messages. Below, one might have such a User message:

syntax="proto3";

import "google/protobuf/timestamp.proto";

message User {
  fixed64 id = 1;
  string email = 2 [debug_redact=true];
  Status status = 3;
  google.protobuf.Timestamp updated = 4;
}

enum Status {
  UNSPECIFIED = 0;
  ACTIVE = 1;
  INACTIVE = 2;
}

protoslog does NOT require any code generation (beyond the output of protoc-gen-go) to properly log a message:

package main

import (
	"log/slog"

	"github.com/rodaine/protoslog"
	"github.com/rodaine/protoslog/internal/gen"
)

func main() {
	msg := &gen.User{
		Id:      123,
		Email:   "rodaine@github.com",
		Status:  gen.ACTIVE,
		Updated: time.Now(),
	}

	slog.Info("hello", protoslog.Message("user", msg))
}

Outputs:

2022/11/08 15:28:26 INFO hello user.id=123 user.email=REDACTED user.status=ACTIVE user.updated=2022-11-08T15:28:26.000Z

Field Value Types

Messages are lazily converted into a slog.GroupValue with each of its populated field converted into a slog.Attr with the field name as the key and value produced based on its type (similar to the canonical JSON encoding rules)

Scalar Types
  • bool: slog.BoolValue
  • floats: slog.Float64Value
  • bytes: base64 encoded in a slog.StringValue
  • string: slog.StringValue
  • enum: slog.StringValue of the value name if it's defined, or slog.Int64Value otherwise
  • signed integer: slog.Int64Value
  • unsigned integer: slog.Uint64Value
Composite Types

Populated composite fields are encoded as a slog.GroupValue:

  • message: each field converted into a slog.Attr with its name as the key and the value recursively applying these rules
  • repeated: each item converted into a slog.Attr with its index string-ified as the key and the value recursively applying these rules
  • map: each entry converted into a slog.Attr with its key string-ified and the value recursively applying these rules
Well-Known Types (WKTs)

Similar to the canonical JSON encoding, some of the WKTs produce special-cased slog.Value:

  • google.protobuf.NullValue: empty slog.Value{} (equivalent of nil)\
  • google.protobuf.Timestamp: slog.TimeValue
  • google.protobuf.Duration: slog.DurationValue
  • wrappers: it's value field, applying these rules
  • google.protobuf.ListValue: its values field, applying the repeated rule above
  • google.protobuf.Struct: its fields field, applying the map rule above
  • google.protobuf.Value: the field set in its kind oneof, applying these rules
  • google.protobuf.Any: see [Any WKT Resolution] below

Redaction

Messages may contain personal identifiable information (PII), secrets, or similar data that should not be written into a log. Message fields can be annotated with the debug_redact option to identify such values. By default, protoslog will redact these fields, with the behavior customizable via options.

Populated redacted fields are replaced with a slog.StringValue("REDACTED"):

msg := &gen.User{Email: "personal@identifiable.info"}
slog.Info("default", protoslog.Message("user", msg))
// Stderr: 2022/11/08 15:28:26 INFO default user.email=REDACTED

To elide redacted fields instead of including them, WithElideRedactions can be used:

slog.Info("elide", protoslog.Message("user", msg, protoslog.WithElideRedactions()))
// Stderr: 2022/11/08 15:28:26 INFO elide

Redaction may also be disabled via WithDisableRedactions:

slog.Info("disable", protoslog.Message("user", msg, protoslog.WithDisableRedactions()))
// Stderr: 2022/11/08 15:28:26 INFO disable email=personal@identifiable.info

All Fields

By default, protoslog only emits fields that are populated on the message (via the behavior of protoreflect.Message#Has):

msg := &gen.Location{Latitude: 1.23}
slog.Info("default", protoslog.Message("loc", msg))
// Stderr: 2022/11/08 15:28:26 INFO default loc.latitude=1.23

To emit all fields regardless of presence, use WithAllFields:

slog.Info("all", protoslog.Message("loc", msg, protoslog.WithAllFields()))
// Stderr: 2022/11/08 15:28:26 INFO all loc.latitude=1.23 loc.longitude=0

For unpopulated "nullable," repeated, and map fields, the zero slog.Value is emitted (which is equivalent to nil). All other fields emit their default values.

Any WKT Resolution

protoslog emits the Any field's type_url with the key @type. By default, protoslog attempts to resolve the field's value and on success emits it:

msg := &gen.User{Id: 123}
anyPB, _ := anypb.New(msg)
slog.Info("success", protoslog.Message("any", anyPB))
// Stderr: 2022/11/08 15:28:26 INFO success any.@type=type.googleapis.com/User any.id=123

If the inner value does not resolve to a slog.GroupValue (e.g., it's a WKT), the result is added as @value:

msg := durationpb.New(5*time.Second)
anyPB, _ := anypb.New(msg)
slog.Info("wkt", protoslog.Message("any", anyPB))
// Stderr: 2022/11/08 15:28:26 INFO wkt any.@type=type.googleapis.com/google.protobuf.Duration any.@value=5s

If the value cannot be resolved (either unknown or an error occurs), only the @type attribute will be present:

anyPB := &anypb.Any{TypeUrl: "foobar"}
slog.Info("unknown", protoslog.Message("any", anyPB))
// Stderr: 2022/11/08 15:28:26 INFO unknown any.@type=foobar

By default, protoslog uses protoregistry.GlobalTypes to resolve Any WKTs. A custom resolver can be provided via WithAnyResolver:

slog.Info("custom", protoslog.Message("any", anyPB, protoslog.WithAnyResolver(myResolver)))

To skip resolving Any WKTs, use WithSkipAnys. Only the @type attribute will be emitted:

slog.Info("skip", protoslog.Message("any", anyPB, protoslog.WithSkipAnys()))

slog Handler

If a message is not wrapped via protoslog, it will be presented in the logs with the behavior of slog.AnyValue. To ensure all messages are resolved correctly regardless, a protoslog.Handler can wrap a slog.Handler:

handler := protoslog.NewHandler(slog.Default().Handler())
logger := slog.New(handler)

msg := &gen.User{Id: 123}
logger.Info("handler", "user", msg)
// Stderr: 2022/11/08 15:28:26 INFO handler user.id=123

The options on protoslog.Handler supersede those on messages wrapped via other protoslog functions.

protoc-gen-slog

To make the generated message types produced by protoc-gen-go implement slog.LogValuer, protoc-gen-slog can be used to generate LogValue methods.

go install github.com/rodaine/protoslog/protoc-gen-slog
Buf CLI

When using buf, ensure the out path and opt values are equivalent for both protoc-gen-go and protoc-gen-slog plugins:

# buf.gen.yaml
version: v1
plugins:
  - plugin: buf.build/protocolbuffers/go:v1.32.0
    out: gen
    opt:
      - paths=source_relative
  - plugin: slog
    out: gen
    opt:
      - paths=source_relative
protoc

When using protoc, ensure both plugin options and output path are equivalent:

protoc \
  --go_out="$OUT" \
  --slog_out="$OUT" \
  $PROTOS

Documentation

Overview

Package protoslog provides utilities for using protocol buffer messages with the log/slog package.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Message 

func Message(key string, msg proto.Message, options ...Option) slog.Attr

Message returns a slog.Attr for a proto.Message using the provided options. Note that these options are ignored if the slog.Value is handled by a Handler.

Example 
updated := time.Date(2012, time.September, 2, 15, 53, 0, 0, time.UTC)
msg := &pb.User{
	Id:    123,
	Name:  "foobar",
	Email: "foo@bar.com", // debug_redact
	Location: &pb.Location{
		Latitude:  1.23,
		Longitude: 4.56,
	},
	Hobbies: []string{"track", "field"},
	Pets: map[string]pb.PetType{
		"Rover": pb.PetType_PET_TYPE_DOG,
		"Fifi":  pb.PetType_PET_TYPE_CAT,
	},
	Updated:       timestamppb.New(updated),
	Best_100MTime: durationpb.New(9*time.Second + 580*time.Millisecond),
}

logger := slog.New(slogHandler())
logger.Info("some event", protoslog.Message("user", msg))

Output:

level=INFO msg="some event" user.id=123 user.name=foobar user.email=REDACTED user.location.latitude=1.23 user.location.longitude=4.56 user.hobbies.0=track user.hobbies.1=field user.pets.Fifi=PET_TYPE_CAT user.pets.Rover=PET_TYPE_DOG user.updated=2012-09-02T15:53:00.000Z user.best_100m_time=9.58s

func MessageValue 

func MessageValue(msg proto.Message, options ...Option) slog.Value

MessageValue returns a slog.Value for a proto.Message using the provided options. Note that these options are ignored if the slog.Value is handled by a Handler.

Types

type Handler 

type Handler struct {
	// contains filtered or unexported fields
}

Handler is a slog.Handler that properly converts proto.Message attributes into the appropriate slog.Value, before delegating to a child slog.Handler. The Handler's options merge with the options associated with proto.Message types that implement slog.LogValuer. Handler must be constructed via NewHandler.

Example 
msg := &pb.User{
	Id:            123,
	Best_100MTime: durationpb.New(9*time.Second + 580*time.Millisecond),
}

handler := protoslog.NewHandler(slogHandler())
logger := slog.New(handler)
logger.Info("hello world", "user", msg)

Output:

level=INFO msg="hello world" user.id=123 user.best_100m_time=9.58s
Example (AllFields) 
msg := &pb.Location{
	Latitude: 1.23,
}

childHandler := slogHandler()
logger := slog.New(protoslog.NewHandler(childHandler))
logger.Info("default", "loc", msg)

logger = slog.New(protoslog.NewHandler(childHandler, protoslog.WithAllFields()))
logger.Info("all", "loc", msg)

Output:

level=INFO msg=default loc.latitude=1.23
level=INFO msg=all loc.latitude=1.23 loc.longitude=0
Example (Any) 
msg, _ := anypb.New(&pb.Location{
	Latitude:  1.23,
	Longitude: 4.56,
})

childHandler := slogHandler()
logger := slog.New(protoslog.NewHandler(childHandler))
logger.Info("default", "any", msg)
logger.Info("unknown", "any", &anypb.Any{TypeUrl: "foobar"})

logger = slog.New(protoslog.NewHandler(childHandler, protoslog.WithSkipAnys()))
logger.Info("skip", "any", msg)

Output:

level=INFO msg=default any.@type=type.googleapis.com/Location any.latitude=1.23 any.longitude=4.56
level=INFO msg=unknown any.@type=foobar
level=INFO msg=skip any.@type=type.googleapis.com/Location
Example (Redaction)

Messages may contain personal identifiable information (PII), secrets, or similar data that should not be written into a log. Message fields can be annotated with the debug_redact option to identify such values. By default, protoslog will redact these fields, with the behavior customizable via [options]. 

// message User { fixed64 id = 1; string email = 2 [debug_redact=true]; }
msg := &pb.User{
	Id:    123,
	Email: "personal@identifiable.info",
}

childHandler := slogHandler()
logger := slog.New(protoslog.NewHandler(childHandler))
logger.Info("default", "user", msg)

logger = slog.New(protoslog.NewHandler(childHandler, protoslog.WithDisableRedactions()))
logger.Info("disabled", "user", msg)

logger = slog.New(protoslog.NewHandler(childHandler, protoslog.WithElideRedactions()))
logger.Info("elided", "user", msg)

Output:

level=INFO msg=default user.id=123 user.email=REDACTED
level=INFO msg=disabled user.id=123 user.email=personal@identifiable.info
level=INFO msg=elided user.id=123

func NewHandler 

func NewHandler(child slog.Handler, options ...Option) *Handler

NewHandler creates a Handler that delegates to child, using the given options. Note that these options merge with any options used in Message, MessageValue, or MessageValuer.

func (Handler) Enabled 

func (h Handler) Enabled(ctx context.Context, level slog.Level) bool

Enabled delegates this check to its child handler.

func (Handler) Handle 

func (h Handler) Handle(ctx context.Context, record slog.Record) error

Handle converts the proto.Message attributes on record before delegating the record to its child handler.

func (Handler) WithAttrs 

func (h Handler) WithAttrs(attrs []slog.Attr) slog.Handler

WithAttrs converts the proto.Message attributes before delegating them to its child handler.

Example 
loc := &pb.Location{Latitude: 1.23}
msg := &pb.User{Id: 456}

logger := slog.New(protoslog.NewHandler(slogHandler()))
logger.With("loc", loc).Info("attrs", "user", msg)

Output:

level=INFO msg=attrs loc.latitude=1.23 user.id=456

func (Handler) WithGroup 

func (h Handler) WithGroup(name string) slog.Handler

WithGroup delegates starting this group on its child handler.

Example 
msg := &pb.User{Id: 123}

logger := slog.New(protoslog.NewHandler(slogHandler()))
logger.WithGroup("foo").Info("group", "user", msg)

Output:

level=INFO msg=group foo.user.id=123

type Option 

type Option func(o *options)

Option functions customize generation of a slog.Value from a proto.Message beyond the default behavior.

func WithAllFields 

func WithAllFields() Option

WithAllFields indicates that all fields, including unpopulated ones, should be included in slog.Value. Unpopulated members of a oneof are still excluded from the output.

func WithAnyResolver 

func WithAnyResolver(resolver protoregistry.MessageTypeResolver) Option

WithAnyResolver is the protoregistry.MessageTypeResolver used to resolve the google.protobuf.Any well-known type into a valid slog.Value. When nil, protoregistry.GlobalTypes is used. If the type cannot be found in the resolver or if unmarshaling fails, only a "@type" field is emitted.

func WithDisableRedactions 

func WithDisableRedactions() Option

WithDisableRedactions indicates that fields annotated with the debug_redact option should not be redacted from the slog.Value.

func WithElideRedactions 

func WithElideRedactions() Option

WithElideRedactions indicates that redacted fields should be removed from the slog.Value instead of being replaced with REDACTED. WithElideRedactions supersedes WithAllFields.

func WithSkipAnys 

func WithSkipAnys() Option

WithSkipAnys indicates that google.protobuf.Any fields should not be unmarshalled during construction of the slog.Value, emitting only a "@type" field.

type Valuer 

type Valuer struct {
	// Message is the proto.Message to produce the slog.Value.
	Message proto.Message
	// contains filtered or unexported fields
}

Valuer implements slog.LogValuer for a proto.Message to defer computing a slog.Value until it's needed.

func MessageValuer 

func MessageValuer(msg proto.Message, options ...Option) Valuer

MessageValuer returns a Valuer for a proto.Message using the provided options. Note that these options are ignored if the Valuer is handled by a Handler.

func (Valuer) LogValue 

func (v Valuer) LogValue() slog.Value

LogValue satisfies the slog.LogValuer interface.

Source Files

View all Source files

Directories

Path Synopsis
internal
gen

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.