gooseberry

package module
v0.0.0-...-dc23390 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 23, 2018 License: BSD-3-Clause Imports: 1 Imported by: 0

README

Gooseberry: Common Packages for Go Microservices

Build Status Go Report Card GoDoc Maintainability Test Coverage

Gooseberry is a collection of common Go packages that Voicera uses in microservices. It's an incomplete library, named after a fruit that looks like an ungrown clementine.

Features

  • Lightweight REST clients, web client with built-in pluggable debug logging (useful for new projects), basic auth support, etc.
  • Container structs like immutable maps, priority queues, sets, etc.
  • Error aggregation (multiple errors into one with a header message)
  • Leveled logger with a prefix and a wrapper for zap
  • Polling with an exponential backoff and Bernoulli trials for resetting the backoff
  • Uniform Resource Name struct that implements RFC8141 and URN helper functions generator

Quick Start

To get the latest version: go get -u github.com/voicera/gooseberry

REST Client and Polling Example

The example below creates a RESTful Twilio client to make a phone call and to poll for call history. The client uses a debug logger for requests and responses and keeps polling for calls made using an exponential backoff poller.

package main

import (
	"net/http"
	"time"

	"github.com/voicera/gooseberry"
	"github.com/voicera/gooseberry/log"
	"github.com/voicera/gooseberry/log/zap"
	"github.com/voicera/gooseberry/polling"
	"github.com/voicera/gooseberry/web"
	"github.com/voicera/gooseberry/web/rest"
)

const (
	baseURL    = "https://api.twilio.com/2010-04-01/Accounts/"
	accountSid = "AC072dcbab90350495b2c0fabf9a7817bb"
	authToken  = "883XXXXXXXXXXXXXXXXXXXXXXXXX1985"
)

type call struct {
	SID    string `json:"sid"`
	Status string `json:"status"`
}

type receiver struct {
	restClient rest.Client
}

func main() {
	gooseberry.Logger = zap.DefaultLogger
	gooseberry.Logger.Info("starting example")
	transport := web.NewBasicAuthRoundTripper(
		web.NewLeveledLoggerRoundTripper(
			http.DefaultTransport,
			log.NewPrefixedLeveledLogger(gooseberry.Logger, "TWL:")),
		accountSid, authToken)
	httpClient := &http.Client{Transport: transport}
	twilioClient := rest.NewURLEncodedRequestJSONResponseClient(httpClient).
		WithBaseURL(baseURL + accountSid)
	go makeCall(twilioClient)
	go poll(&receiver{twilioClient})
	time.Sleep(3 * time.Second)
	gooseberry.Logger.Sync()
}

func makeCall(twilioClient rest.Client) {
	parameters := map[string]string{
		"From": "+15005550006",
		"To":   "+14108675310",
		"Url":  "http://demo.twilio.com/docs/voice.xml",
	}
	call := &call{}
	if _, err := twilioClient.Post("Calls.json", parameters, &call); err != nil {
		gooseberry.Logger.Error("error making a call", "err", err)
	} else {
		gooseberry.Logger.Debug("made a call", "sid", call.SID)
	}
}

func poll(receiver *receiver) {
	poller, err := polling.NewBernoulliExponentialBackoffPoller(
		receiver, "twilio", 0.95, time.Second, time.Minute)
	if err != nil {
		gooseberry.Logger.Error("error creating a poller", "err", err)
	}
	go poller.Start()
	for batch := range poller.Channel() {
		calls := batch.([]*call)
		gooseberry.Logger.Debug("found calls", "callsCount", len(calls))
	}
}

func (r *receiver) Receive() (interface{}, bool, error) {
	calls := []*call{}
	_, err := r.restClient.Get("Calls", nil, &calls)
	return calls, len(calls) > 0, err
}

Running the above example produces output that looks like the following (which was heavily edited for brevity):

{"level":"info","ts":"2018-04-02T20:54:22Z","caller":"runtime/proc.go:198","msg":"starting example"}
{"level":"debug","ts":"2018-04-02T20:54:22Z","caller":"runtime/asm_amd64.s:2361","msg":"Started","poller":"twilio"}
{"level":"debug","ts":"2018-04-02T20:54:22Z","caller":"web/web.go:90","msg":"TWL:Request","request":"GET /2010-04-01/Accounts/AC072dcbab90350495b2c0fabf9a7817bb/Calls HTTP/1.1\r\nHost: api.twilio.com\r\nUser-Agent: gooseberry\r\nAuthorization: *******STRIPPED OUT*******\r\nContent-Type: application/x-www-fo...
{"level":"debug","ts":"2018-04-02T20:54:22Z","caller":"web/web.go:90","msg":"TWL:Request","request":"POST /2010-04-01/Accounts/AC072dcbab90350495b2c0fabf9a7817bb/Calls.json HTTP/1.1\r\nHost: api.twilio.com\r\nUser-Agent: gooseberry\r\nContent-Length: 89\r\nAuthorization: *******STRIPPED OUT*******\r\nConten...
{"level":"debug","ts":"2018-04-02T20:54:22Z","caller":"web/web.go:108","msg":"TWL:Response","response":"HTTP/1.1 401 UNAUTHORIZED\r\nContent-Length: 293\r\nAccess-Control-Allow-Credentials: true\r\nAccess-Control-Allow-Headers: Accept, Authorization, Content-Type, If-Match, If-Modified-Since, If-None-Match,...
{"level":"error","ts":"2018-04-02T20:54:22Z","caller":"runtime/asm_amd64.s:2361","msg":"HTTP Status Code 401: <?xml version='1.0' encoding='UTF-8'?>\n<TwilioResponse><RestException><Code>20003</Code><Detail>Your AccountSid or AuthToken was incorrect.</Detail><Message>Authenticate</Message><MoreInfo>https://...
{"level":"debug","ts":"2018-04-02T20:54:22Z","caller":"polling/poller.go:98","msg":"Relaxing","poller":"twilio"}
{"level":"debug","ts":"2018-04-02T20:54:22Z","caller":"web/web.go:108","msg":"TWL:Response","response":"HTTP/1.1 401 UNAUTHORIZED\r\nContent-Length: 171\r\nAccess-Control-Allow-Credentials: true\r\nAccess-Control-Allow-Headers: Accept, Authorization, Content-Type, If-Match, If-Modified-Since, If-None-Match,...
{"level":"error","ts":"2018-04-02T20:54:22Z","caller":"runtime/asm_amd64.s:2361","msg":"error making a call","err":"HTTP Status Code 401: {\"code\": 20003, \"detail\": \"Your AccountSid or AuthToken was incorrect.\", \"message\": \"Authenticate\", \"more_info\": \"https://www.twilio.com/docs/errors/20003\",...
{"level":"debug","ts":"2018-04-02T20:54:23Z","caller":"web/web.go:90","msg":"TWL:Request","request":"GET /2010-04-01/Accounts/AC072dcbab90350495b2c0fabf9a7817bb/Calls HTTP/1.1\r\nHost: api.twilio.com\r\nUser-Agent: gooseberry\r\nAuthorization: *******STRIPPED OUT*******\r\nContent-Type: application/x-www-fo...
{"level":"debug","ts":"2018-04-02T20:54:23Z","caller":"web/web.go:108","msg":"TWL:Response","response":"HTTP/1.1 401 UNAUTHORIZED\r\nContent-Length: 293\r\nAccess-Control-Allow-Credentials: true\r\nAccess-Control-Allow-Headers: Accept, Authorization, Content-Type, If-Match, If-Modified-Since, If-None-Match,...
{"level":"error","ts":"2018-04-02T20:54:23Z","caller":"runtime/asm_amd64.s:2361","msg":"HTTP Status Code 401: <?xml version='1.0' encoding='UTF-8'?>\n<TwilioResponse><RestException><Code>20003</Code><Detail>Your AccountSid or AuthToken was incorrect.</Detail><Message>Authenticate</Message><MoreInfo>https://...
{"level":"debug","ts":"2018-04-02T20:54:23Z","caller":"polling/poller.go:98","msg":"Relaxing","poller":"twilio"}
URN Example

The following example uses scripts/urns/main.go and the urn package to autogenerate helper functions for input URN namespace IDs:

go run scripts/urns/main.go -m "User=user Email=email" > urns.go

The above command results in the following go file:

package urns // auto-generated using make - DO NOT EDIT!

import (
	"strings"

	"github.com/voicera/gooseberry/urn"
)

// NewEmailURN creates a new URN with the "email"
// namespace ID.
func NewEmailURN(namespaceSpecificString string) *urn.URN {
	return urn.NewURN("email", namespaceSpecificString)
}

// IsEmailURN determines whether the specified URN uses
// "email" as its namespace ID.
func IsEmailURN(u *urn.URN) bool {
	return strings.EqualFold(u.GetNamespaceID(), "email")
}

// IsEmailURNWithValue determines whether the specified URN uses
// "email" as its namespace ID and the specified
// namespaceSpecificString as its namespace-specific string.
func IsEmailURNWithValue(u *urn.URN, namespaceSpecificString string) bool {
	return IsEmailURN(u) && strings.EqualFold(u.GetNamespaceSpecificString(), namespaceSpecificString)
}

// NewUserURN creates a new URN with the "user"
// namespace ID.
func NewUserURN(namespaceSpecificString string) *urn.URN {
	return urn.NewURN("user", namespaceSpecificString)
}

// IsUserURN determines whether the specified URN uses
// "user" as its namespace ID.
func IsUserURN(u *urn.URN) bool {
	return strings.EqualFold(u.GetNamespaceID(), "user")
}

// IsUserURNWithValue determines whether the specified URN uses
// "user" as its namespace ID and the specified
// namespaceSpecificString as its namespace-specific string.
func IsUserURNWithValue(u *urn.URN, namespaceSpecificString string) bool {
	return IsUserURN(u) && strings.EqualFold(u.GetNamespaceSpecificString(), namespaceSpecificString)
}
Inspecting Logging Calls

Logging using loosely typed key-value pairs context is convenient; for example:

log.Error("failed to run command", "exitCode", exitCode, "command", command)

However, this way of constructing arguments is susceptible to runtime issues; since the logger expects a key to be a string, the following will fail:

log.Error("failed to run command", exitCode, command)

To prevent such issues, which we've seen happen, run the following script to check that log calls are made as expected:

go run scripts/inspector/main.go -v -i .

Motivation

Common software libraries help us be more productive: saving us from reinventing the wheel; increasing transferrable knowledge across projects; and allowing us to become experts as we build them once and use them often. Moreover, they tend to have fewer bugs as they're better battle-tested and maintained. Using a common collection of libraries is a good thing for teams, companies, and the entire OSS community.

Nowadays, our ability to build complex software systems has significantly increased thanks to the growth of OSS. When we started Voicera, we intended to contribute back to the OSS community whenever possible; so when the time came to build our Go microservices, we created common packages that were intentionally designed to be OSS. After a year of using those packages in production, we felt it's time to share them with the Go community. We called the repo Gooseberry: Go is in the name; it's named after a fruit that looks like an ungrown clementine — just like our incomplete library; and it sounds like a fun name to say! With the community's help, we'd like to build gooseberry to be as commonly used as Guava is for Java.

So, nothing earth-shattering here; just a bunch of common code that we think the vast majority of Go developers should reuse in their projects. We welcome your contributions and feedback. Happy coding!

Learn More

The following can also be found at https://godoc.org/github.com/voicera/gooseberry

Documentation

Overview

Package gooseberry provides common Go libraries to import in other Go projects.

Example (PluggableLogging) 
originalLogger := Logger
defer func() { Logger = originalLogger }()

Logger = &testutil.Logger{InDebugMode: true}
fmt.Printf("Logger: %T, IsDebugEnabled: %t\n", Logger, Logger.IsDebugEnabled())
for _, log := range []func(string, ...interface{}){Logger.Debug, Logger.Info, Logger.Warn, Logger.Error} {
	log("message", "answer", 42)
}

Output:

Logger: *testutil.Logger, IsDebugEnabled: true
[Debug] message: [answer 42]
[Info] message: [answer 42]
[Warn] message: [answer 42]
[Error] message: [answer 42]

Index

Examples

Constants

This section is empty.

Variables

View Source 
var Logger log.LeveledLogger = &log.NoOpLogger{}

Logger provides a hook for importing applications to wire up their own logger for gooseberry to use. By default, logging in gooseberry is a NOOP. Optionally, one can set this logger to zap.Logger from the log/zap package.

Functions

This section is empty.

Types

This section is empty.

Source Files

View all Source files

Directories

Path Synopsis
containers
maps
Package maps provides map data structures.
 Package maps provides map data structures.
queues
Package queues provides queue data structures.
 Package queues provides queue data structures.
sets
Package sets provides set data structures.
 Package sets provides set data structures.
Package errors provides utilities for error handling.
 Package errors provides utilities for error handling.
log
Package log provides a logger interface so that logging in gooseberry is pluggable; gooseberry provides a wrapper for https://go.uber.org/zap that implements said interface.
 Package log provides a logger interface so that logging in gooseberry is pluggable; gooseberry provides a wrapper for https://go.uber.org/zap that implements said interface.
zap
Package zap provides a wrapper for https://go.uber.org/zap that implements the log.LeveledLogger interface and adds convenience features to logging with zap.
 Package zap provides a wrapper for https://go.uber.org/zap that implements the log.LeveledLogger interface and adds convenience features to logging with zap.
Package must provides functions used in package and variables initialization.
 Package must provides functions used in package and variables initialization.
Package polling provides cost-effective ways to reduce cost of polling resources.
 Package polling provides cost-effective ways to reduce cost of polling resources.
Package regex provides regular expressions utilities.
 Package regex provides regular expressions utilities.
scripts
example
inspector
Package main inspects go source files for the proper use of the log package.
 Package main inspects go source files for the proper use of the log package.
urns
This program generates URN helper functions.
 This program generates URN helper functions.
Package testutil provide test utilities for testing common packages.
 Package testutil provide test utilities for testing common packages.
Package urn provides a Uniform Resource Name that implemnets RFC8141.
 Package urn provides a Uniform Resource Name that implemnets RFC8141.
Package validate provides utilities for commonly-used validation purposes (e.g., for input validation).
 Package validate provides utilities for commonly-used validation purposes (e.g., for input validation).
web
Package web provides utilities for sending and receiving data via the web.
 Package web provides utilities for sending and receiving data via the web.
rest
Package rest provides REST clients.
 Package rest provides REST clients.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.