README
¶
Server Utils Library
serverutils are utilities used by several backend services. These includes starting up and running the server.
Installing it
serverutils is compatible with modern Go releases in module mode, with Go installed:
go get -u github.com/savannahghi/serverutils
will resolve and add the package to the current development module, along with its dependencies.
Alternatively the same can be achieved if you use import in a package:
import "github.com/savannahghi/serverutils"
and run go get without parameters.
The package name is serverutils
Developing
The default branch library is main
We try to follow semantic versioning ( https://semver.org/ ). For that reason, every major, minor and point release should be tagged.
git tag -m "v0.0.1" "v0.0.1"
git push --tags
Continuous integration tests must pass on Travis CI. Our coverage threshold is 90% i.e you must keep coverage above 90%.
Environment variables
In order to run tests, you need to have an env.sh file similar to this one:
# Application settings
export DEBUG=true
export IS_RUNNING_TESTS=true
export SENTRY_DSN=<a Sentry Data Source Name>
export GOOGLE_CLOUD_PROJECT="Google Cloud project id"
This file must not be committed to version control.
It is important to export the environment variables. If they are not exported,
they will not be visible to child processes e.g go test ./....
These environment variables should also be set up on Travis CI environment variable section.
Contributing
I would like to cover the entire GitHub API and contributions are of course always welcome. The
calling pattern is pretty well established, so adding new methods is relatively
straightforward. See CONTRIBUTING.md for details.
Versioning
In general, serverutils follows semver as closely as we can for tagging releases of the package. For self-contained libraries, the application of semantic versioning is relatively straightforward and generally understood. We've adopted the following versioning policy:
- We increment the major version with any incompatible change to non-preview functionality, including changes to the exported Go API surface or behavior of the API.
- We increment the minor version with any backwards-compatible changes to functionality, as well as any changes to preview functionality in the GitHub API. GitHub makes no guarantee about the stability of preview functionality, so neither do we consider it a stable part of the go-github API.
- We increment the patch version with any backwards-compatible bug fixes.
License
This library is distributed under the MIT license found in the LICENSE file.
Documentation
¶
Index ¶
- Constants
- Variables
- func BoolEnv(envVarName string) bool
- func CloseStackDriverErrorClient(errorClient *errorreporting.Client)
- func CloseStackDriverLoggingClient(loggingClient *logging.Client)
- func ConvertStringToInt(w http.ResponseWriter, val string) int
- func CustomHTTPRequestMetricsMiddleware() func(http.Handler) http.Handler
- func DecodeJSONToTargetStruct(w http.ResponseWriter, r *http.Request, targetStruct interface{})
- func EnableStatsAndTraceExporters(ctx context.Context, service string) (func(), error)
- func ErrorMap(err error) map[string]string
- func GenerateLatencyBounds(max, interval int) []float64
- func GetEnvVar(envVarName string) (string, error)
- func GetRunningEnvironment() string
- func HealthStatusCheck(w http.ResponseWriter, r *http.Request)
- func InitOtelSDK(ctx context.Context, serviceName string) (*tracesdk.TracerProvider, error)
- func IsDebug() bool
- func IsRunningTests() bool
- func ListenAddress() string
- func LogStartupError(ctx context.Context, err error)
- func MetricsCollectorService(serviceName string) string
- func MustGetEnvVar(envVarName string) string
- func NewImportPlugin(earlySources, lateSources []*ast.Source, generate bool, path string) plugin.Plugin
- func RecordGraphqlResolverMetrics(ctx context.Context, startTime time.Time, name string, e error)
- func RecordHTTPStats(w *MetricsResponseWriter, r *http.Request)
- func RequestDebugMiddleware() func(http.Handler) http.Handler
- func Sentry() error
- func StackDriver(ctx context.Context) *errorreporting.Client
- func StartTestServer(ctx context.Context, prepareServer PrepareServer, allowedOrigins []string) (*http.Server, string, error)
- func WriteJSONResponse(w http.ResponseWriter, source interface{}, status int)
- type ErrorResponseWriter
- type ImportPlugin
- func (m *ImportPlugin) CreateSourceDirectory(path string) string
- func (m *ImportPlugin) GenerateSchemaFile(dir string, source *ast.Source)
- func (m *ImportPlugin) InjectSourceEarly() *ast.Source
- func (m *ImportPlugin) InjectSourceLate(schema *ast.Schema) *ast.Source
- func (m *ImportPlugin) MutateConfig(cfg *config.Config) error
- func (m *ImportPlugin) Name() string
- type MetricsResponseWriter
- type PrepareServer
Constants ¶
const ( // AppName is the name of "this server" AppName = "api-gateway" // DSNEnvVarName is the Sentry reporting config DSNEnvVarName = "SENTRY_DSN" // AppVersion is the app version (used for StackDriver error reporting) AppVersion = "0.0.1" // PortEnvVarName is the name of the environment variable that defines the // server port PortEnvVarName = "PORT" // DefaultPort is the default port at which the server listens if the port // environment variable is not set DefaultPort = "8080" // GoogleCloudProjectIDEnvVarName is used to determine the ID of the GCP project e.g for setting up StackDriver client GoogleCloudProjectIDEnvVarName = "GOOGLE_CLOUD_PROJECT" // DebugEnvVarName is used to determine if we should print extended tracing / logging (debugging aids) // to the console DebugEnvVarName = "DEBUG" // IsRunningTestsEnvVarName is used to determine if we are running in a test environment IsRunningTestsEnvVarName = "IS_RUNNING_TESTS" // Environment points to where this service is running e.g staging, testing, prod Environment = "ENVIRONMENT" // StagingEnv runs the service under staging StagingEnv = "staging" // DemoEnv runs the service under demo DemoEnv = "demo" // TestingEnv runs the service under testing TestingEnv = "testing" // ProdEnv runs the service under production ProdEnv = "prod" // TraceSampleRateEnvVarName indicates the percentage of transactions to be captured when doing performance monitoring TraceSampleRateEnvVarName = "SENTRY_TRACE_SAMPLE_RATE" )
const ( ResolverSuccessValue = "OK" ResolverFailureValue = "FAILED" )
Resolver status values
Variables ¶
var ( GraphqlResolverLatency = stats.Float64( "graphql_resolver_latency", "The Latency in milliseconds per graphql resolver execution", "ms", ) // Resolver is the Graphql resolver used when making a GraphQl request ResolverName = tag.MustNewKey("resolver.name") // Error is the error recorded if an error occurs ResolverErrorMessage = tag.MustNewKey("resolver.error") // ResolverStatus is used to tag whether passed or failed // it is either pass/fail...make constants ResolverStatus = tag.MustNewKey("resolver.status") GraphqlResolverLatencyView = &view.View{ Name: "graphql_resolver_latency_distribution", Description: "Time taken by a graphql resolver", Measure: GraphqlResolverLatency, Aggregation: view.Distribution(LatencyBounds...), TagKeys: []tag.Key{ResolverName, ResolverErrorMessage, ResolverStatus}, } GraphqlResolverCountView = &view.View{ Name: "graphql_resolver_request_count", Description: "The number of times a graphql resolver is executed", Measure: GraphqlResolverLatency, Aggregation: view.Count(), TagKeys: []tag.Key{ResolverName, ResolverErrorMessage, ResolverStatus}, } )
Server HTTP measures used to record metrics
var ( HTTPRequestLatency = stats.Float64( "http_request_latency", "The Latency in milliseconds per http request execution", "ms", ) // Path is the URL path (not including query string) in the request. HTTPPath = tag.MustNewKey("http.path") // StatusCode is the numeric HTTP response status code. HTTPStatusCode = tag.MustNewKey("http.status") // Method is the HTTP method of the request. HTTPMethod = tag.MustNewKey("http.method") ServerRequestLatencyView = &view.View{ Name: "http_request_latency_distribution", Description: "Time taken to process a http request", Measure: HTTPRequestLatency, Aggregation: view.Distribution(LatencyBounds...), TagKeys: []tag.Key{HTTPPath, HTTPStatusCode, HTTPMethod}, } ServerRequestCountView = &view.View{ Name: "http_request_count", Description: "The number of HTTP requests", Measure: HTTPRequestLatency, Aggregation: view.Count(), TagKeys: []tag.Key{HTTPPath, HTTPStatusCode, HTTPMethod}, } )
Views for the collected metrics i.e how they are exported to the various backends
var DefaultServiceViews = []*view.View{GraphqlResolverLatencyView, GraphqlResolverCountView, ServerRequestLatencyView, ServerRequestCountView}
DefaultServiceViews are the default/common server views provided by base package The views can be used by the various services
var LatencyBounds = GenerateLatencyBounds(60000, 200) //1 min in intervals of 200ms
LatencyBounds used in aggregating latency should be in ms i.e seconds written in ms eg 1s --> 1000ms [>=0ms, >=10ms, >=20ms, >=30ms,...., >=4s, >=5s, >=6s >=7s]
Disclaimer: The interval value should be reasonable so as to avoid many buckets. If the distribution metrics has many buckets, it will not export the metrics.
Functions ¶
func CloseStackDriverErrorClient ¶
func CloseStackDriverErrorClient(errorClient *errorreporting.Client)
CloseStackDriverErrorClient closes a StackDriver error client and logs any arising error.
It was written to be defer()'d in servrer initialization code.
func CloseStackDriverLoggingClient ¶
CloseStackDriverLoggingClient closes a StackDriver logging client and logs any arising error.
It was written to be defer()'d in servrer initialization code.
func ConvertStringToInt ¶
func ConvertStringToInt(w http.ResponseWriter, val string) int
ConvertStringToInt converts a supplied string value to an integer. It writes an error to the JSON response writer if the conversion fails.
func CustomHTTPRequestMetricsMiddleware ¶
CustomHTTPRequestMetricsMiddleware is used to implement custom metrics for our http requests The custom middleware used to collect any custom http request stats It will also be used to capture distributed trace requests and propagate them through context
func DecodeJSONToTargetStruct ¶
func DecodeJSONToTargetStruct(w http.ResponseWriter, r *http.Request, targetStruct interface{})
DecodeJSONToTargetStruct maps JSON from a HTTP request to a struct. TODO: Move to common helpers
func EnableStatsAndTraceExporters ¶
EnableStatsAndTraceExporters a wrapper for initializing metrics exporters TODO:Look into improvements
func GenerateLatencyBounds ¶
GenerateLatencyBounds is used in generating latency bounds The arguments provided should be in millisecond format e.g 1s == 1000ms interval will be used as an increment value [>=0ms, >=100ms, >=200ms, >=300ms,...., >=1000ms]
func GetEnvVar ¶
GetEnvVar retrieves the environment variable with the supplied name and fails if it is not able to do so
func GetRunningEnvironment ¶
func GetRunningEnvironment() string
GetRunningEnvironment returns the environment where the service is running. Important so as to point to the correct deps
func HealthStatusCheck ¶
func HealthStatusCheck(w http.ResponseWriter, r *http.Request)
HealthStatusCheck endpoint to check if the server is working.
func InitOtelSDK ¶ added in v0.0.3
InitOtelSDK returns an OpenTelemetry TracerProvider configured to use the Jaeger exporter for sending traces/spans. The returned TracerProvider will also use a Resource configured with all the information about the service deployment.
func IsDebug ¶
func IsDebug() bool
IsDebug returns true if debug has been turned on in the environment
func IsRunningTests ¶
func IsRunningTests() bool
IsRunningTests returns true if debug has been turned on in the environment
func ListenAddress ¶
func ListenAddress() string
ListenAddress determines what port to listen on and falls back to a default port if the environment does not supply a port
func LogStartupError ¶
LogStartupError is used to e.g log fatal startup errors. It logs, attempts to report the error to StackDriver then panics/crashes.
func MetricsCollectorService ¶
MetricsCollectorService returns name of service suffixed by it's running environment this helps identify metrics from different services at the backend/metrics viewer. e.g namespace in prometheus exporter
func MustGetEnvVar ¶
MustGetEnvVar returns the value of the environment variable with the indicated name or panics. It is intended to be used in the INTERNALS of the server when we can guarantee (through orderly coding) that the environment variable was set at server startup. Since the env is required, kill the app if the env is not set. In the event a variable is not super required, set a sensible default or don't call this method
func NewImportPlugin ¶ added in v0.0.5
func NewImportPlugin(earlySources, lateSources []*ast.Source, generate bool, path string) plugin.Plugin
NewImportPlugin initializes a new import plugin
early sources are the source files to add before loading the service schema late sources are the source files to add after loading the service schema generate is a flag to determine whether to generate schema files or not path represents the path to store the imported schema files the folder name is `exported`
func RecordGraphqlResolverMetrics ¶
RecordGraphqlResolverMetrics records the metrics for a specific graphql resolver It should be deferred until the execution of the resolver function is completed
func RecordHTTPStats ¶
func RecordHTTPStats(w *MetricsResponseWriter, r *http.Request)
RecordHTTPStats adds tags and records the metrics for a request
func RequestDebugMiddleware ¶
RequestDebugMiddleware dumps the incoming HTTP request to the log for inspection
func StackDriver ¶
func StackDriver(ctx context.Context) *errorreporting.Client
StackDriver initializes StackDriver logging, error reporting, profiling etc
func StartTestServer ¶
func StartTestServer(ctx context.Context, prepareServer PrepareServer, allowedOrigins []string) (*http.Server, string, error)
StartTestServer starts up test server
func WriteJSONResponse ¶
func WriteJSONResponse(w http.ResponseWriter, source interface{}, status int)
WriteJSONResponse writes the content supplied via the `source` parameter to the supplied http ResponseWriter. The response is returned with the indicated status. TODO: Move to common helpers
Types ¶
type ErrorResponseWriter ¶
type ErrorResponseWriter struct {
// contains filtered or unexported fields
}
ErrorResponseWriter is a http.ResponseWriter that always errors on attempted writes.
It is necessary for tests.
func NewErrorResponseWriter ¶
func NewErrorResponseWriter(err error) *ErrorResponseWriter
NewErrorResponseWriter returns an initialized ErrorResponseWriter
func (*ErrorResponseWriter) Header ¶
func (w *ErrorResponseWriter) Header() http.Header
Header delegates reading of headers to the underlying response writer
func (*ErrorResponseWriter) Write ¶
func (w *ErrorResponseWriter) Write([]byte) (int, error)
Write always returns the supplied error on any attempt to write.
func (*ErrorResponseWriter) WriteHeader ¶
func (w *ErrorResponseWriter) WriteHeader(statusCode int)
WriteHeader delegates writing of headers to the underlying response writer
type ImportPlugin ¶ added in v0.0.5
type ImportPlugin struct {
// contains filtered or unexported fields
}
ImportPlugin is a gqlgen plugin that hooks into the gqlgen code generation lifecycle and adds schema definitions from an imported library
func (*ImportPlugin) CreateSourceDirectory ¶ added in v0.0.5
func (m *ImportPlugin) CreateSourceDirectory(path string) string
CreateSourceDirectory creates the directory for the additional sources The files are necessary when publishing a service's schema to the registry
func (*ImportPlugin) GenerateSchemaFile ¶ added in v0.0.5
func (m *ImportPlugin) GenerateSchemaFile(dir string, source *ast.Source)
GenerateSchemaFile generates the associated schema file from ast source
func (*ImportPlugin) InjectSourceEarly ¶ added in v0.0.5
func (m *ImportPlugin) InjectSourceEarly() *ast.Source
InjectSourceEarly is used to inject the library schema before loading the service schema.
func (*ImportPlugin) InjectSourceLate ¶ added in v0.0.5
func (m *ImportPlugin) InjectSourceLate(schema *ast.Schema) *ast.Source
InjectSourceLate is used to inject more sources after loading the service souces
func (*ImportPlugin) MutateConfig ¶ added in v0.0.5
func (m *ImportPlugin) MutateConfig(cfg *config.Config) error
MutateConfig implements the ConfigMutator interface
func (*ImportPlugin) Name ¶ added in v0.0.5
func (m *ImportPlugin) Name() string
Name is the name of the plugin
type MetricsResponseWriter ¶
type MetricsResponseWriter struct {
StatusCode int
StartTime time.Time
// contains filtered or unexported fields
}
MetricsResponseWriter implements the http.ResponseWriter Interface it is a wrapper of http.ResponseWriter and enables obtaining measures
func NewMetricsResponseWriter ¶
func NewMetricsResponseWriter(w http.ResponseWriter) *MetricsResponseWriter
NewMetricsResponseWriter new http.ResponseWriter wrapper
func (*MetricsResponseWriter) Header ¶
func (m *MetricsResponseWriter) Header() http.Header
Header ...
func (*MetricsResponseWriter) Write ¶
func (m *MetricsResponseWriter) Write(b []byte) (int, error)
Write ...
func (*MetricsResponseWriter) WriteHeader ¶
func (m *MetricsResponseWriter) WriteHeader(code int)
WriteHeader ...
Source Files