gop

package module
v0.0.0-...-d4fa126 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 12, 2018 License: BSD-3-Clause Imports: 28 Imported by: 0

README

Gop

Gop (Go-in-Production) is an application container framework.

Gop makes it easier to write, deploy and maintain Go applications in a production environment.

Gop strongly follows convention-over-configuration principles. Things like file locations and formats are standardized wherever it makes sense for them to be.

Features

Gop applications get the following features:

  • Config file
  • Logging
  • HTTP microframework
  • Graceful restarts
  • Process manager
  • Resource tracking and limits
  • Runtime config overrides
  • Introspection
  • Go runtime control

More information about each of these features is below.

Getting started

Gop requires go 1.2 or higher.

First, install the gop package:

go get github.com/trendmicro/gop

Gop applications are identified by a project name and an app name. Let's imagine our app is called helloworld and is part of a project called gopexamples.

Create helloworld.go:

package main

import (
	"github.com/trendmicro/gop"
)

// hello is a basic HTTP handler
func hello(g *gop.Req) error {
	return g.SendText([]byte("Hello, world!\n"))
}

// main initializes the gop app
func main() {
	app := gop.Init("gopexamples", "helloworld")
	app.HandleFunc("/hello", hello)
	app.Run()
}

Every gop app has a config file.

Create /etc/gopexamples/helloworld.conf. The filename is important. It must match the project and app names of your app.

[gop]
listen_addr=:8888
stdout_only_logging=true

And run your app:

go run helloworld.go

You should be able to access your handler at http://localhost:8888/hello.

Getting help

Visit the Gop website.

Issues and bugs can be reported at the Github issue tracker for gop.

Contents

Authors

  • John Berthels
  • Mark Boyd

Documentation

Overview

GOP (GOlang-in-Production) is a collection of code intended to ease writing production-safe golang applications, particularly web applications.

Providing configuration

GOP expects you to create an ini-formatted configuration file for your application. It will attempt to load it during the call gop.Init(), and will raise an error if it cannot find it.

Here is how GOP determines the location of the configuration file:

  • If you set the environment variable called $PROJECT_$APP_CFG_FILE (all uppercase), where $PROJECT and $APP are the projectName and appName parameters that you have passed to gop.Init() as arguments, GOP will check that location for the configuration file

  • If you set the environment variable called $PROJECT_CFG_ROOT (all uppercase), GOP will check that directory for the file named $APP.conf

  • If you did not set any of these environment variables, GOP will look for a file named $APP.conf in /etc/$PROJECT

It should be emphasized that GOP will check only one location. It means that if you specified $PROJECT_$APP_CFG_FILE and the file does not exist, GOP will raise an error.

To summarize it: 

pathToConfigFile = $PROJECT_$APP_CFG_FILE || $PROJECT_CFG_ROOT/$APP.conf || /etc/$PROJECT/$APP.conf

Overriding configuration

There are certain cases, when you may want to override parts of your configuration. GOP provides a mechanism for doing that. Simply create a JSON file next to the config file that GOP will use. The file should have the same name as that config file, but also have the ".override" extension appended to it. Example: 

Config:    /etc/my_gop_project/my_gop_app.conf
Override:  /etc/my_gop_project/my_gop_app.conf.override

In fact, GOP will warn you if it does not find the overrides file. And an empty file will not satisfy it - it has to be a valid JSON.

There is also a restriction to the contents of the overrides file:

  • The root element must be an associative array (can be empty)
  • The keys of the root element must be strings (section names)
  • The values of the root element must be associative arrays (section options)
  • The keys and values of the associative arrays that are the values of the root element must be quoted

To illustrate these requirements: 

[]                              # Bad. Root element is not an associative array.
{"version": "2"}                # Bad. Values of the root element must be associative arrays.
{"overrides": {"version": 2}}   # Bad. Version is not quoted.
{"overrides": {"version": "2"}} # Good.
{}                              # Good. Minimum viable config.

Accessing configuration

You can access the application's configuration via the Cfg property of the app instance returned by gop.Init(). This property has type Config.

Logging

GOP uses Timber (https://github.com/jbert/timber) for logging. A *gop.App instance embeds the interface of timber.Logger, which means all of its methods can be accessed like this: 

app := gop.Init("myproject", "myapp")
app.Debug("My debug message")

Configuring Logging

The logger is configured during the call to gop.Init(). The following options are available in the gop section of the configuration file (values shown below are default): 

log_pattern         = "[%D %T] [%L] %S"                 # Log message format accepted by Timber
log_filename        = false                             # Show file path and line number of the method that created log message.
                                                        #   This option may not work with custom log pattern (include %S to avoid it).

log_dir             = /var/log                          # Directory where GOP will look for the project's log directory
log_file            = $log_dir/$project_name/$app.log   # Full path to the log file
log_level           = INFO                              # Case-insensitive log level accepted by Timber: Finest, Fine, Debug, Trace, Info, Warn, Error, Critical
stdout_only_logging = false                             # Output log to STDOUT instead of the log file

If the path to the log_file does not exist and stdout_only_logging is false, GOP will raise an error.

GOP HTTP Handlers

GOP provides a few HTTP handlers, all beginning with "/gop", that you can enable by setting enable_gop_urls to true in the gop section of your configuration file. Otherwise, GOP will respond with "not enabled" when you will try to access those handlers.

The following handlers are available: 

  /gop/config/:section/:key

    When the HTTP verb is PUT, GOP will override the config setting specified by :section and :key (the value
    should be specified in the body of the request).

    An example command line to change a config value using curl is:

    echo -n info | curl -s -T - http://127.0.0.1:1732/gop/config/gop/log_level

    When the HTTP verb is GET, you can read a specific key value. You can also omit :key or both :key and :section to return sections or the entire config.

 /gop/status

    TODO

 /gop/stack

    TODO

 /gop/mem

    TODO

 /gop/test?secs=int&kbytes=int

    TODO

	GOP "Go in Production" is an attempt to provide a useful set of services for running
	(primarily http) applications in production service.

	This includes:
		- configuration
		- logging
		- statsd integration
		- signal handling
		- resource management
		- basic web framework

Index

Constants

This section is empty.

Variables

View Source 
var CloseAbnormalClosure = WebSocketCloseMessage{Code: websocket.CloseAbnormalClosure}
View Source 
var CloseGoingAway = WebSocketCloseMessage{Code: websocket.CloseGoingAway}
View Source 
var ClosePolicyViolation = WebSocketCloseMessage{Code: websocket.ClosePolicyViolation}

Functions

func BadRequest 

func BadRequest(body string) error

Helper to generate a BadRequest HTTPError

func NotFound 

func NotFound(body string) error

Helper to generate a NotFound HTTPError

func PolicyViolation 

func PolicyViolation(body string) error

func ServerError 

func ServerError(body string) error

Helper to generate an InternalServerError HTTPError

Types

type App 

type App struct {
	AppName       string
	ProjectName   string
	GorillaRouter *mux.Router
	// contains filtered or unexported fields
}

Represents a gop application. Create with gop.Init(projectName, applicationName)

func Init 

func Init(projectName, appName, version string) *App

Set up the application. Reads config. Panic if runtime environment is deficient.

func InitCmd 

func InitCmd(projectName, appName, version string) *App

For test code and command line tools

func InitCmdWithLogFormatter 

func InitCmdWithLogFormatter(
	projectName, appName, version string,
	logFormatterFactory LogFormatterFactory,
) *App

For test code and command line tools

func InitWithLogFormatter 

func InitWithLogFormatter(
	projectName, appName, version string,
	logFormatterFactory LogFormatterFactory,
) *App

Set up the application. Reads config. Panic if runtime environment is deficient.

func (*App) ConfigServe 

func (a *App) ConfigServe()

func (*App) Finish 

func (a *App) Finish()

Shut down the app cleanly. (Needed to flush logs)

func (*App) GetStats 

func (a *App) GetStats() AppStats

func (*App) HTTPHandler 

func (a *App) HTTPHandler(u string, h http.Handler)

func (*App) HandleFunc 

func (a *App) HandleFunc(u string, h HandlerFunc, requiredParams ...string) *mux.Route

Register an http handler managed by gop. We use Gorilla muxxer, since it is back-compatible and nice to use :-)

func (*App) HandleMap 

func (a *App) HandleMap(hm map[string]func(g *Req) error)

func (*App) HandleWebSocketFunc 

func (a *App) HandleWebSocketFunc(u string, h HandlerFunc, requiredParams ...string) *mux.Route

func (*App) Hostname 

func (a *App) Hostname() string

Hostname returns the apps hostname, os.Hostname() by default, but this can be overridden via gop.hostname config. This call is used when setting up logging and stats allowing a gop app to lie about it's hostname, useful in environments where the hostname may be the same across machines.

func (*App) Run 

func (a *App) Run()

func (*App) Serve 

func (a *App) Serve(l net.Listener)

func (*App) Shutdown 

func (a *App) Shutdown(reason string)

func (*App) Start 

func (a *App) Start()

func (*App) StartTime 

func (a *App) StartTime() time.Time

Uptime returns time instant the app was initialized.

func (*App) WrapHandler 

func (a *App) WrapHandler(h HandlerFunc, requiredParams ...string) http.HandlerFunc

func (*App) WrapWebSocketHandler 

func (a *App) WrapWebSocketHandler(h HandlerFunc, requiredParams ...string) http.HandlerFunc

func (*App) WriteAccessLog 

func (a *App) WriteAccessLog(req *Req, dur time.Duration)

type AppStats 

type AppStats struct {
	// contains filtered or unexported fields
}

func (AppStats) String 

func (as AppStats) String() string

type Client 

type Client struct {
	AppURL url.URL
}

Client to access the web api exposed under the gop/ url. For use by code that wants to talk to your gop applications. See the gopctl commandline tool, which wraps this client.

func NewClient 

func NewClient(appstr string) (client *Client, err error)

NewClient creates a new client for given gop app. Pass a string url pointing at the application, e.g. http://localhost:2342/gop. You can drop the /gop bit of the path and use a root url, /gop get added automatically. You can drop http:// and it will be added for you. If string starts with a : it is assumed to be a url starting with the port, on localhost. 

// These all point to the same app
c, err := NewClient("http://localhost:2342/gop")
c, err := NewClient("http://localhost:2342")
c, err := NewClient("localhost:2342")
c, err := NewClient(":2342")

func (*Client) Cfg 

func (c *Client) Cfg() (cfg ConfigMap, err error)

Cfg reads the config gop/url returning the current ConfigMap.

func (*Client) GetJSON 

func (c *Client) GetJSON(path string, v interface{}) (err error)

GetJSON gets the path from the path given relative to the app URL, parsing it into the interface v given.

func (*Client) GetText 

func (c *Client) GetText(path string) (txt string, err error)

GetText gets the path from the path given relative to the app URL, returning it as a string.

func (*Client) Mem 

func (c *Client) Mem() (mem MemInfo, err error)

Mem reads the gop/mem url.

func (*Client) SetCfg 

func (c *Client) SetCfg(section, key, val string) (resptxt string, err error)

SetCfg updates the given section key with a new value by doing a PUT to the gop/config/{section}/{key} url.

func (*Client) Stack 

func (c *Client) Stack() (stack StackInfo, err error)

Stack reads the gop/stack url.

func (*Client) Status 

func (c *Client) Status() (status StatusInfo, err error)

Status reads the gop/status url.

type Config 

type Config struct {
	// contains filtered or unexported fields
}

func (*Config) AddOnChangeCallback 

func (cfg *Config) AddOnChangeCallback(f func(cfg *Config))

func (*Config) AsMap 

func (cfg *Config) AsMap() map[string]map[string]string

Get a copy of the config as a map that maps each section to a map that maps the options to the values.

func (*Config) Get 

func (cfg *Config) Get(sectionName, optionName string, defaultValue string) (string, bool)

func (*Config) GetBool 

func (cfg *Config) GetBool(sectionName, optionName string, defaultValue bool) (bool, bool)

Same as Config.Get, but returns the value as boolean. The option value should be one that strconv.ParseBool understands.

func (*Config) GetDuration 

func (cfg *Config) GetDuration(sectionName, optionName string, defaultValue time.Duration) (time.Duration, bool)

Same as Config.Get but returns the value as time.Duration. The value in the config file should be in the format that time.ParseDuration() understands.

func (*Config) GetFloat32 

func (cfg *Config) GetFloat32(sectionName, optionName string, defaultValue float32) (float32, bool)

Same as Config.Get, but returns the value as float32.

func (*Config) GetFloat64 

func (cfg *Config) GetFloat64(sectionName, optionName string, defaultValue float64) (float64, bool)

Same as Config.Get, but returns the value as float64

func (*Config) GetInt 

func (cfg *Config) GetInt(sectionName, optionName string, defaultValue int) (int, bool)

Same as Config.Get, but returns the value as int.

func (*Config) GetInt64 

func (cfg *Config) GetInt64(sectionName, optionName string, defaultValue int64) (int64, bool)

Same as Config.Get, but returns the value as int64. The integer has to be written in the config in decimal format. This means that for the value written in the config as "08" this method will return 8 instead of 10. And "0x8" will generate an error.

func (*Config) GetList 

func (cfg *Config) GetList(sectionName, optionName string, defaultValue []string) ([]string, bool)

Return a list of strings for a config value that is written as a comma-separated list. Each value will be stripped out of leading and trailing white spaces as defined by Unicode.

func (*Config) GetMap 

func (cfg *Config) GetMap(sectionName, kPrefix string, defaultValue map[string]string) (map[string]string, bool)

func (*Config) GetPath 

func (cfg *Config) GetPath(sectionName, optionName string, defaultValue string) (string, bool)

Same as Config.Get but consider the string as a filename path and expands ~ characters to the homedir of the current uid

func (*Config) PersistentOverride 

func (cfg *Config) PersistentOverride(sectionName, optionName, optionValue string)

func (*Config) SectionKeys 

func (cfg *Config) SectionKeys(sectionName string) []string

Get a list of options for the named section, including those specified in the override file.

func (*Config) Sections 

func (cfg *Config) Sections() []string

Get a list of the names of the available sections, including those specified in the override file.

func (*Config) TransientOverride 

func (cfg *Config) TransientOverride(sectionName, optionName, optionValue string)

type ConfigMap 

type ConfigMap map[string]map[string]string

func (*ConfigMap) Add 

func (cfgMap *ConfigMap) Add(sectionName, optionName, optionValue string)

Set the given option to the specified value for the named section. Create the section if it does not exist.

func (*ConfigMap) Get 

func (cfgMap *ConfigMap) Get(sectionName, optionName string, defaultValue string) (string, bool)

Get an option value for the given sectionName. Will return defaultValue if the section or the option does not exist. The second return value is True if the requested option value was returned and False if the default value was returned.

func (*ConfigMap) SectionKeys 

func (cfgMap *ConfigMap) SectionKeys(sectionName string) []string

Get a list of options for the named section. Will return an empty list if the section does not exist.

func (*ConfigMap) Sections 

func (cfgMap *ConfigMap) Sections() []string

Get a list of the names of the avaliable sections.

type ConfigSource 

type ConfigSource interface {
	Get(sectionName, optionName string, defaultValue string) (string, bool)
	Add(sectionName, optionName, optionValue string)
	Sections() []string
	SectionKeys(sectionName string) []string
}

type GoroInfo 

type GoroInfo struct {
	// contains filtered or unexported fields
}

GoroInfo contains the info on a single goro from a stack parsed by StackInfo.

func (GoroInfo) Head 

func (gi GoroInfo) Head() string

Head returns the 1st line of the goro info.

func (GoroInfo) RoutineLines 

func (gi GoroInfo) RoutineLines() []string

RoutineLines returns the all the lines after the first.

func (GoroInfo) String 

func (gi GoroInfo) String() string

type HTTPError 

type HTTPError struct {
	Code int
	Body string
}

Return one of these from a handler to control the error response Returning nil if you have sent your own response (as is typical on success) 

var ErrBadRequest HTTPError = HTTPError{Code: http.StatusBadRequest}
var ErrNotFound HTTPError = HTTPError{Code: http.StatusNotFound}

Simple helpers for common HTTP error cases 

var ErrServerError HTTPError = HTTPError{Code: http.StatusInternalServerError}

func (HTTPError) Error 

func (h HTTPError) Error() string

To satisfy the interface only

func (HTTPError) Write 

func (h HTTPError) Write(w *responseWriter)

type HandlerFunc 

type HandlerFunc func(g *Req) error

The function signature your http handlers need.

type LogFormatterFactory 

type LogFormatterFactory interface {
	Create() timber.LogFormatter
}

type Logger 

type Logger timber.Logger

type LogglyWriter 

type LogglyWriter struct {
	// contains filtered or unexported fields
}

LogglyWriter is a Timber writer to send logging to the loggly service. See: https://loggly.com.

func NewLogglyWriter 

func NewLogglyWriter(token string, tags ...string) (*LogglyWriter, error)

NewLogEntriesWriter creates a new writer for sending logging to Loggly.

func (*LogglyWriter) Close 

func (w *LogglyWriter) Close()

Close the write. Satifies the timber.LogWriter interface.

func (*LogglyWriter) LogWrite 

func (w *LogglyWriter) LogWrite(msg string)

LogWrite the message to the logenttries server async. Satifies the timber.LogWrite interface.

type MemInfo 

type MemInfo runtime.MemStats

func (MemInfo) String 

func (mem MemInfo) String() (s string)

type Req 

type Req struct {
	R            *http.Request
	RealRemoteIP string
	IsHTTPS      bool
	// Only one of these is valid to use...
	W           *responseWriter
	WS          *websocket.Conn
	WsCloseChan chan struct{}
	CanBeSlow   bool //set this to true to suppress the "Slow Request" warning
	// contains filtered or unexported fields
}

Per request struct. has convenience references to functionality in the app singleton. Passed into the request handler.

func (*Req) Param 

func (g *Req) Param(key string) (string, error)

func (*Req) ParamBool 

func (g *Req) ParamBool(key string) (bool, error)

func (*Req) ParamDuration 

func (g *Req) ParamDuration(key string) (time.Duration, error)

func (*Req) ParamInt 

func (g *Req) ParamInt(key string) (int, error)

func (*Req) ParamTime 

func (g *Req) ParamTime(key string) (time.Time, error)

func (*Req) Params 

func (g *Req) Params() map[string]string

func (*Req) Render 

func (g *Req) Render(templateData interface{}, templates ...string) error

func (*Req) SendHtml 

func (g *Req) SendHtml(v []byte) error

SendHtml sends the given []byte with the mimetype "text/html". The []byte must be in UTF-8 encoding.

func (*Req) SendJson 

func (g *Req) SendJson(what string, v interface{}) error

SendJson marshals the given v into JSON and sends it with the mimetype "application/json". what is a human-readable name for the thing being marshalled.

func (*Req) SendText 

func (g *Req) SendText(v []byte) error

SendText sends the given []byte with the mimetype "text/plain". The []byte must be in UTF-8 encoding.

func (*Req) WebSocketWriteBinary 

func (g *Req) WebSocketWriteBinary(buf []byte) error

func (*Req) WebSocketWriteText 

func (g *Req) WebSocketWriteText(buf []byte) error

type RequestInfo 

type RequestInfo struct {
	Id       int
	Method   string
	Url      string
	Duration float64
	RemoteIP string
	IsHTTPS  bool
}

RequestInfo details of an open request

func (RequestInfo) String 

func (r RequestInfo) String() string

type StackInfo 

type StackInfo struct {
	// contains filtered or unexported fields
}

StringInfo represents a go runtime stack.

func ParseStackInfo 

func ParseStackInfo(txt string) StackInfo

ParseStackInfo produces a StackInfo struct by parsing a stack string.

func (StackInfo) Goros 

func (si StackInfo) Goros() (gr []*GoroInfo)

Goros returns list of goroutines in the stack.

func (StackInfo) String 

func (si StackInfo) String() string

type StatsdClient 

type StatsdClient struct {
	*App
	sync.Mutex
	// contains filtered or unexported fields
}

func (*StatsdClient) Dec 

func (s *StatsdClient) Dec(stat string, value int64)

func (*StatsdClient) Gauge 

func (s *StatsdClient) Gauge(stat string, value int64)

func (*StatsdClient) GaugeDelta 

func (s *StatsdClient) GaugeDelta(stat string, value int64)

func (*StatsdClient) Inc 

func (s *StatsdClient) Inc(stat string, value int64)

func (*StatsdClient) Timing 

func (s *StatsdClient) Timing(stat string, delta int64)

func (*StatsdClient) TimingDuration 

func (s *StatsdClient) TimingDuration(stat string, delta time.Duration)

func (*StatsdClient) TimingTrack 

func (s *StatsdClient) TimingTrack(stat string, start time.Time)

TimingTrack records the time something took to run using a defer, very good for timing a function call, just add a the defer at the top like so: 

func timeMe() {
    defer app.Stats.TimingTrack("timeMe.run_time", time.Now())
    // rest of code, can return anywhere and run time tracked
}

This works because the time.Now() is run when defer line is reached, so gets the time timeMe statred

type StatusInfo 

type StatusInfo struct {
	AppName       string
	ProjectName   string
	Pid           int
	StartTime     time.Time
	UptimeSeconds float64
	NumGoros      int
	RequestInfo   []RequestInfo
}

Status decodes the JSON status url return

func (StatusInfo) String 

func (status StatusInfo) String() string

type TimberLogFormatterFactory 

type TimberLogFormatterFactory struct {
}

func (*TimberLogFormatterFactory) Create 

func (t *TimberLogFormatterFactory) Create() timber.LogFormatter

type WebSocketCloseMessage 

type WebSocketCloseMessage struct {
	Code int
	Body string
}

func (WebSocketCloseMessage) Error 

func (h WebSocketCloseMessage) Error() string

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.