dlshared

package module
v0.0.0-...-2d94450 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 30, 2014 License: Apache-2.0 Imports: 42 Imported by: 0

README

dlshared

Libraries for Go development. I would wait a bit before using this because the libraries are changing rapidly.

Installation: go get github.com/deftlabs/dlshared

Usage: import "github.com/deftlabs/dlshared"

GoDoc

Documentation

Overview

Package dlshared provides a set of reusable tools/components for Go application development.

Details about the dlshared library are found its project page: 

https://github.com/deftlabs/dlshared

External libraries: 

       github.com/mreiferson/go-httpclient
       labix.org/v2/mgo
       github.com/daviddengcn/go-ljson-conf
       github.com/gorilla/mux
       code.google.com/p/go.crypto/bcrypt
       github.com/nranchev/go-libGeoIP
       github.com/robfig/cron
		  github.com/hashicorp/golang-lru

To initialize the external libraries, run: `make init.libs`

For more information, see the documentation below...

Index

Constants

View Source 
const (
	TrueStrTrue = "true"
	TrueStrT    = "t"
	TrueStrYes  = "yes"
	TrueStrY    = "y"
	TrueStr1    = "1"

	FalseStrFalse = "false"
	FalseStrF     = "f"
	FalseStrN     = "n"
	FalseStrNo    = "no"
	FalseStr0     = "0"
)
View Source 
const (
	DistributedLockLocked   = 2
	DistributedLockUnlocked = 1

	// These are error log timeout codes:
	DistributedLockErrTimeoutWillOccur = "DISTRIBUTED_LOCK_TIMEOUT_WILL_OCCUR"
	DistributedLockErrNoLockDoc        = "DISTRIBUTED_LOCK_NO_LOCK_DOC"
)
View Source 
const (
	DefaultSocketTimeout = 40000
	HttpPostMethod       = "POST"
	HttpGetMethod        = "GET"

	ContentTypeHeader = "Content-Type"

	NoStatusCode = -1 // This is the value used if an error is generated before the status code is available

	ContentTypeTextPlain = "text/plain; charset=utf-8"
	ContentTypePostForm  = "application/x-www-form-urlencoded"
	ContentTypeJson      = "application/json"
)
View Source 
const (
	HttpIntParam       = HttpParamDataType(0)
	HttpStringParam    = HttpParamDataType(1)
	HttpFloatParam     = HttpParamDataType(2)
	HttpBoolParam      = HttpParamDataType(3) // Boolean types include: 1, t, T, TRUE, true, True, 0, f, F, FALSE, false
	HttpObjectIdParam  = HttpParamDataType(4)
	HttpJsonParam      = HttpParamDataType(5)
	HttpJsonArrayParam = HttpParamDataType(6)

	// All of the param types only support single values (i.e., no slices). If multiple values are present, the
	// first is taken.
	HttpParamPost     = HttpParamType(0)
	HttpParamJsonPost = HttpParamType(1) // When the content body is posted in json format. This only supports one level
	HttpParamQuery    = HttpParamType(2)
	HttpParamHeader   = HttpParamType(3)
	HttpParamPath     = HttpParamType(4) // This must be declared as {someName} in the path mapping
)
View Source 
const (
	Counter metricType = 0
	Gauge   metricType = 1

	CounterStr = "counter"
	GaugeStr   = "gauge"
)
View Source 
const (
	MongosConnectionType     = MongoConnectionType("mongos")
	StandaloneConnectionType = MongoConnectionType("standalone")
	ReplicaSetConnectionType = MongoConnectionType("replicaSet")
)
View Source 
const (
	AwsSesEndpoint = "https://email.us-east-1.amazonaws.com"
)

The AWS SES email ds.

View Source 
const (
	HostnameAndPortPattern = "%s:%d"
)
View Source 
const (
	LibratoMetricsPostUrl = "https://%s:%s@metrics-api.librato.com/v1/metrics"
)
View Source 
const NullCharacter = '\x00'

Variables

This section is empty.

Functions

func AppendBson 

func AppendBson(data []byte, v interface{}) ([]byte, error)

func AppendByte 

func AppendByte(slice []byte, data ...byte) []byte

Append N bytes to the slice. This function came from to golang blog post on slices and internals.

func AppendCStr 

func AppendCStr(data []byte, v string) []byte

func AppendInt32 

func AppendInt32(data []byte, v int32) []byte

func AppendInt64 

func AppendInt64(data []byte, v int64) []byte

func AssembleHostnameAndPort 

func AssembleHostnameAndPort(hostname string, port int16) string

func ByteSliceContainsByte 

func ByteSliceContainsByte(slice []byte, c byte) bool

func CallBoolChanParamNoReturnValueMethod 

func CallBoolChanParamNoReturnValueMethod(val interface{}, methodValue reflect.Value, channel chan bool)

func CallNoParamNoReturnValueMethod 

func CallNoParamNoReturnValueMethod(val interface{}, methodValue reflect.Value)

func CapLogCache 

func CapLogCache(size int)

func CompressBytes 

func CompressBytes(val []byte) ([]byte, error)

Compress the byte slice using zlib.

func CreateDir 

func CreateDir(path string) error

CreateDir will create a new directory with all parent directories created. If the directory already exists, nothing is done. This sets the file permissions to 0750. This will use the default group for the user. To override, change at the OS level.

func CurrentTimeInMillis 

func CurrentTimeInMillis() int64

Get the current time in millis since epoch. Source from stackoverflow: http://stackoverflow.com/questions/6161839/go-time-milliseconds

func CurrentTimeInSeconds 

func CurrentTimeInSeconds() int32

Returns the current time in seconds since epoch (i.e., a unix timestamp). Source from stackoverflow: http://stackoverflow.com/questions/9539108/obtaining-a-unix-timestamp-in-go-language-current-time-in-seconds-since-epoch

func DecodeBase64 

func DecodeBase64(v string) ([]byte, error)

Decodes the base64 string. This method panics if the value passed is nil or length is zero.

func DecodeBase64ToStr 

func DecodeBase64ToStr(v string) (string, error)

Decodes the base64 string to a string. This method panics if the value passed is nil or length is zero.

func DeleteDir 

func DeleteDir(path string) error

Delete a directory. This function checks to see if the directory exists and if it does, nothing is done. If the directory does exists, it is deleted.

func DurationToMillis 

func DurationToMillis(dur *time.Duration) int64

Convert a duration to milliseconds.

func EncodeStrToBase64 

func EncodeStrToBase64(v string) string

Encodes the string to base64. This method panics if the value passed is nil or length is zero.

func EncodeToBase64 

func EncodeToBase64(v []byte) string

Encodes the byte array to base64. This method panics if the value passed is nil or length is zero.

func ExtractBsonDoc 

func ExtractBsonDoc(data []byte, offset int) (*bson.M, int, error)

func ExtractBytes 

func ExtractBytes(data []byte, offset int, length int) ([]byte, error)

func ExtractCStr 

func ExtractCStr(data []byte, offset int) (string, error)

Extracts a cstring from the data.

func ExtractInt32 

func ExtractInt32(data []byte, offset int) int32

func ExtractInt64 

func ExtractInt64(data []byte, offset int) int64

func ExtractUInt16 

func ExtractUInt16(data []byte, offset int) uint16

func ExtractUInt32 

func ExtractUInt32(data []byte, offset int) uint32

func FileOrDirExists 

func FileOrDirExists(fileOrDir string) (bool, error)

Check to see if a file or directory exists and the user has access. The original source came from: http://bit.ly/18GDn5Q

func FormatLog 

func FormatLog(log *Log) string

func GeneratePassword 

func GeneratePassword(minPasswordLength,
	maxPasswordLength,
	minLowerCaseCount,
	minUpperCaseCount,
	minNumberCount,
	minSpecialCount int32) string

Generate a random password based on the inputs. Make sure you seed random on server/process startup E.g., rand.Seed( time.Now().UTC().UnixNano()) This logic is based on the example on StackOverflow seen here: http://stackoverflow.com/questions/4090021/need-a-secure-password-generator-recommendation

func GetFunctionName 

func GetFunctionName(i interface{}) string

Return the function name.

Initial code came from: http://stackoverflow.com/questions/7052693/how-to-get-the-name-of-a-function-in-go

This does not include parameter names/types or package, simply the name. If the function is attached to a struct, it returns the struct name.

This method does not work if you have [ '(' || ')' || '*' ] in your file path. If you use this method add a unit test to confirm it works for you needs. If it does not work for your application, please reach out and let us know: https://github.com/deftlabs/dlshared/issues

func GetMethodValueByName 

func GetMethodValueByName(val interface{}, methodName string, allowedArgs, allowedReturn int) (error, reflect.Value)

func HashPassword 

func HashPassword(password string, cost int) ([]byte, error)

Encrypt a password. The cost option is 4 - 31. If the cost is above 31, then an error is displayed. If the cost is below four, then four is used. If a nil or empty password is passed, this method returns an error.

func HttpGetBson 

func HttpGetBson(url string, headers map[string]string) (int, bson.M, error)

func HttpPost 

func HttpPost(url string, values url.Values, headers map[string]string) (int, []byte, error)

func HttpPostBson 

func HttpPostBson(url string, bsonDoc interface{}, headers map[string]string) (int, []byte, error)

func HttpPostJson 

func HttpPostJson(url string, value interface{}, headers map[string]string) (int, []byte, error)

func HttpPostStr 

func HttpPostStr(url string, value string, headers map[string]string) (int, []byte, error)

func IsHttpMethodPost 

func IsHttpMethodPost(request *http.Request) bool

This method returns true if the http request method is a HTTP post. If the field missing or incorrect, false is returned. This method will panic if the request is nil.

func JsonEncodeAndWriteResponse 

func JsonEncodeAndWriteResponse(response http.ResponseWriter, value interface{}) error

Encode and write a json response. If there is a problem encoding an http 500 is sent and an error is returned. If there are problems writting the response an error is returned.

func Md5Hex 

func Md5Hex(v string) (string, error)

func Md5HexFromBytes 

func Md5HexFromBytes(v []byte) (string, error)

func MinuteHourOfTheDay 

func MinuteHourOfTheDay(minuteOfTheDay int16) int8

Returns the hour of the day for the minute of the day passed. The minute of the day is a value within 0-1439. The hour of the day is a value between 0-23. This expects your data to be in the proper range, it will return a bad result if bad data is passed.

func MinuteOfTheDay 

func MinuteOfTheDay(checkTime *time.Time) int16

Returns the minute of the day (0 - 1439) for the time passed. This does not look at seconds or nanoseconds.

func NowInUtc 

func NowInUtc() *time.Time

func NowInUtcMinusSeconds 

func NowInUtcMinusSeconds(seconds int) *time.Time

func NowTimeUnixStr 

func NowTimeUnixStr() string

NowTimeUnixStr returns the date in unix date string format e.g., Wed Dec 11 19:03:18 EST 2013

func ObjectIdHex 

func ObjectIdHex(objectIdHex string) *bson.ObjectId

func PasswordMatchesHash 

func PasswordMatchesHash(hashedPassword []byte, password string) (bool, error)

Check to see if the password is the same as the hashed value. If the values match, true is returned. If a nil or empty password or hash is passed, this method returns an error.

func RunKernelAndListenForInterrupt 

func RunKernelAndListenForInterrupt(id string, addComponentsFunction func(kernel *Kernel)) error

This method will load the configuration file, start the kernel and then listen for the interrupt.

func StrIsFalse 

func StrIsFalse(val string) bool

Returns false if the string is (case insensitive): (false | f | no | n | 0)

func StrIsTrue 

func StrIsTrue(val string) bool

Returns true if the string is (case insensitive): (true | t | yes | y | 1)

func StrSliceValuesInSlice 

func StrSliceValuesInSlice(values []string, toCheck []string) bool

Returns true if all the values are found in the toCheck slice. If the values slice is empty, false is returned. This is intended to be used on very small slices.

func TimeFromMillis 

func TimeFromMillis(timeInMillis int64) *time.Time

Returns the time from the milliseconds since epoch. This returns the time in UTC.

func TimeStartOfCurrentDay 

func TimeStartOfCurrentDay() *time.Time

Returns the time configured to the start of the current day (00:00:00 etc). The current day is defined using UTC.

func TimeStartOfNextDay 

func TimeStartOfNextDay() *time.Time

Returns the time configured to the start of the next day (00:00:00 etc). The current day is defined using UTC.

func TimeToMillis 

func TimeToMillis(tv *time.Time) int64

Convert a time struct to milliseconds since epoch.

func UncompressBytes 

func UncompressBytes(val []byte) ([]byte, error)

Uncompress the byte slice using zlib

func WriteInt32 

func WriteInt32(data []byte, v int32, offset int)

func WriteInt64 

func WriteInt64(data []byte, v int64, offset int)

func WriteOkResponseString 

func WriteOkResponseString(response http.ResponseWriter, msg string) error

Write an http ok response string. The content type is text/plain.

Types

type ApnAlert 

type ApnAlert struct {
	Body         string   `json:"body,omitempty"`
	ActionLocKey string   `json:"action-loc-key,omitempty"`
	LocKey       string   `json:"loc-key,omitempty"`
	LocArgs      []string `json:"loc-args,omitempty"`
	LaunchImage  string   `json:"launch-image,omitempty"`
}

type ApnAps 

type ApnAps struct {
	Alert            interface{} `json:"alert,omitempty"`
	Badge            int         `json:"badge,omitempty"`
	Sound            string      `json:"sound,omitempty"`
	ContentAvailable int         `json:"content-available,omitempty"`
}

The apn aps struct. These values are mostly ints, but Go json considers a value of zero to to be empty causing 'omitempty' to be true and will not serialize the json with a zero. If you need to set the badge to zero, set it to -1. This is ugly, but it is either this or defining Badge as an interface. The alert field is either a nested json structure or a string. The nested json struct for the alert is defined by using the ApnAlert struct.

func NewApnAps 

func NewApnAps() *ApnAps

type ApnFeedbackMsg 

type ApnFeedbackMsg struct {
	Timestamp   uint32
	DeviceToken string
}

The apn feedback message is a response from Apple's feedback gateway. If the app does not exist on the device or the token has changed, they will send this information back.

type ApnMsg 

type ApnMsg struct {
	Payload map[string]interface{}

	// These fields are used by the feedback service.
	Timestamp   uint32
	DeviceToken string

	// This field is used for error conditions. Anything other than a zero
	// means a send failure - see: http://bit.ly/1kNgAwn for more info.
	ApnErrorCode int
	// contains filtered or unexported fields
}

The apple push notification message. For more information, see: http://bit.ly/1kFnCnj

func NewApnMsg 

func NewApnMsg(deviceToken string, expiration uint32, priority uint8, apnAps *ApnAps) *ApnMsg

Create a new apn message. You must specify the device token, the priority, expiration time and aps. The expiration time is a Unix epoch expressed in seconds. Set this to zero to have the message expire immediately. You must also specify the priority. Ten (10) is the highest priority, but cannot be used with the content-available key (error will result). You can add additional values to the payload by saying: msg.Payload["key"] = something. Do *not* use the key of "aps" because it is reserved by the apple for the apn aps struct.

type ApnResponseMsg 

type ApnResponseMsg struct {
	DeviceToken string
	Feedback    *ApnFeedbackMsg
}

type Appender 

type Appender interface {
	Append(log *Log) error
}

type ApplePushNotificationSvc 

type ApplePushNotificationSvc struct {
	Logger
	// contains filtered or unexported fields
}

The apple push notification (apn) service. This component dispatches JSON messages to Apple via a custom Apple binary protocol. The component is configured by passing in the configuration path. 

"apn": {
	"gateway": "gateway.sandbox.push.apple.com:2195",
	"feedback": "feedback.sandbox.push.apple.com:2196",
	"certificateFile": "WHATEVER_YOUR_PEM_FILE",
	"keyFile": "WHATEVER_YOUR_KEY_PEM_FILE",
	"socketTimeoutInMs": "4000",
	"msgCacheElementCount": "2000"
}

In the example above, the configuration path would be "apn" (passed in New function). This assumes that "apn" is located as child of the root of the document.

This uses a little code from Alan Haris open source (Copyright 2013) apns project.

https://github.com/anachronistic/apns

Alan's libraries were released under the MIT license.

func NewApplePushNotificationSvc 

func NewApplePushNotificationSvc(configPath string, requestChannel, responseChannel chan *ApnMsg) *ApplePushNotificationSvc

Create the apple push notification service. Do not close the response channel. This will be closed by this component because this is the component that writes to it. Make sure you close the request channel before calling Stop() or you will leak goroutines.

func (*ApplePushNotificationSvc) Start 

func (self *ApplePushNotificationSvc) Start(kernel *Kernel) error

func (*ApplePushNotificationSvc) Stop 

func (self *ApplePushNotificationSvc) Stop(kernel *Kernel) error

type AwsEmailDs 

type AwsEmailDs struct {
	Logger
	// contains filtered or unexported fields
}

func NewAwsEmailDs 

func NewAwsEmailDs(awsAccessKeyId, awsSecretKey string, httpClient HttpRequestClient, logger Logger) *AwsEmailDs

Create a new aws email ds.

func (*AwsEmailDs) SendHtmlEmailToOneAddress 

func (self *AwsEmailDs) SendHtmlEmailToOneAddress(from, to, subject, bodyHtml, bodyText string) (interface{}, error)

Send an html email to one address.

func (*AwsEmailDs) SendTextEmailToOneAddress 

func (self *AwsEmailDs) SendTextEmailToOneAddress(from, to, subject, body string) (interface{}, error)

Send a text email to one address.

type AwsEmailResponse 

type AwsEmailResponse struct {
	MessageId      string `xml:"SendEmailResult>MessageId"`
	RequestId      string `xml:"ResponseMetadata>RequestId"`
	HttpStatusCode int
}

type AwsEmailSvc 

type AwsEmailSvc struct {
	Logger
	MongoDataSource
	// contains filtered or unexported fields
}

func (*AwsEmailSvc) SendHtmlEmailToOneAddress 

func (self *AwsEmailSvc) SendHtmlEmailToOneAddress(from, to, subject, htmlTemplateFileName, textTemplateFileName string, params map[string]interface{}) error

Send an html email.

func (*AwsEmailSvc) Start 

func (self *AwsEmailSvc) Start(kernel *Kernel) error

func (*AwsEmailSvc) Stop 

func (self *AwsEmailSvc) Stop(kernel *Kernel) error

type CoRContext 

type CoRContext struct {
	Params map[string]interface{}
	Kernel *Kernel
	Logger
}

type CoRFunction 

type CoRFunction func(ctx *CoRContext) error

type CoRSvc 

type CoRSvc struct {
	Logger
	Kernel *Kernel
	// contains filtered or unexported fields
}

A chain-of-responsibility service implementation in Go. To use, define your chain, add your function calls in order and then execute. You can pass anything you want into context object. Init this struct using the kernel to set the global logger and kernel struct. The kernel will be nil if you do not init as component. The logger will be a new (i.e., not configured) struct if you do not use the kernel to init the component.

func NewCoRSvc 

func NewCoRSvc() *CoRSvc

func (*CoRSvc) AddNextFunction 

func (self *CoRSvc) AddNextFunction(chainId string, next CoRFunction)

func (*CoRSvc) RunChain 

func (self *CoRSvc) RunChain(chainId string) error

func (*CoRSvc) RunChainWithContext 

func (self *CoRSvc) RunChainWithContext(chainId string, ctx *CoRContext) error

func (*CoRSvc) RunChainWithParams 

func (self *CoRSvc) RunChainWithParams(chainId string, params map[string]interface{}) error

func (*CoRSvc) Start 

func (self *CoRSvc) Start() error

func (*CoRSvc) Stop 

func (self *CoRSvc) Stop() error

type Component 

type Component struct {
	// contains filtered or unexported fields
}

type Configuration 

type Configuration struct {
	Version     string
	PidFile     string
	Pid         int
	Environment string
	Hostname    string
	FileName    string
	// contains filtered or unexported fields
}

func NewConfiguration 

func NewConfiguration(fileName string) (*Configuration, error)

func (*Configuration) Bool 

func (self *Configuration) Bool(key string, def bool) bool

func (*Configuration) BoolWithPath 

func (self *Configuration) BoolWithPath(path, key string, def bool) bool

func (*Configuration) EnvironmentIs 

func (self *Configuration) EnvironmentIs(expected string) bool

func (*Configuration) Float 

func (self *Configuration) Float(key string, def float64) float64

func (*Configuration) FloatWithPath 

func (self *Configuration) FloatWithPath(path, key string, def float64) float64

func (*Configuration) Int 

func (self *Configuration) Int(key string, def int) int

func (*Configuration) IntList 

func (self *Configuration) IntList(key string, def []int) []int

func (*Configuration) IntWithPath 

func (self *Configuration) IntWithPath(path, key string, def int) int

func (*Configuration) Interface 

func (self *Configuration) Interface(key string, def interface{}) interface{}

func (*Configuration) InterfaceWithPath 

func (self *Configuration) InterfaceWithPath(path, key string, def interface{}) interface{}

func (*Configuration) List 

func (self *Configuration) List(key string, def []interface{}) []interface{}

func (*Configuration) ListWithPath 

func (self *Configuration) ListWithPath(path, key string, def []interface{}) []interface{}

func (*Configuration) StrList 

func (self *Configuration) StrList(key string, def []string) []string

func (*Configuration) String 

func (self *Configuration) String(key string, def string) string

func (*Configuration) StringWithPath 

func (self *Configuration) StringWithPath(path, key string, def string) string

type Consumer 

type Consumer struct {
	Logger
	// contains filtered or unexported fields
}

The consumer is a generic component that is meant to be embedded into your applications. It allows you to create a specific number of goroutines to handle the realtime data/msg processing. The contact is that you must call the NewConsumer method to create the struct. After that, you must call the Start method before attempting to pass in a msg. To stop the consumer, close the receive channel and THEN call the Stop method.

func NewConsumer 

func NewConsumer(name string,
	receiveChannel chan interface{},
	consumeFunc func(msg interface{}),
	maxGoroutines,
	maxWaitOnStopInMs int,
	logger Logger) *Consumer

Create the consumer. You must call this method to create a consumer. If you set the maxGoroutines to zero (or less) then it will spawn a new goroutine for each request and there is no limit. If maxGoroutines is set, these goroutines are created and added to a pool when Start is called. The maxWaitOnStopInMs is the amount of time the Stop method will wait for the goroutines to finish clearing out what they are processing. A maxWaitOnStopInMs value of zero indicates an unlimited wait. You MUST close the receive channel before calling stop.

func (*Consumer) Start 

func (self *Consumer) Start() error

func (*Consumer) Stop 

func (self *Consumer) Stop() error

This method will block until all goroutines exit. It is up to the caller to clear the receiveChannel.

type ContentSvc 

type ContentSvc struct {
	// contains filtered or unexported fields
}

The content service is a very simple content lookup based on passed locale. ContentSvc is stored in a json file. The path/structure of your content is basically up to you. However, to support locales, the following structure is required: 

{
	"something": {
		"else": {
			"en_US": "Hello",
			"de_DE": "Hallo",
			"pt_BR": "Olá"
		}
	}
}

With this structure, the path you would pass is: "something.else". The locale param would be "end_US" or another locale. You can store the locale keys in any format you like (e.g., you could use "en-US" or just "English", or whatever you like).

func NewContentSvc 

func NewContentSvc(fileName string) (*ContentSvc, error)

func (*ContentSvc) Lookup 

func (self *ContentSvc) Lookup(path, def string) string

Lookup content. If the content is not found, the default is returned. This method requires the full path. For the example above, you would have to pass the path of: something.else.en_US to pull some content.

func (*ContentSvc) LookupWithLocale 

func (self *ContentSvc) LookupWithLocale(path, locale, defaultLocale, def string) string

Lookup content. If the content is not found, the default locale string is returned. If the default locale is missing too, the default string is returned. See above for information about content structure.

type CronJob 

type CronJob interface{ Run() }

type CronSchedule 

type CronSchedule interface{ Next(time.Time) time.Time }

type CronSvc 

type CronSvc struct {
	Logger
	// contains filtered or unexported fields
}

The cron service is a wrapper around robfig's cron library that adds audit/tracking data that is stored in MongoDB. For more information on the core cron library, see: 

http://godoc.org/github.com/robfig/cron

The cron service can also be used in conjunction with the DistributedLock to ensure that crons only execute in one process in the cluster.

Similar to the wwy the mongo component works, the cron service is configured via the configuration json file. 

 "cron": {
     "scheduled": {
         "mongoComponentId": "MongoDbData",

		  "definitionDbName": "cron",
         "definitionCollectionName": "cron.definitions",

		  "auditDbName": "cron",
         "auditCollectionName": "cron.audit",
         "auditTimeoutInSec": 126144000,

         "distributedLockComponentId": "MyDistributedLock",

         "scheduledFunctions": [
             { "jobId": "testCronJob-Run",
               "componentId": "testComponentId",
               "methodName": "Run",
               "schedule": "0 30 * * * *",
               "requiresDistributedLock": true,
               "audit": true,
               "enabled": true,
               "maxRunTimeInSec": 30},

             { "jobId": "testCronJob-Test",
               "componentId": "testComponentId",
               "methodName": "Test",
               "schedule": "0 30 * * * *",
               "requiresDistributedLock": true,
               "audit": true,
               "enabled": false,
               "maxRunTimeInSec": 30 }
         ]
     }
 }

The configPath for this component would be "cron.scheduled". The path can be any arbitrary set of nested json documents (json path). If the path is incorrect, the Start() method will panic when called by the kernel. The configuration file currently only supports scheduling methods by component id. You need to register your component in the kernel and define the method name as a member of that struct. The method must take a single bool channel param and cannot return any values. The boolean channel is used to signal the job to stop. A stop signal can occur if the maxRunTimeInSec is exceeded or if the process is stopped. The method must also be declared public (i.e., the first character must be uppercase). The method name should not have a bracket/parentheses. When defining scheduled methods, the job ids must be unique or the service will error on Start. You can disable the db audit for jobs by setting "audit" to false in the scheduled method. Disabling the audit is usually not recommended.

If you set the auditTimeoutInSec to zero, then it will never timeout the audit/history. When the auditTimeoutInSec is greater than zero, it will remove the audit/history from the database after the configured number of seconds. Ten years in seconds is ~: 315360000 (leap year etc. not included simply 86,400 * 365 * 10).

If you wish to use other definition options, add the items to the component in your Start method. The cron service must be added to the kernel after all required components have been added.

For supported cron expression format options, see: http://godoc.org/github.com/robfig/cron

If you change "enabled" for a scheduled function in the database directly, the app will update after a bit. The component polls the db for changes.

See cron_test.go for usage example.

func NewCronSvc 

func NewCronSvc(configPath string) *CronSvc

func (*CronSvc) Start 

func (self *CronSvc) Start(kernel *Kernel) error

func (*CronSvc) Stop 

func (self *CronSvc) Stop(kernel *Kernel) error

type DistributedLock 

type DistributedLock interface {
	Start(kernel *Kernel) error
	Stop(kernel *Kernel) error
	Lock()
	TryLock() bool
	Unlock()
	HasLock() bool
	LockId() string
}

func NewMongoDistributedLock 

func NewMongoDistributedLock(lockId,
	mongoComponentId,
	dbName,
	collectionName string,
	heartbeatFreqInSec,
	lockCheckFreqInSec,
	lockTimeoutInSec int,
	historyTimeoutInSec int) DistributedLock

Create the distributed lock. If you want to enable history tracking, set the historyTimeoutInSec param something greater than zero. Locks and unlocks will be stored for this amount of time in a separate collection by appending the suffix "History" to your collection name.

type EmailDoc 

type EmailDoc struct {
	Id bson.ObjectId `bson:"_id"`

	HtmlTemplateName string `bson:"htmlTemplateName,omitempty"`
	TextTemplateName string `bson:"textTemplateName,omitempty"`

	SendMethod string `bson:"sendMethod"`

	EmailType string `bson:"type"`

	MessageId      string `bson:"messageId,omitempty"`
	RequestId      string `bson:"requestId,omitempty"`
	HttpStatusCode int    `bson:"httpStatusCode,omitempty"`

	ToAddrs  []string   `bson:"toAddrs"`
	FromAddr string     `bson:"fromAddr"`
	Sent     *time.Time `bson:"sent"`

	Subject  []byte `bson:"subject"`
	BodyHtml []byte `bson:"bodyHtml"`
	BodyText []byte `bson:"bodyText"`

	Error string `bson:"error,omitempty"`
}

type EmailDs 

type EmailDs interface {
	SendTextEmailToOneAddress(from, to, subject, body string) (interface{}, error)
	SendHtmlEmailToOneAddress(from, to, subject, bodyHtml, bodyText string) (interface{}, error)
}

The email ds interface

type EmailSvc 

type EmailSvc interface {
	SendHtmlEmailToOneAddress(from, to, subject, htmlTemplateFileName, textTemplateFileName string, params map[string]interface{}) error
	Start(kernel *Kernel) error
	Stop(kernel *Kernel) error
}

func NewAwsEmailSvc 

func NewAwsEmailSvc(dbComponentName,
	templateComponentName,
	dbName, collectionName,
	awsAccessKeyId,
	awsSecretKey string,
	cappedCollectionSizeInBytes int) EmailSvc

Create a new email service. Set cappedCollectionSizeInBytes to less than one to create a permanent collection (i.e., otherwise it creates a capped collection). Currently, the email service only supports sending via AWS SES.

type FileAppender 

type FileAppender struct {
	*os.File
}

func DevNullAppender 

func DevNullAppender() (*FileAppender, error)

func StdErrAppender 

func StdErrAppender() *FileAppender

func StdOutAppender 

func StdOutAppender() *FileAppender

func (FileAppender) Append 

func (self FileAppender) Append(log *Log) error

type Filter 

type Filter func(log *Log) bool

Return true if the log should be passed to the underlying `Appender`

type FilterAppender 

type FilterAppender struct {
	Appender Appender
	Filter   Filter
}

func LevelFilter 

func LevelFilter(threshold Level, appender Appender) *FilterAppender

func (*FilterAppender) Append 

func (self *FilterAppender) Append(log *Log) error

type GeoLocation 

type GeoLocation struct {
	CountryCode string  `json:"countryCode"`
	CountryName string  `json:"countryName"`
	Region      string  `json:"region"`
	City        string  `json:"city"`
	PostalCode  string  `json:"postalCode"`
	Latitude    float32 `json:"latitude"`
	Longitude   float32 `json:"longitude"`
}

A wrapper around the libgeo struct. This adds json tags and provides abstraction against geo impls.

type GeoLocationSvc 

type GeoLocationSvc interface {
	LocationByIp(ipAddress string) *GeoLocation
	Start() error
	Stop() error
}

The geo location service interface.

func NewMaxMindLocationSvc 

func NewMaxMindLocationSvc(dbFile string) GeoLocationSvc

type GoogleCloudMessagingMsgResponseHandler 

type GoogleCloudMessagingMsgResponseHandler func(*GoogleCloudMsgResponse)

type GoogleCloudMessagingSvc 

type GoogleCloudMessagingSvc struct {
	Logger

	sync.WaitGroup
	// contains filtered or unexported fields
}

The google cloud messaging service. This component dispatches JSON messages to Google via HTTP and implements exponential backoff to ensure that if there is a failure, everything slows down as to not overload the consuming service. This component wraps a consumer component. The component is configured by passing in the configuration path. 

		"gcm": {
			"postUrl": "https://android.googleapis.com/gcm/send",
			"authKey": "WHATEVER_IT_IS_FOR_YOUR_API_ACCESS",
			"acceptableGoogleCloudMsgFailurePercent": 10,
			"initialGoogleCloudMsgBackoffInMs": 100,
			"maxGoogleCloudMsgBackoffInMs": 10000,
			"consumer": {
				"maxGoroutines": 1000,
				"maxWaitOnStopInMs": 30000
         }
		}

In the example above, the configuration path would be "gcm" (passed in New function). This assumes that "gcm" is located as child of the root of the document.

func NewGoogleCloudMessagingSvc 

func NewGoogleCloudMessagingSvc(configPath string, httpClient HttpRequestClient, requestChannel, responseChannel chan interface{}) *GoogleCloudMessagingSvc

func (*GoogleCloudMessagingSvc) Start 

func (self *GoogleCloudMessagingSvc) Start(kernel *Kernel) error

func (*GoogleCloudMessagingSvc) Stop 

func (self *GoogleCloudMessagingSvc) Stop(kernel *Kernel) error

type GoogleCloudMsg 

type GoogleCloudMsg struct {
	RegistrationIds       []string               `json:"registration_ids"`
	NotificationKey       string                 `json:"notification_key,omitempty"`
	CollapseKey           string                 `json:"collapse_key,omitempty"`
	DelayWhileIdle        bool                   `json:"delay_while_idle,omitempty"`
	TimeToLive            int                    `json:"time_to_live,omitempty"`
	RestrictedPackageName string                 `json:"restricted_package_name,omitempty"`
	DryRun                bool                   `json:"dry_run,omitempty"`
	Data                  map[string]interface{} `json:"data,omitempty"`

	// This can contain anything - usually, it is source info/data that can be used
	// when the response is processed. These items should be in the same order as
	// the registration id.
	AdditionalInfo []interface{} `json:"-"`
}

The user must set the required fields in the message.

type GoogleCloudMsgResponse 

type GoogleCloudMsgResponse struct {
	MulticastId       float64                         `json:"multicast_id"`
	SuccessCount      float64                         `json:"success"`
	FailureCount      float64                         `json:"failure"`
	CanonicalIdsCount float64                         `json:"canonical_ids"`
	Results           []*GoogleCloudMsgResponseResult `json:"results"`
	OrigMsg           *GoogleCloudMsg                 `json:"-"`
	Err               error                           `json:"-"` // If there is an error caused by io or a problem with gcm.
	HttpStatusCode    int                             `json:"-"`
}

The response object returned by google.

type GoogleCloudMsgResponseResult 

type GoogleCloudMsgResponseResult struct {
	MessageId      string `json:"message_id"`
	RegistrationId string `json:"registration_id"`
	Error          string `json:"error"`
}

type GoogleCloudMsgSendStats 

type GoogleCloudMsgSendStats struct {
	BackoffTimeInMs      int
	CurrentSuccessCount  int
	CurrentFailureCount  int
	PreviousSuccessCount int
	PreviousFailureCount int

	sync.Mutex
	// contains filtered or unexported fields
}

The cloud message stats.

type HttpContext 

type HttpContext struct {
	Response   http.ResponseWriter
	Request    *http.Request
	Params     map[string]*HttpParam
	ErrorCodes []string
	Errors     []error
	// contains filtered or unexported fields
}

func NewHttpContext 

func NewHttpContext(response http.ResponseWriter, request *http.Request) *HttpContext

Call this method to init the http context struct.

func (*HttpContext) DefineBoolParam 

func (self *HttpContext) DefineBoolParam(name, invalidErrorCode string, paramType HttpParamType, required bool)

Boolean types include: 1, t, T, TRUE, true, True, 0, f, F, FALSE, false

func (*HttpContext) DefineFloatParam 

func (self *HttpContext) DefineFloatParam(name, invalidErrorCode string, paramType HttpParamType, required bool)

func (*HttpContext) DefineIntParam 

func (self *HttpContext) DefineIntParam(name, invalidErrorCode string, paramType HttpParamType, required bool)

func (*HttpContext) DefineJsonArrayParam 

func (self *HttpContext) DefineJsonArrayParam(name, invalidErrorCode string, paramType HttpParamType, required bool)

func (*HttpContext) DefineJsonParam 

func (self *HttpContext) DefineJsonParam(name, invalidErrorCode string, paramType HttpParamType, required bool)

func (*HttpContext) DefineObjectIdParam 

func (self *HttpContext) DefineObjectIdParam(name, invalidErrorCode string, paramType HttpParamType, required bool)

func (*HttpContext) DefineStringParam 

func (self *HttpContext) DefineStringParam(name, invalidErrorCode string, paramType HttpParamType, required bool, minLength, maxLength int)

func (*HttpContext) HasParam 

func (self *HttpContext) HasParam(name string) bool

func (*HttpContext) HasRawErrors 

func (self *HttpContext) HasRawErrors() bool

func (*HttpContext) ParamBool 

func (self *HttpContext) ParamBool(name string) bool

func (*HttpContext) ParamFloat 

func (self *HttpContext) ParamFloat(name string) float64

func (*HttpContext) ParamInt 

func (self *HttpContext) ParamInt(name string) int

func (*HttpContext) ParamJson 

func (self *HttpContext) ParamJson(name string) map[string]interface{}

func (*HttpContext) ParamJsonArray 

func (self *HttpContext) ParamJsonArray(name string) []interface{}

func (*HttpContext) ParamObjectId 

func (self *HttpContext) ParamObjectId(name string) *bson.ObjectId

func (*HttpContext) ParamString 

func (self *HttpContext) ParamString(name string) string

func (*HttpContext) ParamsAreValid 

func (self *HttpContext) ParamsAreValid() bool

Validate the params. If any of the params are invalid, false is returned. You must call this first before calling the ErrorCodes []string. If not params are defined, this always returns "true". If there are raw data extraction errors, this is always false (e.g., body missing or incorrect).

type HttpParam 

type HttpParam struct {
	Name             string
	InvalidErrorCode string
	DataType         HttpParamDataType
	Type             HttpParamType
	Required         bool
	MinLength        int
	MaxLength        int
	Post             bool
	Value            interface{}
	Raw              string
	Valid            bool
	Present          bool // If value is present and parsed properly
}

func (*HttpParam) Bool 

func (self *HttpParam) Bool() bool

func (*HttpParam) Float 

func (self *HttpParam) Float() float64

func (*HttpParam) Int 

func (self *HttpParam) Int() int

Make sure your params are present and valid before trying to access.

func (*HttpParam) Json 

func (self *HttpParam) Json() map[string]interface{}

func (*HttpParam) JsonArray 

func (self *HttpParam) JsonArray() []interface{}

func (*HttpParam) ObjectId 

func (self *HttpParam) ObjectId() *bson.ObjectId

func (*HttpParam) String 

func (self *HttpParam) String() string

type HttpParamDataType 

type HttpParamDataType int8

type HttpParamType 

type HttpParamType int8

type HttpRequestClient 

type HttpRequestClient interface {
	Post(url string, values url.Values, headers map[string]string) (int, []byte, error)
	PostStr(url string, value string, headers map[string]string) (int, []byte, error)
	PostJson(url string, value interface{}, headers map[string]string) (int, []byte, error)
	PostBson(url string, bsonDoc interface{}, headers map[string]string) (int, []byte, error)
	GetBson(url string, headers map[string]string) (int, bson.M, error)
	Clone() HttpRequestClient
}

The http request client interface allows you to configure global values for timeout/connect etc. It also handles closing the transport and provides convenience methods for posting some types of data.

func NewDefaultHttpRequestClient 

func NewDefaultHttpRequestClient() HttpRequestClient

func NewHttpRequestClient 

func NewHttpRequestClient(disableKeepAlives,
	disableCompression,
	skipSslVerify bool,
	maxIdleConnsPerHost,
	connectTimeoutInMs,
	responseHeaderTimeoutInMs,
	requestTimeoutInMs,
	readWriteTimeoutInMs int) HttpRequestClient

func NewHttpRequestClientMock 

func NewHttpRequestClientMock() HttpRequestClient

type HttpRequestClientImpl 

type HttpRequestClientImpl struct {
	// contains filtered or unexported fields
}

func (*HttpRequestClientImpl) Clone 

func (self *HttpRequestClientImpl) Clone() HttpRequestClient

Use the clone method if you need to override some/all of the configured values.

func (*HttpRequestClientImpl) GetBson 

func (self *HttpRequestClientImpl) GetBson(url string, headers map[string]string) (int, bson.M, error)

Issue a GET to retrieve a bson doc.

func (*HttpRequestClientImpl) Post 

func (self *HttpRequestClientImpl) Post(url string, values url.Values, headers map[string]string) (int, []byte, error)

Post the values to the url.

func (*HttpRequestClientImpl) PostBson 

func (self *HttpRequestClientImpl) PostBson(url string, bsonDoc interface{}, headers map[string]string) (int, []byte, error)

Post the bson doc to the url.

func (*HttpRequestClientImpl) PostJson 

func (self *HttpRequestClientImpl) PostJson(url string, value interface{}, headers map[string]string) (int, []byte, error)

Post the json struct to the url.

func (*HttpRequestClientImpl) PostStr 

func (self *HttpRequestClientImpl) PostStr(url string, value string, headers map[string]string) (int, []byte, error)

Post the raw string to the url.

type HttpRequestClientMock 

type HttpRequestClientMock struct {
	// contains filtered or unexported fields
}

The simple http request client mock object. This implements the HttpRequestClient interface. Set the struct fields to your desired return values before calling the function and add them by url before the method is called. The method will panic if no response is found for an url (i.e., you don't define the response for an url by calling AddMock). For usage, see TestHttpRequestClientMock in http_client_test.go

func (*HttpRequestClientMock) AddMock 

func (self *HttpRequestClientMock) AddMock(url string, response *HttpRequestClientMockResponse)

func (*HttpRequestClientMock) Clone 

func (self *HttpRequestClientMock) Clone() HttpRequestClient

func (*HttpRequestClientMock) GetBson 

func (self *HttpRequestClientMock) GetBson(url string, headers map[string]string) (int, bson.M, error)

func (*HttpRequestClientMock) Post 

func (self *HttpRequestClientMock) Post(url string, values url.Values, headers map[string]string) (int, []byte, error)

func (*HttpRequestClientMock) PostBson 

func (self *HttpRequestClientMock) PostBson(url string, bsonDoc interface{}, headers map[string]string) (int, []byte, error)

func (*HttpRequestClientMock) PostJson 

func (self *HttpRequestClientMock) PostJson(url string, value interface{}, headers map[string]string) (int, []byte, error)

func (*HttpRequestClientMock) PostStr 

func (self *HttpRequestClientMock) PostStr(url string, value string, headers map[string]string) (int, []byte, error)

type HttpRequestClientMockResponse 

type HttpRequestClientMockResponse struct {
	Data           []byte
	HttpStatusCode int
	Error          error
}

type HttpServer 

type HttpServer struct {
	Logger
	// contains filtered or unexported fields
}

func NewHttpServer 

func NewHttpServer(handlerDefs ...*HttpServerHandlerDef) *HttpServer

func (*HttpServer) Id 

func (self *HttpServer) Id() string

func (*HttpServer) Start 

func (self *HttpServer) Start(kernel *Kernel) error

func (*HttpServer) Stop 

func (self *HttpServer) Stop(kernel *Kernel) error

type HttpServerHandlerDef 

type HttpServerHandlerDef struct {
	Path        string
	HandlerFunc http.HandlerFunc
}

type Kernel 

type Kernel struct {
	Configuration *Configuration
	Components    map[string]Component

	Id string
	Logger
	Pid int
	// contains filtered or unexported fields
}

func StartKernel 

func StartKernel(id string, configFileName string, addComponentsFunction func(kernel *Kernel)) (*Kernel, error)

Call this from your main to create the kernel. After init kernel is called you must add your components and then call kernel.Start()

func (*Kernel) AddComponent 

func (self *Kernel) AddComponent(componentId string, singleton interface{})

Register a component without a start or stop method.

func (*Kernel) AddComponentWithStartMethod 

func (self *Kernel) AddComponentWithStartMethod(componentId string, singleton interface{}, startMethodName string)

Register a component with a start method.

func (*Kernel) AddComponentWithStartStopMethods 

func (self *Kernel) AddComponentWithStartStopMethods(componentId string, singleton interface{}, startMethodName, stopMethodName string)

Register a component with a start and stop methods. This method will panic if a nil component is passed.

func (*Kernel) AddComponentWithStopMethod 

func (self *Kernel) AddComponentWithStopMethod(componentId string, singleton interface{}, stopMethodName string)

Register a component with a stop method.

func (*Kernel) GetComponent 

func (self *Kernel) GetComponent(componentId string) interface{}

Access another component. This method will panic if you attempt to reference a non-existent component. If the component id has a length of zero, it is also panics.

func (*Kernel) HasComponent 

func (self *Kernel) HasComponent(componentId string) (found bool)

Returns true if the component is present. This panics if the component id is empty.

func (*Kernel) ListenForInterrupt 

func (self *Kernel) ListenForInterrupt() error

ListenForInterrupt blocks until an interrupt signal is detected.

func (*Kernel) Start 

func (self *Kernel) Start() error

Call this after the kernel has been created and components registered.

func (*Kernel) Stop 

func (self *Kernel) Stop() error

Stop the kernel. Call this before exiting.

type Level 

type Level uint8
const (
	Off Level = iota
	Debug
	Info
	Warn
	Error
)

The level is in an order such that the expressions `level < Warn`, `level >= Info` have intuitive meaning. Note: The all caps log level constants are now deprecated and will likely be removed from future versions in favor of the Go preferred CamelCase constants. Note: The two types should not be mixed. If you use Debug in your Logger initialization, you should use Debug in your Logf etc. calls.

func (Level) Type 

func (self Level) Type() string

type Librato 

type Librato struct {
	Logger
	// contains filtered or unexported fields
}

func NewLibrato 

func NewLibrato(apiUser, apiToken string, logger Logger) *Librato

func (*Librato) SendMetricsToLibrato 

func (self *Librato) SendMetricsToLibrato(sourceName string, metrics []Metric)

This method can be used as the Metrics relay function.

type Log 

type Log struct {
	Prefix    string
	Level     Level
	Filename  string
	Line      int
	Timestamp time.Time
	// contains filtered or unexported fields
}

func (*Log) Message 

func (self *Log) Message() string

type LogCache 

type LogCache struct {
	// A `LogCache` might be accessed concurrently throughout the
	// program. Therefore, the code calling `Log` acquires a mutex for
	// writing to (and potentially reading from) the
	// ring. Alternatively, if channels are more efficient, the `Log`
	// method can instead pass the *Log through a channel. A goroutine
	// on the other end can be the sole maintainer of the `LogCache`,
	// removing the need for a mutex.
	sync.Mutex
	// contains filtered or unexported fields
}
var Cache LogCache

func (*LogCache) Add 

func (self *LogCache) Add(log *Log)

func (*LogCache) Copy 

func (self *LogCache) Copy() []*Log

func (*LogCache) Len 

func (self *LogCache) Len() int

type LogMetrics 

type LogMetrics struct {
	Logger
	// contains filtered or unexported fields
}

This can be used to print metrics out to a configured logger.

func NewLogMetrics 

func NewLogMetrics(logger Logger) *LogMetrics

Pass the logger struct to the log metrics.

func (*LogMetrics) Disable 

func (self *LogMetrics) Disable()

func (*LogMetrics) Enable 

func (self *LogMetrics) Enable()

func (*LogMetrics) Log 

func (self *LogMetrics) Log(sourceName string, metrics []Metric)

This logs an info message with the following format: [source: %s - type: %s - metric: %s - value: %f]

type Logger 

type Logger struct {
	Prefix    string
	Appenders []Appender
}

func (*Logger) Errorf 

func (self *Logger) Errorf(level Level, messageFmt string, args ...interface{}) error

Log and return a formatted error string. Example: 

if whatIsExpected != whatIsReturned {
    return slogger.Errorf(slogger.WARN, "Unexpected return value. Expected: %v Received: %v",
        whatIsExpected, whatIsReturned)
}

func (*Logger) Logf 

func (self *Logger) Logf(level Level, messageFmt string, args ...interface{}) (*Log, []error)

Log a message and a level to a logger instance. This returns a pointer to a Log and a slice of errors that were gathered from every Appender (nil errors included).

func (*Logger) Stackf 

func (self *Logger) Stackf(level Level, stackErr error, messageFmt string, args ...interface{}) (*Log, []error)

Stackf is designed to work in tandem with `NewStackError`. This function is similar to `Logf`, but takes a `stackErr` parameter. `stackErr` is expected to be of type StackError, but does not have to be.

type MaxMindLocationSvc 

type MaxMindLocationSvc struct {
	// contains filtered or unexported fields
}

This is a thin wrapper around libgeo, which is built on top of MaxMind's geo databases. Currently this only supports IPV4 addresses.

func (*MaxMindLocationSvc) LocationByIp 

func (self *MaxMindLocationSvc) LocationByIp(ipAddress string) *GeoLocation

This method returns nil if not found.

func (*MaxMindLocationSvc) Start 

func (self *MaxMindLocationSvc) Start() error

func (*MaxMindLocationSvc) Stop 

func (self *MaxMindLocationSvc) Stop() error

type Metric 

type Metric struct {
	Name  string
	Type  metricType
	Value float64
}

type Metrics 

type Metrics struct {
	// contains filtered or unexported fields
}

func NewMetrics 

func NewMetrics(sourceName string,
	relayFuncs []MetricsRelayFunction,
	relayPeriodInSecs int,
	metricQueueLength int) *Metrics

The relay function is only called if there are metrics to relay.

func (*Metrics) Count 

func (self *Metrics) Count(metricName string)

Increases the counter by one.

func (*Metrics) CountWithValue 

func (self *Metrics) CountWithValue(metricName string, value float64)

Increase the counter

func (*Metrics) Gauge 

func (self *Metrics) Gauge(metricName string, value float64)

Update the gauge value

func (*Metrics) Start 

func (self *Metrics) Start() error

func (*Metrics) Stop 

func (self *Metrics) Stop() error

type MetricsMongo 

type MetricsMongo struct {
	Logger
	MongoDataSource
	// contains filtered or unexported fields
}

A metrics relay function that stores count and guage values in mongo. This does not store historical values, simply current ones.

func NewMetricsMongo 

func NewMetricsMongo(dbName, collectionName, mongoComponentName string, fireAndForget bool) *MetricsMongo

func (*MetricsMongo) FindById 

func (self *MetricsMongo) FindById(id string) (*bson.M, error)

func (*MetricsMongo) FindDistinctMetricNames 

func (self *MetricsMongo) FindDistinctMetricNames() ([]string, error)

Returns all of the distinct metric names

func (*MetricsMongo) FindMetricsByName 

func (self *MetricsMongo) FindMetricsByName(metricName string, batchSize int) *MongoCursor

Returns the cursor for metrics by metric name. The caller must close the cursor when done.

func (*MetricsMongo) Start 

func (self *MetricsMongo) Start(kernel *Kernel) error

func (*MetricsMongo) Stop 

func (self *MetricsMongo) Stop(kernel *Kernel) error

func (*MetricsMongo) StoreMetricsInMongo 

func (self *MetricsMongo) StoreMetricsInMongo(sourceName string, metrics []Metric)

This method can be used as the Metrics relay function.

type MetricsRelayFunction 

type MetricsRelayFunction func(string, []Metric)

type Mongo 

type Mongo struct {
	Logger

	DefaultSafe *mgo.Safe
	// contains filtered or unexported fields
}

func NewMongo 

func NewMongo(componentId,
	mongoUrl string,
	connectionType MongoConnectionType,
	mode string,
	dialTimeoutInMs,
	socketTimeoutInMs,
	syncTimeoutInMs,
	cursorTimeoutInMs int) *Mongo

Create a new Mongo component. This method will panic if either of the params are nil or len == 0.

func NewMongoFromConfigPath 

func NewMongoFromConfigPath(componentId, configPath string) *Mongo

Create a new Mongo component from a configuration path. The path passed must be in the following format. 

"mongodb": {
    "configDb": {
        "mongoUrl": "mongodb://localhost:27017/test",
        "mode": "strong",
        "dialTimeoutInMs": 3000,
        "socketTimeoutInMs": 3000,
        "syncTimeoutInMs": 3000,
        "cursorTimeoutInMs": 30000,
        "type": "standalone",
    }
}

The configPath for this component would be "mongodb.configDb". The path can be any arbitrary set of nested json documents (json path). If the path is incorrect, the Start() method will panic when called by the kernel.

All of the params above must be present or the Start method will panic. If the componentId or configPath param is nil or empty, this method will panic.

func (*Mongo) Collection 

func (self *Mongo) Collection(dbName, collectionName string) *mgo.Collection

Returns the collection from the session.

func (*Mongo) Db 

func (self *Mongo) Db(name string) *mgo.Database

Returns the database from the session.

func (*Mongo) Session 

func (self *Mongo) Session() *mgo.Session

Returns the session struct.

func (*Mongo) SessionClone 

func (self *Mongo) SessionClone() *mgo.Session

Returns a clone of the session struct.

func (*Mongo) SessionCopy 

func (self *Mongo) SessionCopy() *mgo.Session

Returns a copy of the session struct.

func (*Mongo) Start 

func (self *Mongo) Start(kernel *Kernel) error

func (*Mongo) Stop 

func (self *Mongo) Stop(kernel *Kernel) error

Stop the component. This will close the base session.

type MongoConnectionType 

type MongoConnectionType string

type MongoCursor 

type MongoCursor struct {
	Iter    *mgo.Iter
	Session *mgo.Session
}

The mongo cursor is a wrapper around an mgo.Iter and session object. This alows you to close both the iter and the session in the same call. The cursor supports all of the methods in the iter struct.

func (*MongoCursor) All 

func (self *MongoCursor) All(result interface{}) error

func (*MongoCursor) Close 

func (self *MongoCursor) Close() error

Close closes the session and the iter. If an error is returned, it is from the iter.

func (*MongoCursor) Err 

func (self *MongoCursor) Err() error

func (*MongoCursor) For 

func (self *MongoCursor) For(result interface{}, f func() error) error

func (*MongoCursor) Next 

func (self *MongoCursor) Next(result interface{}) bool

func (*MongoCursor) Timeout 

func (self *MongoCursor) Timeout() bool

type MongoDataSource 

type MongoDataSource struct {
	DbName         string
	CollectionName string
	Mongo          *Mongo
	Logger
}

func (*MongoDataSource) Collection 

func (self *MongoDataSource) Collection() *mgo.Collection

Returns the collection from the session.

func (*MongoDataSource) CollectionFromSession 

func (self *MongoDataSource) CollectionFromSession(session *mgo.Session) *mgo.Collection

Returns the collection from the session.

func (*MongoDataSource) Count 

func (self *MongoDataSource) Count(selector interface{}) (int, error)

Get the count by selector.

func (*MongoDataSource) CreateCappedCollection 

func (self *MongoDataSource) CreateCappedCollection(sizeInBytes int) error

Create a capped collection.

func (*MongoDataSource) Db 

func (self *MongoDataSource) Db() *mgo.Database

Returns the database from the session.

func (*MongoDataSource) Delete 

func (self *MongoDataSource) Delete(selector interface{}) error

Delete one or more documents from the collection. If the document(s) is/are not found, no error is returned.

func (*MongoDataSource) DeleteById 

func (self *MongoDataSource) DeleteById(id interface{}) error

Delete one document by the _id.

func (*MongoDataSource) DeleteOne 

func (self *MongoDataSource) DeleteOne(selector interface{}) error

Delete one document from the collection. If the document is not found, no error is returned.

func (*MongoDataSource) EnsureIndex 

func (self *MongoDataSource) EnsureIndex(fields []string) error

Ensure a non-unique, non-sparse index is created. This does not create in the background.

func (*MongoDataSource) EnsureSparseIndex 

func (self *MongoDataSource) EnsureSparseIndex(fields []string) error

Ensure a non-unique, sparse index is created. This does not create in the background.

func (*MongoDataSource) EnsureTtlIndex 

func (self *MongoDataSource) EnsureTtlIndex(field string, expireAfterSeconds int) error

Create a ttl index.

func (*MongoDataSource) EnsureUniqueIndex 

func (self *MongoDataSource) EnsureUniqueIndex(fields []string) error

Ensure a unique, non-sparse index is created. This does not create in the background. This does NOT drop duplicates if they exist. Duplicates will cause an error.

func (*MongoDataSource) EnsureUniqueSparseIndex 

func (self *MongoDataSource) EnsureUniqueSparseIndex(fields []string) error

Ensure a unique, sparse index is created. This does not create in the background. This does NOT drop duplicates if they exist. Duplicates will cause an error.

func (*MongoDataSource) FindAllWithBatchSize 

func (self *MongoDataSource) FindAllWithBatchSize(batchSize int) *MongoCursor

The caller must close the cursor when done. Use: defer cursor.Close()

func (*MongoDataSource) FindById 

func (self *MongoDataSource) FindById(id interface{}, result interface{}) error

Find by the _id. Returns false if not found.

func (*MongoDataSource) FindDistinctStrs 

func (self *MongoDataSource) FindDistinctStrs(selector interface{}, fieldName string) ([]string, error)

Find the distinct string fields. Do not use this on datasets with a large amount of distinct values or you will blow out memory. The selector can be nil.

func (*MongoDataSource) FindFirst 

func (self *MongoDataSource) FindFirst(selector interface{}, sortFields ...string) *MongoCursor

You must close this cursor.

func (*MongoDataSource) FindManyWithBatchSize 

func (self *MongoDataSource) FindManyWithBatchSize(selector interface{}, batchSize int) *MongoCursor

The caller must close the cursor when done. Use: defer cursor.Close()

func (*MongoDataSource) FindManyWithBatchSizeAndSort 

func (self *MongoDataSource) FindManyWithBatchSizeAndSort(selector interface{}, batchSize int, sortFields ...string) *MongoCursor

The caller must close the cursor when done. Use: defer cursor.Close()

func (*MongoDataSource) FindManyWithOffsetMaxBatchSize 

func (self *MongoDataSource) FindManyWithOffsetMaxBatchSize(selector interface{}, offset, max, batchSize int) *MongoCursor

The caller must close the cursor when done. Use: defer cursor.Close()

func (*MongoDataSource) FindManyWithOffsetMaxBatchSizeAndSort 

func (self *MongoDataSource) FindManyWithOffsetMaxBatchSizeAndSort(selector interface{}, offset, max, batchSize int, sortFields ...string) *MongoCursor

The caller must close the cursor when done. Use: defer cursor.Close()

func (*MongoDataSource) FindOne 

func (self *MongoDataSource) FindOne(query *bson.M, result interface{}) error

Finds one document or returns false.

func (*MongoDataSource) Insert 

func (self *MongoDataSource) Insert(doc interface{}) error

Insert a document into a collection with the base configured write concern.

func (*MongoDataSource) InsertSafe 

func (self *MongoDataSource) InsertSafe(doc interface{}) error

Insert a document into a collection with the passed write concern.

func (*MongoDataSource) IsDupErr 

func (self *MongoDataSource) IsDupErr(err error) bool

Returns true if this is a dup error. If nil is passed, false is returned.

func (*MongoDataSource) NewObjectId 

func (self *MongoDataSource) NewObjectId() *bson.ObjectId

func (*MongoDataSource) NotFoundErr 

func (self *MongoDataSource) NotFoundErr(err error) bool

Returns true if this is a document not found error. If nil is passed, false is returned.

func (*MongoDataSource) Now 

func (self *MongoDataSource) Now() *time.Time

func (*MongoDataSource) NowInUtc 

func (self *MongoDataSource) NowInUtc() *time.Time

Returns the current time in utc.

func (*MongoDataSource) PullSafe 

func (self *MongoDataSource) PullSafe(query interface{}, fieldsDoc interface{}) error

Pull from an array call using a "safe" operation. If this is a standalone mongo or a mongos, it will use: WMode: "majority". If this is a standalone mongo, it will use: w: 1

func (*MongoDataSource) PushSafe 

func (self *MongoDataSource) PushSafe(query interface{}, fieldsDoc interface{}) error

Push to an array call using a "safe" operation. If this is a standalone mongo or a mongos, it will use: WMode: "majority". If this is a standalone mongo, it will use: w: 1

func (*MongoDataSource) RemoveNotFoundErr 

func (self *MongoDataSource) RemoveNotFoundErr(err error) error

Returns nil if this is a NOT a document not found error.

func (*MongoDataSource) Session 

func (self *MongoDataSource) Session() *mgo.Session

Returns the session struct.

func (*MongoDataSource) SessionClone 

func (self *MongoDataSource) SessionClone() *mgo.Session

Returns a clone of the session struct.

func (*MongoDataSource) SessionCopy 

func (self *MongoDataSource) SessionCopy() *mgo.Session

Returns a copy of the session struct.

func (*MongoDataSource) SetFieldSafe 

func (self *MongoDataSource) SetFieldSafe(query interface{}, field string, value interface{}) error

Set a property using a "safe" operation. If this is a standalone mongo or a mongos, it will use: WMode: "majority". If this is a standalone mongo, it will use: w: 1

func (*MongoDataSource) SetFieldsSafe 

func (self *MongoDataSource) SetFieldsSafe(query interface{}, fieldsDoc interface{}) error

Set fields using a "safe" operation. If this is a standalone mongo or a mongos, it will use: WMode: "majority". If this is a standalone mongo, it will use: w: 1

func (*MongoDataSource) UnsetFieldSafe 

func (self *MongoDataSource) UnsetFieldSafe(query interface{}, field string) error

Unset a field with the default safe enabled - uses $unset

func (*MongoDataSource) UpdateSafe 

func (self *MongoDataSource) UpdateSafe(selector interface{}, change interface{}) error

Updates a document into a collection with the passed write concern.

func (*MongoDataSource) Upsert 

func (self *MongoDataSource) Upsert(selector interface{}, change interface{}) error

Upsert a document in a collection with the base configured write concern.

func (*MongoDataSource) UpsertSafe 

func (self *MongoDataSource) UpsertSafe(selector interface{}, change interface{}) error

Upsert a document into a collection with the passed write concern.

type MongoDistributedLock 

type MongoDistributedLock struct {
	Logger
	// contains filtered or unexported fields
}

The purpose of the distributed lock is to provide a lock that is available across multiple processes/servers. The distributed lock requires a central synchronization point. In this impl, MongoDB is the central synchronization server. The attempt to lock method allows to try and obtain a lock. If the try is successful, true is returned (and no error). You must call Start(kernel) and Stop(kernel) to start/stop the lock service. Given that Go does garbage collection, it is theoretically possible that the entire app could lock up for periods of time. Extended time in garbage collection could cause a distributed lock to time out. A timeout occurs when the Go process is unable to update the database lock heartbeat within a specific period of time. When a timeout occurs, there is an election to see who is granted the lock. For this reason, we added a HasLock method which returns true if the process still has the distributed lock. Network issues and database issues can also lead to lock timeouts. If a lock times out, it does not automatically allow callers access to the lock. Nothing happens until there is an election amongst the various processes.

This lock is somewhat confusing in the sense that once a process acquires the global lock, it is pinned to that process until the process is killed (or the heartbeat/lock times out/fails). Pinning the lock to a process allows problems to be debugged faster.

The MongoDB backed distributed lock uses a similar lock schema as the MongoDB config.locks collection. It looks like: 

{
	"_id" : "balancer",
	"process" : "example.net:40000:1350402818:16807",
	"state" : 2,
	"ts" : ObjectId("507daeedf40e1879df62e5f3"),
	"when" : ISODate("2012-10-16T19:01:01.593Z"),
	"who" : "example.net:40000:1350402818:16807:Balancer:282475249",
}

The lockId field maps to the _id field. The "ts" field is the "heartbeat" field. The "why" field was removed to support the Go Locker interface. When the lock is active, the "state" field is set to 2.

func (*MongoDistributedLock) HasLock 

func (self *MongoDistributedLock) HasLock() bool

func (*MongoDistributedLock) Lock 

func (self *MongoDistributedLock) Lock()

Call the lock. This method will block until (if ever) the lock is available.

func (*MongoDistributedLock) LockId 

func (self *MongoDistributedLock) LockId() string

func (*MongoDistributedLock) Start 

func (self *MongoDistributedLock) Start(kernel *Kernel) error

func (*MongoDistributedLock) Stop 

func (self *MongoDistributedLock) Stop(kernel *Kernel) error

func (*MongoDistributedLock) TryLock 

func (self *MongoDistributedLock) TryLock() bool

func (*MongoDistributedLock) Unlock 

func (self *MongoDistributedLock) Unlock()

Release the distributed lock. There is no guarantee that the same process will be granted access to the distributed lock when it is released.

type PersistedMetric 

type PersistedMetric struct {
	Id string `bson:"_id" json:"id"`

	Name   string `bson:"name" json:"name"`
	Source string `bson:"source" json:"source"`
	Type   string `bson:"type" json:"type"`

	Value    float64 `bson:"value" json:"value"`
	Previous float64 `bson:"previous" json:"previous"`

	Updated *time.Time `bson:"updated" json:"updated"`
	Created *time.Time `bson:"created" json:"created"`
}

func (*PersistedMetric) Change 

func (self *PersistedMetric) Change() float64

This method returns the change from the current - previous. Usually, only for counters.

type StackError 

type StackError struct {
	Message    string
	Stacktrace []string
}

func NewStackError 

func NewStackError(messageFmt string, args ...interface{}) *StackError

func (*StackError) Error 

func (self *StackError) Error() string

type StringAppender 

type StringAppender struct {
	*bytes.Buffer
}

func NewStringAppender 

func NewStringAppender(buffer *bytes.Buffer) *StringAppender

func (StringAppender) Append 

func (self StringAppender) Append(log *Log) error

type SyslogAppender 

type SyslogAppender struct {
	// contains filtered or unexported fields
}

func NewSyslogAppender 

func NewSyslogAppender(network, raddr, appId string) (*SyslogAppender, error)

func (*SyslogAppender) Append 

func (self *SyslogAppender) Append(log *Log) error

type TcpSocketProcessor 

type TcpSocketProcessor struct {
	Logger
	// contains filtered or unexported fields
}

The TCP socket processor is a wrapper around a TCP socket that reconnects if there is a failure. It also spawns a reading and a writing goroutine to handle data in/out. After the struct is created, you must call Start. When you are done using, call the Stop method. Make sure you check the error returned when calling Start. If this is a tls connection, the component will not run if either either the certificate or key file is not accessible. The component will also not run if it the address is not set.

func NewTcpSocketProcessor 

func NewTcpSocketProcessor(address string,
	connectTimeoutInMs int64,
	readTimeoutInMs int64,
	writeTimeoutInMs int64,
	readBufferSize int,
	writeChannel chan TcpSocketProcessorWrite,
	readChannel chan TcpSocketProcessorRead,
	logger Logger) *TcpSocketProcessor

Create a new tcp socket processor.

func NewTlsTcpSocketProcessor 

func NewTlsTcpSocketProcessor(address string,
	connectTimeoutInMs int64,
	readTimeoutInMs int64,
	writeTimeoutInMs int64,
	readBufferSize int,
	writeChannel chan TcpSocketProcessorWrite,
	readChannel chan TcpSocketProcessorRead,
	logger Logger,
	certificateFile,
	keyFile string) *TcpSocketProcessor

Create a new tcp socket processor.

func (*TcpSocketProcessor) Start 

func (self *TcpSocketProcessor) Start() error

func (*TcpSocketProcessor) Stop 

func (self *TcpSocketProcessor) Stop() error

type TcpSocketProcessorRead 

type TcpSocketProcessorRead struct {
	Data      []byte
	BytesRead int
	Error     error
}

type TcpSocketProcessorWrite 

type TcpSocketProcessorWrite struct {
	Data            []byte
	ResponseChannel chan TcpSocketProcessorWrite
	BytesWritten    int
	Error           error
}

type TemplateSvc 

type TemplateSvc struct {
	// contains filtered or unexported fields
}

func NewTemplateSvc 

func NewTemplateSvc(baseTemplateSvcDir string) *TemplateSvc

Create a new template service struct. This simplifies accessing and rendering templates. End your base template directory with a slash (/).

func (*TemplateSvc) RenderHtml 

func (self *TemplateSvc) RenderHtml(templateFileName string, params interface{}) (string, error)

The template file name should not start with a slash and be under the baseTemplateSvcDir

func (*TemplateSvc) RenderHtmlAndText 

func (self *TemplateSvc) RenderHtmlAndText(htmlTemplateSvcFileName, textTemplateSvcFileName string, params interface{}) (bodyHtml, bodyText string, err error)

The html template output is the first param and the text is the second param.

func (*TemplateSvc) RenderText 

func (self *TemplateSvc) RenderText(templateFileName string, params interface{}) (string, error)

The template file name should not start with a slash and be under the baseTemplateSvcDir

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.