incus

package module
v0.0.0-...-825a8c7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 11, 2015 License: MIT Imports: 23 Imported by: 3

README

Incus Build Status Coverage Status

incus

Middleware for distributing messages via websockets, long polling, and push notifications

Features

  • Websocket authentication and management
  • Long Poll fall back
  • iOS push notifications support through APNS
  • Android push notifications support through GCM
  • Routing messages to specific phone, authenticated user, or webpage url
  • Configurable option for allowing users to send messages to other users
  • Redis pub/sub and Redis List support for sending messages from an application
  • SSL support
  • Stats logging

Usage

diagram

Front-end bindings
JavaScript: incus.js

The incus.js front-end npm browserified module is provided for consuming WebSocket events in the Browser or server-side. Self-contained, minified downloads are also available.

Application to a web browser

UID is a unique identifier for the user. It can be anywhere from an auto incremented ID to something more private such as a session id or OAuth token. If messages to users should be private from other users, then you should consider the UID a shared secret between the user and the server.

To send events to Incus from your webapp you need to publish a json formated string to a Redis pub/sub channel that Incus is listening on. This channel key can be configured but defaults to Incus. The json format is as follows:

{
    "command" : {
        "command" : string (message|setpage),
        "user"    : (optional) string -- Unique User ID,
        "page"    : (optional) string -- page identifier
    },
    "message" : {
        "event" : string,
        "data"  : object,
        "time"  : int
    }
}

the command is used to route the message to the correct user.

  • if user and page are both unset the message object will be sent to all users
  • if both user and page are set the message object will be sent to that user on that page
  • if just user is set, the message object will be sent to all sockets owned by the user identified by UID
  • if just page is set, the message object will be sent to all sockets whose page matches the page identifier
Push notifications

To send push notifications from your app, you need to push a json formated string to a Redis list. The list key is configurable but defaults to Incus_Queue

Android and iOS have slightly different schemas for sending push notifications.

iOS:
{
    "command" : {
        "command"      : "push",
        "push_type"    : "ios",
        "device_token" : string -- device token registered with APNS,
        "build"        : string -- build environment (store|beta|enterprise|development)
    },
    "message" : {
        "event" : string,
        "data"  : {
            "badge_count": optional int,
            "message_text": string,
            ...,
        },
        "time"  : int
    }
}

Notes:

  • At this time, dictionary APNS alerts are not supported.
Android:

Multiple registration ids can be listed in the same command

{
    "command" : {
        "command"          : "push",
        "push_type"        : "android",
        "registration_ids" : string -- one or more registration ids separated by commas
    },
    "message" : {
        "event" : string,
        "data"  : object,
        "time"  : int
    }
}
Presence-based message routing
{
    "command" : {
        "command": "pushormessage",
        "user": string -- Unique User ID,
        "device_token": string -- device token registered with APNS,
        "build": string -- build environment (store|beta|enterprise|development)
        "registration_ids": string -- one or more registration ids separated by commas
    },
    "message" : {
        "push": {
            "ios": {
                "badge_count": optional int,
                "message_text": string,
                ...
            },
            "android": {
                ...
            }
        },
        "websocket": {
            ...
        }
        "time": int
    }
}
APNS and GCM errors

Incus does not interact with the APNS Feedback Service. You should follow the APNS' guidelines on failed push attempts. They require querying their feedback service daily to find bad device tokens.

The GCM service does not offer a feedback service. When a push fails, Incus will add all relevant information to an error list in Redis (defaults to Incus_Android_Error_Queue). This should be used to remove bad registration ids from your app.

Installation

Method 1: Docker
  • Install Docker

  • Create a folder to share with Docker container:

mkdir -p ~/incus
  • Download configuration file and edit it at will:
wget https://raw.githubusercontent.com/Imgur/incus/master/config.yml -O ~/incus/config.yml

  • Start an instance of Redis (optional), skip it if you run your own Redis instance:
docker run -d --name incusredis redis

Start Incus:

docker run -i --name incus -p 4000:4000 --net="host" -v ~/incus:/etc/incus imgur/incus 

To stop Incus run:

docker stop incus

To run it again simply:

docker run incus
Method 2: Source

Install GO: https://golang.org/doc/install

Clone the repo:

mkdir $GOPATH/src/github.com/Imgur/
cd $GOPATH/src/github.com/Imgur/
git clone git@github.com:Imgur/incus.git

To Install:

cd $GOPATH/src/github.com/Imgur/incus
go get -v ./...
go install -v ./incus
cp $GOPATH/bin/incus /usr/sbin/incus
cp scripts/initd.sh /etc/init.d/incus
mkdir /etc/incus
cp config.yml /etc/incus/config.yml
touch /var/log/incus.log

Starting, Stopping, Restarting incus:

sudo /etc/init.d/incus start
sudo /etc/init.d/incus stop
sudo /etc/init.d/incus restart

Configuration

Incus needs to be restarted after any configuration change.

CLIENT_BROADCASTS

true

Clients may send messages to other clients

false

Clients may not send messages to other clients.

Default: true


LISTENING_PORT

This value controls the port that Incus binds to (TCP).

Default: 4000


CONNECTION_TIMEOUT (unstable)

This value controls how long TCP connections are held open for.

0

Connections are held open forever.

anything else

Connections are held open for this many seconds.

Default: 0


LOG_LEVEL

debug

All messages, including errors and debug are printed to standard output.

error

Only errors are printed to standard output.

Default: debug


REDIS_PORT_6379_TCP_ADDR

This value controls the TCP address (or hostname) to connect to Redis.

Note: The variable name is always REDIS_PORT_6379_TCP_ADDR even if the port is not 6379.

Default: 127.0.0.1


REDIS_PORT_6379_TCP_PORT

This value controls the TCP port to connect to Redis.

Note: The variable name is always REDIS_PORT_6379_TCP_PORT even if the port is not 6379.

Default: 6379


REDIS_MESSAGE_CHANNEL

This value controls the Redis PubSub channel to use.

Default: Incus


TLS_ENABLED

This value controls whether the server will also listen on a TLS-enabled port.

false

TLS is disabled, so the server will only listen over its insecure port.

true

TLS is enabled, so the socket will listen over both its insecure port and its TLS-enabled secure port.

Default: false


TLS_PORT

This value controls what TCP port is exposed when using TLS

Default: 443


CERT_FILE

This value controls what X.509 certificate is offered to clients connecting on the TLS port. The certificate is expected in PEM format. The value is a path name resolved relative to the working directory of Incus.

Default: cert.pem


KEY_FILE

This value controls what X.509 private key is used for decrypting the TLS traffic. The key is expected in PEM format.

Default: key.pem


APNS_ENABLED

This value controls whether the server will listen for and send iOS push notifications

false

APSN is disabled, the server will not listen for iOS push notifications

true

APNS is enabled, the server will listen for and send iOS push notifications

Default: false


APNS_[BUILD]_CERT

Where [BUILD] is one of: DEVELOPMENT, STORE, ENTERPRISE, or BETA

This value controls what APNS granted cert the server will use when calling the APNS API. Each build environment has its own instance of this configuration variable.

Default: myapnsappcert.pem


APNS_[BUILD]_PRIVATE_KEY

Where [BUILD] is one of: DEVELOPMENT, STORE, ENTERPRISE, or BETA

This value controls what APNS granted private key the server will use when calling the APNS API. Each build environment has its own instance of this configuration variable.

Default: myapnsappprivatekey.pem


APNS_[BUILD]_URL

Where [BUILD] is one of: DEVELOPMENT, STORE, ENTERPRISE, or BETA

This value controls what APNS url is used when calling the APNS API. Each build environment has its own instance of this configuration variable.

Default: gateway.push.apple.com:2195

APNS_DEVELOPMENT_URL defaults to gateway.sandbox.push.apple.com:2195


IOS_PUSH_SOUND

This value controls what sound plays on push notification receive.

Default: bingbong.aiff


GCM_ENABLED

This value controls whether the server will listen for and send Android push notifications

false

GCM is disabled, the server will not listen for Android push notifications

true

GCM is enabled, the server will listen for and send Android push notifications

Default: false


GCM_API_KEY

This is the GCM granted api key used for calling the GCM API.

Default: foobar


ANDROID_ERROR_QUEUE

This value controls where Android push errors are stored for later retrieval.

Default: Incus_Android_Error_Queue

Documentation

Index

Constants

View Source
const ClientsKey = "SocketClients"
View Source
const PageKey = "PageClients"
View Source
const PresenceKeyPrefix = "ClientPresence"

Variables

View Source
var (
	DEBUG        bool = false
	CLIENT_BROAD bool = false
)

Functions

func ConfigOption

func ConfigOption(key string, default_value interface{}) string

func NewConfig

func NewConfig(configFilePath string)

Types

type CommandMsg

type CommandMsg struct {
	Command map[string]string      `json:"command"`
	Message map[string]interface{} `json:"message,omitempty"`
}

func (*CommandMsg) FromRedis

func (this *CommandMsg) FromRedis(server *Server)

func (*CommandMsg) FromSocket

func (this *CommandMsg) FromSocket(sock *Socket)

type DatadogStats

type DatadogStats struct {
	// contains filtered or unexported fields
}

func NewDatadogStats

func NewDatadogStats(datadogHost string) (*DatadogStats, error)

func (*DatadogStats) LogAPNSError

func (d *DatadogStats) LogAPNSError()

func (*DatadogStats) LogAPNSPush

func (d *DatadogStats) LogAPNSPush()

func (*DatadogStats) LogBroadcastMessage

func (d *DatadogStats) LogBroadcastMessage()

func (*DatadogStats) LogClientCount

func (d *DatadogStats) LogClientCount(clients int64)

func (*DatadogStats) LogCommand

func (d *DatadogStats) LogCommand(from, cmdType string)

func (*DatadogStats) LogGCMError

func (d *DatadogStats) LogGCMError()

func (*DatadogStats) LogGCMFailure

func (d *DatadogStats) LogGCMFailure()

func (*DatadogStats) LogGCMPush

func (d *DatadogStats) LogGCMPush()

func (*DatadogStats) LogGoroutines

func (d *DatadogStats) LogGoroutines(goroutines int)

func (*DatadogStats) LogInvalidJSON

func (d *DatadogStats) LogInvalidJSON()

func (*DatadogStats) LogLongpollConnect

func (d *DatadogStats) LogLongpollConnect()

func (*DatadogStats) LogLongpollDisconnect

func (d *DatadogStats) LogLongpollDisconnect()

func (*DatadogStats) LogPageMessage

func (d *DatadogStats) LogPageMessage()

func (*DatadogStats) LogPendingRedisActivityCommandsListLength

func (d *DatadogStats) LogPendingRedisActivityCommandsListLength(length int)

func (*DatadogStats) LogReadMessage

func (d *DatadogStats) LogReadMessage()

func (*DatadogStats) LogStartup

func (d *DatadogStats) LogStartup()

func (*DatadogStats) LogUserMessage

func (d *DatadogStats) LogUserMessage()

func (*DatadogStats) LogWebsocketConnection

func (d *DatadogStats) LogWebsocketConnection()

func (*DatadogStats) LogWebsocketDisconnection

func (d *DatadogStats) LogWebsocketDisconnection()

func (*DatadogStats) LogWriteMessage

func (d *DatadogStats) LogWriteMessage()

type DiscardStats

type DiscardStats struct{}

func (*DiscardStats) LogAPNSError

func (d *DiscardStats) LogAPNSError()

func (*DiscardStats) LogAPNSPush

func (d *DiscardStats) LogAPNSPush()

func (*DiscardStats) LogBroadcastMessage

func (d *DiscardStats) LogBroadcastMessage()

func (*DiscardStats) LogClientCount

func (d *DiscardStats) LogClientCount(int64)

func (*DiscardStats) LogCommand

func (d *DiscardStats) LogCommand(from, cmdType string)

func (*DiscardStats) LogGCMError

func (d *DiscardStats) LogGCMError()

func (*DiscardStats) LogGCMFailure

func (d *DiscardStats) LogGCMFailure()

func (*DiscardStats) LogGCMPush

func (d *DiscardStats) LogGCMPush()

func (*DiscardStats) LogGoroutines

func (d *DiscardStats) LogGoroutines(int)

func (*DiscardStats) LogInvalidJSON

func (d *DiscardStats) LogInvalidJSON()

func (*DiscardStats) LogLongpollConnect

func (d *DiscardStats) LogLongpollConnect()

func (*DiscardStats) LogLongpollDisconnect

func (d *DiscardStats) LogLongpollDisconnect()

func (*DiscardStats) LogPageMessage

func (d *DiscardStats) LogPageMessage()

func (*DiscardStats) LogPendingRedisActivityCommandsListLength

func (d *DiscardStats) LogPendingRedisActivityCommandsListLength(int)

func (*DiscardStats) LogReadMessage

func (d *DiscardStats) LogReadMessage()

func (*DiscardStats) LogStartup

func (d *DiscardStats) LogStartup()

func (*DiscardStats) LogUserMessage

func (d *DiscardStats) LogUserMessage()

func (*DiscardStats) LogWebsocketConnection

func (d *DiscardStats) LogWebsocketConnection()

func (*DiscardStats) LogWebsocketDisconnection

func (d *DiscardStats) LogWebsocketDisconnection()

func (*DiscardStats) LogWriteMessage

func (d *DiscardStats) LogWriteMessage()

type GCMClient

type GCMClient interface {
	Send(*gcm.Message, int) (*gcm.Response, error)
}

type MemoryStore

type MemoryStore struct {
	// contains filtered or unexported fields
}

func (*MemoryStore) Client

func (this *MemoryStore) Client(UID string) (map[string]*Socket, error)

func (*MemoryStore) Clients

func (this *MemoryStore) Clients() map[string]map[string]*Socket

func (*MemoryStore) Count

func (this *MemoryStore) Count() (int64, error)

func (*MemoryStore) Remove

func (this *MemoryStore) Remove(sock *Socket) error

func (*MemoryStore) Save

func (this *MemoryStore) Save(sock *Socket) error

func (*MemoryStore) SetPage

func (this *MemoryStore) SetPage(sock *Socket) error

func (*MemoryStore) UnsetPage

func (this *MemoryStore) UnsetPage(sock *Socket) error

type Message

type Message struct {
	Event string                 `json:"event"`
	Data  map[string]interface{} `json:"data"`
	Time  int64                  `json:"time"`
	Url   string                 `json:"internal_url,omitempty"`
}

type RedisCallback

type RedisCallback (func(redis.Conn) (interface{}, error))

type RedisCommand

type RedisCommand struct {
	Callback RedisCallback
	Result   chan RedisCommandResult
}

type RedisCommandResult

type RedisCommandResult struct {
	Error error
	Value interface{}
}

type RedisQueue

type RedisQueue struct {
	// contains filtered or unexported fields
}

A queue of redis commands

func NewRedisQueue

func NewRedisQueue(consumers int, stats RuntimeStats, pool *redisPool) *RedisQueue

func (*RedisQueue) DispatchForever

func (r *RedisQueue) DispatchForever()

Dispatch a command to a goroutine that is blocking on receiving a command.

func (*RedisQueue) ReceiveForever

func (r *RedisQueue) ReceiveForever()

Receive a command, save/enqueue it, and wake the dispatching goroutine

func (*RedisQueue) RunAsyncTimeout

func (r *RedisQueue) RunAsyncTimeout(timeout time.Duration, callback RedisCallback) RedisCommandResult

type RedisQueueConsumer

type RedisQueueConsumer struct {
	// contains filtered or unexported fields
}

func NewRedisQueueConsumer

func NewRedisQueueConsumer(commands <-chan RedisCommand, pool *redisPool) *RedisQueueConsumer

func (*RedisQueueConsumer) ConsumeForever

func (r *RedisQueueConsumer) ConsumeForever()

type RedisStore

type RedisStore struct {
	// contains filtered or unexported fields
}

func (*RedisStore) ActivateLongpollKillswitch

func (this *RedisStore) ActivateLongpollKillswitch(seconds int64) error

func (*RedisStore) Clients

func (this *RedisStore) Clients() ([]string, error)

func (*RedisStore) CloseConn

func (this *RedisStore) CloseConn(conn redis.Conn)

func (*RedisStore) Count

func (this *RedisStore) Count() (int64, error)

func (*RedisStore) DeactivateLongpollKillswitch

func (this *RedisStore) DeactivateLongpollKillswitch() error

func (*RedisStore) GetConn

func (this *RedisStore) GetConn() (redis.Conn, error)

func (*RedisStore) GetIsLongpollKillswitchActive

func (this *RedisStore) GetIsLongpollKillswitchActive() (bool, error)

func (*RedisStore) MarkActive

func (this *RedisStore) MarkActive(user, socket_id string, timestamp int64) error

func (*RedisStore) MarkInactive

func (this *RedisStore) MarkInactive(user, socket_id string) error

func (*RedisStore) Poll

func (this *RedisStore) Poll(c chan []byte, queue string) error

func (*RedisStore) Publish

func (this *RedisStore) Publish(channel string, message string)

func (*RedisStore) Push

func (this *RedisStore) Push(queue string, message string)

func (*RedisStore) QueryIsUserActive

func (this *RedisStore) QueryIsUserActive(user string, nowTimestamp int64) (bool, error)

func (*RedisStore) Remove

func (this *RedisStore) Remove(sock *Socket) error

func (*RedisStore) Save

func (this *RedisStore) Save(sock *Socket) error

func (*RedisStore) SetPage

func (this *RedisStore) SetPage(sock *Socket) error

func (*RedisStore) Subscribe

func (this *RedisStore) Subscribe(c chan []byte, channel string) (redis.Conn, error)

func (*RedisStore) UnsetPage

func (this *RedisStore) UnsetPage(sock *Socket) error

type RuntimeStats

type RuntimeStats interface {
	LogStartup()

	LogClientCount(int64)
	LogGoroutines(int)

	LogCommand(from, cmdType string)
	LogPageMessage()
	LogUserMessage()
	LogBroadcastMessage()
	LogReadMessage()
	LogWriteMessage()
	LogInvalidJSON()

	LogWebsocketConnection()
	LogWebsocketDisconnection()

	LogLongpollConnect()
	LogLongpollDisconnect()

	LogAPNSPush()
	LogAPNSError()

	LogGCMPush()
	LogGCMError()
	LogGCMFailure()

	LogPendingRedisActivityCommandsListLength(int)
}

type Server

type Server struct {
	ID    string
	Store *Storage
	Stats RuntimeStats
	// contains filtered or unexported fields
}

func NewServer

func NewServer(store *Storage, stats RuntimeStats) *Server

func (*Server) GetAPNSClient

func (this *Server) GetAPNSClient(build string) apns.APNSClient

func (*Server) GetGCMClient

func (this *Server) GetGCMClient() GCMClient

func (*Server) ListenForHTTPPings

func (this *Server) ListenForHTTPPings()

func (*Server) ListenFromLongpoll

func (this *Server) ListenFromLongpoll()

func (*Server) ListenFromRedis

func (this *Server) ListenFromRedis()

func (*Server) ListenFromSockets

func (this *Server) ListenFromSockets()

func (*Server) LogConnectedClientsPeriodically

func (this *Server) LogConnectedClientsPeriodically(period time.Duration)

func (*Server) MonitorLongpollKillswitch

func (this *Server) MonitorLongpollKillswitch()

func (*Server) RecordStats

func (this *Server) RecordStats(period time.Duration)

func (*Server) SendHeartbeatsPeriodically

func (this *Server) SendHeartbeatsPeriodically(period time.Duration)

type Socket

type Socket struct {
	SID  string // socket ID, randomly generated
	UID  string // User ID, passed in via client
	Page string // Current page, if set.

	Server *Server
	// contains filtered or unexported fields
}

func (*Socket) Authenticate

func (this *Socket) Authenticate(UID string) error

func (*Socket) Close

func (this *Socket) Close() error

type Storage

type Storage struct {
	StorageType string
	// contains filtered or unexported fields
}

func NewStore

func NewStore(stats RuntimeStats) *Storage

func (*Storage) Client

func (this *Storage) Client(UID string) (map[string]*Socket, error)

func (*Storage) ClientList

func (this *Storage) ClientList() ([]string, error)

func (*Storage) Clients

func (this *Storage) Clients() map[string]map[string]*Socket

func (*Storage) Count

func (this *Storage) Count() (int64, error)

func (*Storage) Remove

func (this *Storage) Remove(sock *Socket) error

func (*Storage) Save

func (this *Storage) Save(sock *Socket) error

func (*Storage) SetPage

func (this *Storage) SetPage(sock *Socket) error

func (*Storage) UnsetPage

func (this *Storage) UnsetPage(sock *Socket) error

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL