iris-go.v1: gopkg.in/project-iris/iris-go.v1 Index | Files

package iris

import "gopkg.in/project-iris/iris-go.v1"

Package iris contains the Go binding to the Iris cloud messaging framework.

Note, since GoDoc doesn't support any markup, a significant amount of visual polishes on the documentation were lost. For the fully marked up version of the introductory docs, please consult the package readme file.

https://github.com/project-iris/iris-go/blob/v1/README.md

Background

Iris is an attempt at bringing the simplicity and elegance of cloud computing to the application layer. Consumer clouds provide unlimited virtual machines at the click of a button, but leaves it to developer to wire them together. Iris ensures that you can forget about networking challenges and instead focus on solving your own domain problems.

It is a completely decentralized messaging solution for simplifying the design and implementation of cloud services. Among others, Iris features zero- configuration (i.e. start it up and it will do its magic), semantic addressing (i.e. application use textual names to address each other), clusters as units (i.e. automatic load balancing between apps of the same name) and perfect secrecy (i.e. all network traffic is encrypted).

You can find further infos on the Iris website [http://iris.karalabe.com] and details of the above features in the core concepts [http://iris.karalabe.com/book/core_concepts] section of the book of Iris [http://iris.karalabe.com/book]. For the scientifically inclined, a small collection of papers [http://iris.karalabe.com/papers] is also available featuring Iris. Slides and videos of previously given public presentations are published in the talks [http://iris.karalabe.com/talks] page.

There is a growing community on Twitter @iriscmf [https://twitter.com/iriscmf], Google groups project-iris [https://groups.google.com/group/project-iris] and GitHub project-iris [https://github.com/project-iris].

Installation

To get the package, execute:

go get gopkg.in/project-iris/iris-go.v1

To import this package, add the following line to your code:

import "gopkg.in/project-iris/iris-go.v1"

Refer to it as iris.

Quickstart

Iris uses a relaying architecture, where client applications do not communicate directly with one another, but instead delegate all messaging operations to a local relay process responsible for transferring the messages to the correct destinations. The first step hence to using Iris through any binding is setting up the local relay node [http://iris.karalabe.com/downloads]. You can find detailed infos in the Run, Forrest, Run [http://iris.karalabe.com/book/run_forrest_run] section of the book of Iris [http://iris.karalabe.com/book], but a very simple way would be to start a developer node.

> iris -dev
Entering developer mode
Generating random RSA key... done.
Generating random network name... done.

2014/06/13 18:13:47 main: booting iris overlay...
2014/06/13 18:13:47 scribe: booting with id 369650985814.
2014/06/13 18:13:57 main: iris overlay converged with 0 remote connections.
2014/06/13 18:13:57 main: booting relay service...
2014/06/13 18:13:57 main: iris successfully booted, listening on port 55555.

Since it generates random credentials, a developer node will not be able to connect with other remote nodes in the network. However, it provides a quick solution to start developing without needing to configure a network name and associated access key. Should you wish to interconnect multiple nodes, please provide the -net and -rsa flags.

Attaching to the relay

After successfully booting, the relay opens a local TCP endpoint (port 55555 by default, configurable using -port) through which arbitrarily many entities may attach. Each connecting entity may also decide whether it becomes a simple client only consuming the services provided by other participants, or a full fledged service, also making functionality available to others for consumption.

Connecting as a client can be done trivially by invoking iris.Connect with the port number of the local relay's client endpoint. After the attachment is completed, an iris.Connection instance is returned through which messaging can begin. A client cannot accept inbound requests, broadcasts and tunnels, only initiate them.

conn, err := iris.Connect(55555)
if err != nil {
  log.Fatalf("failed to connect to the Iris relay: %v.", err)
}
defer conn.Close()

To provide functionality for consumption, an entity needs to register as a service. This is slightly more involved, as beside initiating a registration request, it also needs to specify a callback handler to process inbound events. First, the callback handler needs to implement the iris.ServiceHandler interface. After creating the handler, registration can commence by invoking iris.Register with the port number of the local relay's client endpoint; sub-service cluster this entity will join as a member; handler itself to process inbound messages and an optional resource cap.

type EchoHandler struct {}

func (b *EchoHandler) Init(conn *Connection) error              { return nil }
func (b *EchoHandler) HandleBroadcast(msg []byte)               { }
func (b *EchoHandler) HandleRequest(req []byte) ([]byte, error) { return req, nil }
func (b *EchoHandler) HandleTunnel(tun *Tunnel)                 { }
func (b *EchoHandler) HandleDrop(reason error)                  { }

func main() {
  service, err := iris.Register(55555, "echo", new(EchoHandler), nil)
  if err != nil {
    log.Fatalf("failed to register to the Iris relay: %v.", err)
  }
  defer service.Unregister()
}

Upon successful registration, Iris invokes the handler's Init method with the live iris.Connection object - the service's client connection - through which the service itself can initiate outbound requests. Init is called only once and is synchronized before any other handler method is invoked.

Messaging through Iris

Iris supports four messaging schemes: request/reply, broadcast, tunnel and publish/subscribe. The first three schemes always target a specific cluster: send a request to one member of a cluster and wait for the reply; broadcast a message to all members of a cluster; open a streamed, ordered and throttled communication tunnel to one member of a cluster. The publish/subscribe is similar to broadcast, but any member of the network may subscribe to the same topic, hence breaking cluster boundaries.

Schemes overview figure: https://dl.dropboxusercontent.com/u/10435909/Iris/messaging_schemes.png

Presenting each primitive is out of scope, but for illustrative purposes the request/reply was included. Given the echo service registered above, we can send it requests and wait for replies through any client connection. Iris will automatically locate, route and load balanced between all services registered under the addressed name.

request := []byte("some request binary")
if reply, err := conn.Request("echo", request, time.Second); err != nil {
  log.Printf("failed to execute request: %v.", err)
} else {
  fmt.Printf("reply arrived: %v.", string(reply))
}

An expanded summary of the supported messaging schemes can be found in the core concepts [http://iris.karalabe.com/book/core_concepts] section of the book of Iris [http://iris.karalabe.com/book]. A detailed presentation and analysis of each individual primitive will be added soon.

Error handling

The binding uses the idiomatic Go error handling mechanisms of returning error instances whenever a failure occurs. However, there are a few common cases that need to be individually checkable, hence a few special errors values and types have been introduced.

Many operations - such as requests and tunnels - can time out. To allow checking for this particular failure, Iris returns iris.ErrTimeout in such scenarios. Similarly, connections, services and tunnels may fail, in the case of which all pending operations terminate with iris.ErrClosed.

Additionally, the requests/reply pattern supports sending back an error instead of a reply to the caller. To enable the originating node to check whether a request failed locally or remotely, all remote errors are wrapped in an iris.RemoteError type.

_, err := conn.Request("cluster", request, timeout)
switch err {
  case nil:
    // Request completed successfully
  case iris.ErrTimeout:
    // Request timed out
  case iris.ErrClosed:
    // Connection terminated
  default:
    if _, ok := err.(*iris.RemoteError); ok {
      // Request failed remotely
    } else {
      // Requesting failed locally
    }
}

Resource capping

To prevent the network from overwhelming an attached process, the binding places thread and memory limits on the broadcasts/requests inbound to a registered service as well as on the events received by a topic subscription. The thread limit defines the concurrent processing allowance, whereas the memory limit the maximal length of the pending queue.

The default values - listed below - can be overridden during service registration and topic subscription via iris.ServiceLimits and iris.TopicLimits. Any unset fields (i.e. value of zero) will default to the preset ones.

// Default limits of the threading and memory usage of a registered service.
var defaultServiceLimits = ServiceLimits{
  BroadcastThreads: 4 * runtime.NumCPU(),
  BroadcastMemory:  64 * 1024 * 1024,
  RequestThreads:   4 * runtime.NumCPU(),
  RequestMemory:    64 * 1024 * 1024,
}

// Default limits of the threading and memory usage of a subscription.
var defaultTopicLimits = TopicLimits{
  EventThreads: 4 * runtime.NumCPU(),
  EventMemory:  64 * 1024 * 1024,
}

There is also a sanity limit on the input buffer of a tunnel, but it is not exposed through the API as tunnels are meant as structural primitives, not sensitive to load. This may change in the future.

Logging

For logging purposes, the Go binding uses inconshreveable's [https://github.com/inconshreveable] log15 [https://github.com/inconshreveable/log15] library (version v2). By default, INFO level logs are collected and printed to stderr. This level allows tracking life-cycle events such as client and service attachments, topic subscriptions and tunnel establishments. Further log entries can be requested by lowering the level to DEBUG, effectively printing all messages passing through the binding.

The binding's logger can be fine-tuned through the iris.Log variable. Below are a few common configurations.

// Discard all log entries
iris.Log.SetHandler(log15.DiscardHandler())

// Log DEBUG level entries to STDERR
iris.Log.SetHandler(log15.LvlFilterHandler(log15.LvlDebug, log15.StderrHandler))

Each iris.Connection, iris.Service and iris.Tunnel has an embedded logger, through which contextual log entries may be printed (i.e. tagged with the specific id of the attached entity).

conn, _ := iris.Connect(55555)
defer conn.Close()

conn.Log.Debug("debug entry, hidden by default")
conn.Log.Info("info entry, client context included")
conn.Log.Warn("warning entry", "extra", "some value")
conn.Log.Crit("critical entry", "bool", false, "int", 1, "string", "two")

As you can see below, all log entries have been automatically tagged with the 'client' attribute, set to the id of the current connection. Since the default log level is INFO, the conn.Log.Debug invocation has no effect. Additionally, arbitrarily many key-value pairs may be included in the entry.

INFO[06-22|18:39:49] connecting new client                    client=1 relay_port=55555
INFO[06-22|18:39:49] client connection established            client=1
INFO[06-22|18:39:49] info entry, client context included      client=1
WARN[06-22|18:39:49] warning entry                            client=1 extra="some value"
CRIT[06-22|18:39:49] critical entry                           client=1 bool=false int=1 string=two
INFO[06-22|18:39:49] detaching from relay                     client=1

For further capabilities, configurations and details about the logger, please consult the log15 docs [https://godoc.org/github.com/inconshreveable/log15].

Additional goodies

You can find a teaser presentation, touching on all the key features of the library through a handful of challenges and their solutions. The recommended version is the playground [http://play.iris.karalabe.com/talks/binds/go.slide], containing modifiable and executable code snippets, but a read only version [http://iris.karalabe.com/talks/binds/go.slid]) is also available.

Index

Package Files

connection.go doc.go errors.go events.go limits.go logger.go proto.go service.go topic.go tunnel.go

Variables

var ErrClosed = errors.New("entity closed")

Returned if an operation is requested on a closed entity.

var ErrTimeout = errors.New("operation timed out")

Returned whenever a time-limited operation expires.

var Log = log15.New()

User configurable leveled logger.

type Connection Uses

type Connection struct {
    Log log15.Logger // Logger with connection id injected
    // contains filtered or unexported fields
}

Client connection to the Iris network.

func Connect Uses

func Connect(port int) (*Connection, error)

Connects to the Iris network as a simple client.

func (*Connection) Broadcast Uses

func (c *Connection) Broadcast(cluster string, message []byte) error

Broadcasts a message to all members of a cluster. No guarantees are made that all recipients receive the message (best effort).

The call blocks until the message is forwarded to the local Iris node.

func (*Connection) Close Uses

func (c *Connection) Close() error

Gracefully terminates the connection removing all subscriptions and closing all active tunnels.

The call blocks until the connection tear-down is confirmed by the Iris node.

func (*Connection) Publish Uses

func (c *Connection) Publish(topic string, event []byte) error

Publishes an event asynchronously to topic. No guarantees are made that all subscribers receive the message (best effort).

The method blocks until the message is forwarded to the local Iris node.

func (*Connection) Request Uses

func (c *Connection) Request(cluster string, request []byte, timeout time.Duration) ([]byte, error)

Executes a synchronous request to be serviced by a member of the specified cluster, load-balanced between all participant, returning the received reply.

The timeout unit is in milliseconds. Anything lower will fail with an error.

func (*Connection) Subscribe Uses

func (c *Connection) Subscribe(topic string, handler TopicHandler, limits *TopicLimits) error

Subscribes to a topic, using handler as the callback for arriving events.

The method blocks until the subscription is forwarded to the relay. There might be a small delay between subscription completion and start of event delivery. This is caused by subscription propagation through the network.

func (*Connection) Tunnel Uses

func (c *Connection) Tunnel(cluster string, timeout time.Duration) (*Tunnel, error)

Opens a direct tunnel to a member of a remote cluster, allowing pairwise- exclusive, order-guaranteed and throttled message passing between them.

The method blocks until the newly created tunnel is set up, or the time limit is reached.

The timeout unit is in milliseconds. Anything lower will fail with an error.

func (*Connection) Unsubscribe Uses

func (c *Connection) Unsubscribe(topic string) error

Unsubscribes from topic, receiving no more event notifications for it.

The method blocks until the unsubscription is forwarded to the local Iris node.

type RemoteError Uses

type RemoteError struct {
    // contains filtered or unexported fields
}

Wrapper to differentiate between local and remote errors.

type Service Uses

type Service struct {
    Log log15.Logger // Logger with service id injected
    // contains filtered or unexported fields
}

Service instance belonging to a particular cluster in the network.

func Register Uses

func Register(port int, cluster string, handler ServiceHandler, limits *ServiceLimits) (*Service, error)

Connects to the Iris network and registers a new service instance as a member of the specified service cluster.

func (*Service) Unregister Uses

func (s *Service) Unregister() error

Unregisters the service instance from the Iris network, removing all subscriptions and closing all active tunnels.

The call blocks until the tear-down is confirmed by the Iris node.

type ServiceHandler Uses

type ServiceHandler interface {
    // Called once after the service is registered in the Iris network, but before
    // and handlers are activated. Its goal is to initialize any internal state
    // dependent on the connection.
    Init(conn *Connection) error

    // Callback invoked whenever a broadcast message arrives designated to the
    // cluster of which this particular service instance is part of.
    HandleBroadcast(message []byte)

    // Callback invoked whenever a request designated to the service's cluster is
    // load-balanced to this particular service instance.
    //
    // The method should service the request and return either a reply or the
    // error encountered, which will be delivered to the request originator.
    //
    // Returning nil for both or none of the results will result in a panic. Also,
    // since the requests cross language boundaries, only the error string gets
    // delivered remotely (any associated type information is effectively lost).
    HandleRequest(request []byte) ([]byte, error)

    // Callback invoked whenever a tunnel designated to the service's cluster is
    // constructed from a remote node to this particular instance.
    HandleTunnel(tunnel *Tunnel)

    // Callback notifying the service that the local relay dropped its connection.
    HandleDrop(reason error)
}

Callback interface for processing inbound messages designated to a particular service instance.

type ServiceLimits Uses

type ServiceLimits struct {
    BroadcastThreads int // Broadcast handlers to execute concurrently
    BroadcastMemory  int // Memory allowance for pending broadcasts
    RequestThreads   int // Request handlers to execute concurrently
    RequestMemory    int // Memory allowance for pending requests
}

User limits of the threading and memory usage of a registered service.

type TopicHandler Uses

type TopicHandler interface {
    // Callback invoked whenever an event is published to the topic subscribed to
    // by this particular handler.
    HandleEvent(event []byte)
}

Callback interface for processing events from a single subscribed topic.

type TopicLimits Uses

type TopicLimits struct {
    EventThreads int // Event handlers to execute concurrently
    EventMemory  int // Memory allowance for pending events
}

User limits of the threading and memory usage of a subscription.

type Tunnel Uses

type Tunnel struct {
    Log log15.Logger // Logger with connection and tunnel ids injected
    // contains filtered or unexported fields
}

Communication stream between the local application and a remote endpoint. The ordered delivery of messages is guaranteed and the message flow between the peers is throttled.

func (*Tunnel) Close Uses

func (t *Tunnel) Close() error

Closes the tunnel between the pair. Any blocked read and write operation will terminate with a failure.

The method blocks until the local relay node acknowledges the tear-down.

func (*Tunnel) Recv Uses

func (t *Tunnel) Recv(timeout time.Duration) ([]byte, error)

Retrieves a message from the tunnel, blocking until one is available or the operation times out.

Infinite blocking is supported with by setting the timeout to zero (0).

func (*Tunnel) Send Uses

func (t *Tunnel) Send(message []byte, timeout time.Duration) error

Sends a message over the tunnel to the remote pair, blocking until the local Iris node receives the message or the operation times out.

Infinite blocking is supported with by setting the timeout to zero (0).

Package iris imports 12 packages (graph) and is imported by 8 packages. Updated 2016-07-14. Refresh now. Tools for package owners.