README
¶
Detective 🔎
Detective is a distributed application health monitoring library. It allows you to monitor arbitrary dependencies in your application, and compose other detective instances to create a distributed monitoring framework.
Usage
A typical service oriented architecture looks like this:
Detective allows you to enable each application to monitor its own dependencies, including dependencies with contain another detective instance. By doing so, you can monitor your infrastructure in a distributed manner, where each service only monitors it's own dependencies.
Monitoring a single application
Detective exposes a straightforward API to monitor an arbitrary dependency:
// Initialize a new detective instance
d := detective.New("Another application")
// Create a dependency, and register its detector function
d.Dependency("cache").Detect(func() error {
err := cache.Ping()
return err
})
// Create an HTTP endpoint for health checks
http.ListenAndServe(":8081", d)
See the "regular usage" example
The HTTP endpoint can then be used to monitor the health of the application. A GET request to http://localhost:8081/ will return information on the health of the overall application:
{
"name": "Another application",
"active": true,
"status": "Ok",
"latency": 0,
"dependencies": [
{
"name": "cache",
"active": true,
"status": "Ok",
"latency": 500848512
}
]
}
Composing instances
The endpoint in the previous example can also be used by other detective instances. For example, an application that makes use of "Another application" can monitor it as well:
// Create a new detective instance
d := detective.New("your application")
// Add a dependency, and register a function to detect a fault
d.Dependency("database").Detect(func() error {
// `db` can be an instance of sql.DB
err := db.Ping()
return err
})
// Similarly for the cache
d.Dependency("cache").Detect(func() error {
err := cache.Ping()
return err
})
// Add an endpoint, which represents another detective instance ("Another application" in this case)
d.Endpoint("http://localhost:8081/")
// Create a ping endpoint which checks the health of all dependencies
// A Detective instance implements the http.Handler interface
http.ListenAndServe(":8080", d)
See the "composing detective instances" example
Now, when we hit GET http://localhost:8080/, its detective instance will monitor its own dependencies as usual, but also hit the previous dependencies endpoint, and as a result monitor it's dependencies as well :
{
"name": "your application",
"active": true,
"status": "Ok",
"latency": 0,
"dependencies": [
{
"name": "Another application",
"active": true,
"status": "Ok",
"latency": 0,
"dependencies": [
{
"name": "cache",
"active": true,
"status": "Ok",
"latency": 502210954
}
]
},
{
"name": "db",
"active": true,
"status": "Ok",
"latency": 2500328773
},
{
"name": "db",
"active": true,
"status": "Ok",
"latency": 2500248450
}
]
}
Circular dependencies
It's possible for two applications to depend on each other, either directly, or indirectly. Normally, if you registered two detective instances as dependents of each other, it would result in an infinite loop of HTTP calls to each others ping handler. Detective protects against this situation by adding information about a calling instance to the HTTP header of its request. The callee then inspects this header to find out if it was already part of the calling chain, in which case it ceases to send endpoint HTTP requests, and breaks the circular dependency chain.
Dashboard
The dashboard helps visualize your dependency tree and detect any faulty dependencies, along with their latency:
To run the dashboard, install the binary with:
go get github.com/sohamkamani/detective/detective-dashboard
go install github.com/sohamkamani/detective/detective-dashboard
Then start the dashboard with:
detective-dashboard -p 8080
## Starts dashboard on http://localhost:8080/
You will then have to enter the URL of any detective endpoint to view its dashboard.
Examples
API documentation
Detailed API documentation can be found on the [GoDocs page]((https://godoc.org/github.com/sohamkamani/detective)
Other implementations
The detective library has also been implemented in Node.js : https://github.com/sohamkamani/detective-node, is iteroperable with this implementation, and can be used as an endpoint.
Documentation
¶
Overview ¶
Detective is a distributed application health monitoring library.
Detective allows you to simultaneously, and independently monitor the health of any application along with its dependencies.
Using Detective ¶
A Detective instance represents a group of dependencies. A dependency can be anything that the application depends on, and which can return some sort of error. For example, a database whose connection time out, or an HTTP web service that doesn't respond.
// Create a new detective instance, and add a dependency
d := detective.New("application")
dep := d.Dependency("database")
// Register a function to detect a fault in the dependency
dep.Detect(func() error {
// `db` can be an instance of sql.DB
err := db.Ping()
return err
})
// Create a ping endpoint which checks the health of all dependencies
// A Detective instance implements the http.Handler interface
http.ListenAndServe(":8080", d)
The ping endpoint will check the health of all dependencies by calling their detector function registered in the "Detect" method
Composing instances ¶
The endpoint that was defined in the previous example can itself be registered in other detective instances:
d2 := detective.New("dependent_application")
// The ping handler defined in the last example can be registered as an endpoint
d2.Endpoint("http://localhost:8080/")
// The ping handler of `d2` will now call `d`s ping handler and as a result, monitor `d`s dependencies as well
http.ListenAndServe(":8081", d2)
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Dependency ¶
type Dependency struct {
// contains filtered or unexported fields
}
The Dependency type represents a detectable unit. The function provided in the Detect method will be called to monitor the state of the dependency
func (*Dependency) Detect ¶
func (d *Dependency) Detect(df DetectorFunc)
Detect registers a function that will be called to detect the health of a dependency. If the dependency is healthy, a nil value should be returned as the error.
type Detective ¶
type Detective struct {
// contains filtered or unexported fields
}
A Detective instance manages registered dependencies and endpoints. Dependencies can be registered with an instance. Each instance has a state which represents the health of its components.
func New ¶
New creates a new Detective instance. To avoid confusion, the name provided should preferably be unique among dependent detective instances.
func (*Detective) Dependency ¶
func (d *Detective) Dependency(name string) *Dependency
Dependency adds a new dependency to the Detective instance. The name provided should preferably be unique among dependencies registered within the same detective instance.
func (*Detective) Endpoint ¶
Endpoint adds an HTTP endpoint as a dependency to the Detective instance, thereby allowing you to compose detective instances. This method creates a GET request to the provided url. If you want to customize the request (like using a different HTTP method, or adding headers), consider using the EndpointReq method instead.
func (*Detective) EndpointReq ¶
EndpointReq is similar to Endpoint, but takes an HTTP request object instead of a URL. Use this method if you want to customize the request to the ping handler of another detective instance.
func (*Detective) ServeHTTP ¶
func (d *Detective) ServeHTTP(w http.ResponseWriter, r *http.Request)
ServeHTTP is the HTTP handler function for getting the state of the Detective instance
func (*Detective) WithHTTPClient ¶
WithHTTPClient sets the HTTP Client to be used while hitting the endpoint of another detective HTTP ping handler.
type DetectorFunc ¶
type DetectorFunc func() error
The DetectorFunc type represents the function signature to check the health of a dependency
type State ¶
type State struct {
Name string `json:"name"`
Ok bool `json:"active"`
Status string `json:"status"`
Latency time.Duration `json:"latency"`
Dependencies []State `json:"dependencies,omitempty"`
}
State describes the current status of an entity. This entity can be a Dependency, or a Detective instance. A State can contain other States as well.
Source Files
Directories