checkup: github.com/sourcegraph/checkup Index | Files | Directories

package checkup

import "github.com/sourcegraph/checkup"

Package checkup provides means for checking and reporting the status and performance of various endpoints in a distributed, lock-free, self-hosted fashion.

Index

Package Files

checkup.go dnschecker.go fs.go github.go httpchecker.go logger.go s3.go slack.go sql.go tcpchecker.go tlschecker.go

Constants

const FilenameFormatString = "%d-check.json"

FilenameFormatString is the format string used by GenerateFilename to create a filename.

Variables

var DefaultConcurrentChecks = 5

DefaultConcurrentChecks is how many checks, at most, to perform concurrently.

var DefaultHTTPClient = &http.Client{
    Transport: &http.Transport{
        Proxy: http.ProxyFromEnvironment,
        Dial: (&net.Dialer{
            Timeout:   10 * time.Second,
            KeepAlive: 0,
        }).Dial,
        TLSHandshakeTimeout:   5 * time.Second,
        ExpectContinueTimeout: 1 * time.Second,
        MaxIdleConnsPerHost:   1,
        DisableCompression:    true,
        DisableKeepAlives:     true,
        ResponseHeaderTimeout: 5 * time.Second,
    },
    CheckRedirect: func(req *http.Request, via []*http.Request) error {
        return http.ErrUseLastResponse
    },
    Timeout: 10 * time.Second,
}

DefaultHTTPClient is used when no other http.Client is specified on a HTTPChecker.

func DisableColor Uses

func DisableColor()

DisableColor disables ANSI colors in the Result default string.

func GenerateFilename Uses

func GenerateFilename() *string

GenerateFilename returns a filename that is ideal for storing the results file on a storage provider that relies on the filename for retrieval that is sorted by date/timeframe. It returns a string pointer to be used by the AWS SDK...

func Timestamp Uses

func Timestamp() int64

Timestamp returns the UTC Unix timestamp in nanoseconds.

type Attempt Uses

type Attempt struct {
    RTT   time.Duration `json:"rtt"`
    Error string        `json:"error,omitempty"`
}

Attempt is an attempt to communicate with the endpoint.

type Attempts Uses

type Attempts []Attempt

Attempts is a list of Attempt that can be sorted by RTT.

func (Attempts) Len Uses

func (a Attempts) Len() int

func (Attempts) Less Uses

func (a Attempts) Less(i, j int) bool

func (Attempts) Swap Uses

func (a Attempts) Swap(i, j int)

type Checker Uses

type Checker interface {
    Check() (Result, error)
}

Checker can create a Result.

type Checkup Uses

type Checkup struct {
    // Checkers is the list of Checkers to use with
    // which to perform checks.
    Checkers []Checker `json:"checkers,omitempty"`

    // ConcurrentChecks is how many checks, at most, to
    // perform concurrently. Default is
    // DefaultConcurrentChecks.
    ConcurrentChecks int `json:"concurrent_checks,omitempty"`

    // Timestamp is the timestamp to force for all checks.
    // Useful if wanting to perform distributed check
    // "at the same time" even if they might actually
    // be a few milliseconds or seconds apart.
    Timestamp time.Time `json:"timestamp,omitempty"`

    // Storage is the storage mechanism for saving the
    // results of checks. Required if calling Store().
    // If Storage is also a Maintainer, its Maintain()
    // method will be called by c.CheckAndStore().
    Storage Storage `json:"storage,omitempty"`

    // Notifier is a notifier that will be passed the
    // results after checks from all checkers have
    // completed. Notifier may evaluate and choose to
    // send a notification of potential problems.
    Notifier Notifier `json:"notifier,omitempty"`
}

Checkup performs a routine checkup on endpoints or services.

func (Checkup) Check Uses

func (c Checkup) Check() ([]Result, error)

Check performs the health checks. An error is only returned in the case of a misconfiguration or if any one of the Checkers returns an error.

func (Checkup) CheckAndStore Uses

func (c Checkup) CheckAndStore() error

CheckAndStore performs health checks and immediately stores the results to the configured storage if there were no errors. Checks are not performed if c.Storage is nil. If c.Storage is also a Maintainer, Maintain() will be called if Store() is successful.

func (Checkup) CheckAndStoreEvery Uses

func (c Checkup) CheckAndStoreEvery(interval time.Duration) *time.Ticker

CheckAndStoreEvery calls CheckAndStore every interval. It returns the ticker that it's using so you can stop it when you don't want it to run anymore. This function does NOT block (it runs the ticker in a goroutine). Any errors are written to the standard logger. It would not be wise to set an interval lower than the time it takes to perform the checks.

func (Checkup) MarshalJSON Uses

func (c Checkup) MarshalJSON() ([]byte, error)

MarshalJSON marshals c into JSON with type information included on the interface values.

func (*Checkup) UnmarshalJSON Uses

func (c *Checkup) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshales b into c. To succeed, it requires type information for the interface values.

type DNSChecker Uses

type DNSChecker struct {
    // Name is the name of the endpoint.
    Name string `json:"endpoint_name"`
    // This is the name of the DNS server you are testing.
    URL string `json:"endpoint_url"`
    // This is the fqdn of the target server to query the DNS server for.
    Host string `json:"hostname_fqdn,omitempty"`
    // Timeout is the maximum time to wait for a
    // TCP connection to be established.
    Timeout time.Duration `json:"timeout,omitempty"`
    // ThresholdRTT is the maximum round trip time to
    // allow for a healthy endpoint. If non-zero and a
    // request takes longer than ThresholdRTT, the
    // endpoint will be considered unhealthy. Note that
    // this duration includes any in-between network
    // latency.
    ThresholdRTT time.Duration `json:"threshold_rtt,omitempty"`
    // Attempts is how many requests the client will
    // make to the endpoint in a single check.
    Attempts int `json:"attempts,omitempty"`
}

DNSChecker implements a Checker for TCP endpoints.

func (DNSChecker) Check Uses

func (c DNSChecker) Check() (Result, error)

Check performs checks using c according to its configuration. An error is only returned if there is a configuration error.

type Errors Uses

type Errors []error

Errors is an error type that concatenates multiple errors.

func (Errors) Empty Uses

func (e Errors) Empty() bool

Empty returns whether e has any non-nil errors in it.

func (Errors) Error Uses

func (e Errors) Error() string

Error returns a string containing all the errors in e.

type FS Uses

type FS struct {
    // The path to the directory where check files will be stored.
    Dir string `json:"dir"`
    // The URL corresponding to fs.Dir.
    URL string `json:"url"`

    // Check files older than CheckExpiry will be
    // deleted on calls to Maintain(). If this is
    // the zero value, no old check files will be
    // deleted.
    CheckExpiry time.Duration `json:"check_expiry,omitempty"`
}

FS is a way to store checkup results on the local filesystem.

func (FS) Fetch Uses

func (fs FS) Fetch(name string) ([]Result, error)

Fetch fetches results from filesystem for the specified index.

func (FS) GetIndex Uses

func (fs FS) GetIndex() (map[string]int64, error)

GetIndex returns the index from filesystem.

func (FS) Maintain Uses

func (fs FS) Maintain() error

Maintain deletes check files that are older than fs.CheckExpiry.

func (FS) Store Uses

func (fs FS) Store(results []Result) error

Store stores results on filesystem according to the configuration in fs.

type GitHub Uses

type GitHub struct {
    // AccessToken is the API token used to authenticate with GitHub (required).
    AccessToken string `json:"access_token"`

    // RepositoryOwner is the account which owns the repository on GitHub (required).
    // For https://github.com/octocat/kit, the owner is "octocat".
    RepositoryOwner string `json:"repository_owner"`

    // RepositoryName is the name of the repository on GitHub (required).
    // For https://github.com/octocat/kit, the name is "kit".
    RepositoryName string `json:"repository_name"`

    // CommitterName is the display name of the user corresponding to the AccessToken (required).
    // If the AccessToken is for user @octocat, then this might be "Mona Lisa," her name.
    CommitterName string `json:"committer_name"`

    // CommitterEmail is the email address of the user corresponding to the AccessToken (required).
    // If the AccessToken is for user @octocat, then this might be "mona@github.com".
    CommitterEmail string `json:"committer_email"`

    // Branch is the git branch to store the files to (required).
    Branch string `json:"branch"`

    // Dir is the subdirectory in the Git tree in which to store the files (required).
    // For example, to write to the directory "updates" in the Git repo, this should be "updates".
    Dir string `json:"dir"`

    // Check files older than CheckExpiry will be
    // deleted on calls to Maintain(). If this is
    // the zero value, no old check files will be
    // deleted.
    CheckExpiry time.Duration `json:"check_expiry,omitempty"`
    // contains filtered or unexported fields
}

GitHub is a way to store checkup results in a GitHub repository.

func (*GitHub) Fetch Uses

func (gh *GitHub) Fetch(name string) ([]Result, error)

Fetch returns a checkup record -- Not tested!

func (*GitHub) GetIndex Uses

func (gh *GitHub) GetIndex() (map[string]int64, error)

GetIndex returns the checkup index

func (*GitHub) Maintain Uses

func (gh *GitHub) Maintain() error

Maintain deletes check files that are older than gh.CheckExpiry.

func (*GitHub) Store Uses

func (gh *GitHub) Store(results []Result) error

Store stores results in the Git repo & updates the index.

type HTTPChecker Uses

type HTTPChecker struct {
    // Name is the name of the endpoint.
    Name string `json:"endpoint_name"`

    // URL is the URL of the endpoint.
    URL string `json:"endpoint_url"`

    // UpStatus is the HTTP status code expected by
    // a healthy endpoint. Default is http.StatusOK.
    UpStatus int `json:"up_status,omitempty"`

    // ThresholdRTT is the maximum round trip time to
    // allow for a healthy endpoint. If non-zero and a
    // request takes longer than ThresholdRTT, the
    // endpoint will be considered unhealthy. Note that
    // this duration includes any in-between network
    // latency.
    ThresholdRTT time.Duration `json:"threshold_rtt,omitempty"`

    // MustContain is a string that the response body
    // must contain in order to be considered up.
    // NOTE: If set, the entire response body will
    // be consumed, which has the potential of using
    // lots of memory and slowing down checks if the
    // response body is large.
    MustContain string `json:"must_contain,omitempty"`

    // MustNotContain is a string that the response
    // body must NOT contain in order to be considered
    // up. If both MustContain and MustNotContain are
    // set, they are and-ed together. NOTE: If set,
    // the entire response body will be consumed, which
    // has the potential of using lots of memory and
    // slowing down checks if the response body is large.
    MustNotContain string `json:"must_not_contain,omitempty"`

    // Attempts is how many requests the client will
    // make to the endpoint in a single check.
    Attempts int `json:"attempts,omitempty"`

    // AttemptSpacing spaces out each attempt in a check
    // by this duration to avoid hitting a remote too
    // quickly in succession. By default, no waiting
    // occurs between attempts.
    AttemptSpacing time.Duration `json:"attempt_spacing,omitempty"`

    // Client is the http.Client with which to make
    // requests. If not set, DefaultHTTPClient is
    // used.
    Client *http.Client `json:"-"`

    // Headers contains headers to added to the request
    // that is sent for the check
    Headers http.Header `json:"headers,omitempty"`
}

HTTPChecker implements a Checker for HTTP endpoints.

func (HTTPChecker) Check Uses

func (c HTTPChecker) Check() (Result, error)

Check performs checks using c according to its configuration. An error is only returned if there is a configuration error.

type Maintainer Uses

type Maintainer interface {
    Maintain() error
}

Maintainer can maintain a store of results by deleting old check files that are no longer needed or performing other required tasks.

type Notifier Uses

type Notifier interface {
    Notify([]Result) error
}

Notifier can notify ops or sysadmins of potential problems. A Notifier should keep state to avoid sending repeated notices more often than the admin would like.

type ProvisionInfo Uses

type ProvisionInfo struct {
    // The ID of a user that was created for accessing checks.
    UserID string `json:"user_id"`

    // The username of a user that was created for accessing checks.
    Username string `json:"username"`

    // The ID or name of the ID/key used to access checks. Expect
    // this value to be made public. (It should have read-only
    // access to the checks.)
    PublicAccessKeyID string `json:"public_access_key_id"`

    // The "secret" associated with the PublicAccessKeyID, but
    // expect this value to be made public. (It should provide
    // read-only access to the checks.)
    PublicAccessKey string `json:"public_access_key"`
}

ProvisionInfo contains the results of provisioning a new storage facility for check files. Its values should be used by the status page in order to obtain read-only access to the check files.

func (ProvisionInfo) String Uses

func (i ProvisionInfo) String() string

String returns the information in i in a human-readable format along with an important notice.

type Provisioner Uses

type Provisioner interface {
    Provision() (ProvisionInfo, error)
}

Provisioner is a type of storage mechanism that can provision itself for use with checkup. Provisioning need only happen once and is merely a convenience so that the user can get up and running with their status page more quickly. Presumably, the info returned from Provision should be used on the status page side of things ot access the check files (like a key pair that is used for read-only access).

type Result Uses

type Result struct {
    // Title is the title (or name) of the thing that was checked.
    // It should be unique, as it acts like an identifier to users.
    Title string `json:"title,omitempty"`

    // Endpoint is the URL/address/path/identifier/locator/whatever
    // of what was checked.
    Endpoint string `json:"endpoint,omitempty"`

    // Timestamp is when the check occurred; UTC UnixNano format.
    Timestamp int64 `json:"timestamp,omitempty"`

    // Times is a list of each individual check attempt.
    Times Attempts `json:"times,omitempty"`

    // ThresholdRTT is the maximum RTT that was tolerated before
    // considering performance to be degraded. Leave 0 if irrelevant.
    ThresholdRTT time.Duration `json:"threshold,omitempty"`

    // Healthy, Degraded, and Down contain the ultimate conclusion
    // about the endpoint. Exactly one of these should be true;
    // any more or less is a bug.
    Healthy  bool `json:"healthy,omitempty"`
    Degraded bool `json:"degraded,omitempty"`
    Down     bool `json:"down,omitempty"`

    // Notice contains a description of some condition of this
    // check that might have affected the result in some way.
    // For example, that the median RTT is above the threshold.
    Notice string `json:"notice,omitempty"`

    // Message is an optional message to show on the status page.
    // For example, what you're doing to fix a problem.
    Message string `json:"message,omitempty"`
}

Result is the result of a health check.

func (Result) ComputeStats Uses

func (r Result) ComputeStats() Stats

ComputeStats computes basic statistics about r.

func (Result) Status Uses

func (r Result) Status() StatusText

Status returns a text representation of the overall status indicated in r.

func (Result) String Uses

func (r Result) String() string

String returns a human-readable rendering of r.

type S3 Uses

type S3 struct {
    AccessKeyID     string `json:"access_key_id"`
    SecretAccessKey string `json:"secret_access_key"`
    Region          string `json:"region,omitempty"`
    Bucket          string `json:"bucket"`

    // Check files older than CheckExpiry will be
    // deleted on calls to Maintain(). If this is
    // the zero value, no old check files will be
    // deleted.
    CheckExpiry time.Duration `json:"check_expiry,omitempty"`
}

S3 is a way to store checkup results in an S3 bucket.

func (S3) Maintain Uses

func (s S3) Maintain() error

Maintain deletes check files that are older than s.CheckExpiry.

func (S3) Provision Uses

func (s S3) Provision() (ProvisionInfo, error)

Provision creates a new IAM user in the account specified by s, and configures a bucket according to the values in s. The credentials in s must have the IAMFullAccess and AmazonS3FullAccess permissions in order to succeed.

The name of the created IAM user is "checkup-monitor-s3-public". It will have read-only permission to S3.

Provision need only be called once per status page (bucket), not once per endpoint.

func (S3) Store Uses

func (s S3) Store(results []Result) error

Store stores results on S3 according to the configuration in s.

type SQL Uses

type SQL struct {
    // SqliteDBFile is the sqlite3 DB where check results will be stored.
    SqliteDBFile string `json:"sqlite_db_file,omitempty"`

    // PostgreSQL contains the Postgres connection settings.
    PostgreSQL *struct {
        Host     string `json:"host,omitempty"`
        Port     int    `json:"port,omitempty"`
        User     string `json:"user"`
        Password string `json:"password,omitempty"`
        DBName   string `json:"dbname"`
        SSLMode  string `json:"sslmode,omitempty"`
    }   `json:"postgresql"`

    // Check files older than CheckExpiry will be
    // deleted on calls to Maintain(). If this is
    // the zero value, no old check files will be
    // deleted.
    CheckExpiry time.Duration `json:"check_expiry,omitempty"`
}

SQL is a way to store checkup results in a SQL database.

func (SQL) Fetch Uses

func (sql SQL) Fetch(name string) ([]Result, error)

Fetch fetches results of the check with given name.

func (SQL) GetIndex Uses

func (sql SQL) GetIndex() (map[string]int64, error)

GetIndex returns the list of check results for the database.

func (SQL) Maintain Uses

func (sql SQL) Maintain() error

Maintain deletes check files that are older than sql.CheckExpiry.

func (SQL) Store Uses

func (sql SQL) Store(results []Result) error

Store stores results in the database.

type Slack Uses

type Slack struct {
    Name     string `json:"name"`
    Username string `json:"username"`
    Channel  string `json:"channel"`
    Webhook  string `json:"webhook"`
}

Slack consist of all the sub components required to use Slack API

func (Slack) Notify Uses

func (s Slack) Notify(results []Result) error

Notify implements notifier interface

func (Slack) Send Uses

func (s Slack) Send(result Result) error

Send request via Slack API to create incident

type Stats Uses

type Stats struct {
    Total  time.Duration `json:"total,omitempty"`
    Mean   time.Duration `json:"mean,omitempty"`
    Median time.Duration `json:"median,omitempty"`
    Min    time.Duration `json:"min,omitempty"`
    Max    time.Duration `json:"max,omitempty"`
}

Stats is a type that holds information about a Result, especially its various Attempts.

type StatusText Uses

type StatusText string

StatusText is the textual representation of the result of a status check.

const (
    Healthy  StatusText = "healthy"
    Degraded StatusText = "degraded"
    Down     StatusText = "down"
    Unknown  StatusText = "unknown"
)

Text representations for the status of a check.

func (StatusText) PriorityOver Uses

func (s StatusText) PriorityOver(other StatusText) bool

PriorityOver returns whether s has priority over other. For example, a Down status has priority over Degraded.

type Storage Uses

type Storage interface {
    Store([]Result) error
}

Storage can store results.

type StorageReader Uses

type StorageReader interface {
    // Fetch returns the contents of a check file.
    Fetch(checkFile string) ([]Result, error)
    // GetIndex returns the storage index, as a map where keys are check
    // result filenames and values are the associated check timestamps.
    GetIndex() (map[string]int64, error)
}

StorageReader can read results from the Storage.

type TCPChecker Uses

type TCPChecker struct {
    // Name is the name of the endpoint.
    Name string `json:"endpoint_name"`

    // URL is the URL of the endpoint.
    URL string `json:"endpoint_url"`

    // TLSEnabled controls whether to enable TLS or not.
    // If set, TLS is enabled.
    TLSEnabled bool `json:"tls,omitempty"`

    // TLSSkipVerify controls whether to skip server TLS
    // certificat validation or not.
    TLSSkipVerify bool `json:"tls_skip_verify,omitempty"`

    // TLSCAFile is the Certificate Authority used
    // to validate the server TLS certificate.
    TLSCAFile string `json:"tls_ca_file,omitempty"`

    // Timeout is the maximum time to wait for a
    // TCP connection to be established.
    Timeout time.Duration `json:"timeout,omitempty"`

    // ThresholdRTT is the maximum round trip time to
    // allow for a healthy endpoint. If non-zero and a
    // request takes longer than ThresholdRTT, the
    // endpoint will be considered unhealthy. Note that
    // this duration includes any in-between network
    // latency.
    ThresholdRTT time.Duration `json:"threshold_rtt,omitempty"`

    // Attempts is how many requests the client will
    // make to the endpoint in a single check.
    Attempts int `json:"attempts,omitempty"`
}

TCPChecker implements a Checker for TCP endpoints.

func (TCPChecker) Check Uses

func (c TCPChecker) Check() (Result, error)

Check performs checks using c according to its configuration. An error is only returned if there is a configuration error.

type TLSChecker Uses

type TLSChecker struct {
    // Name is the name of the endpoint.
    Name string `json:"endpoint_name"`

    // URL is the host:port of the remote endpoint to check.
    URL string `json:"endpoint_url"`

    // Timeout is the maximum time to wait for a
    // TLS connection to be established.
    Timeout time.Duration `json:"timeout,omitempty"`

    // ThresholdRTT is the maximum round trip time to
    // allow for a healthy endpoint. If non-zero and a
    // request takes longer than ThresholdRTT, the
    // endpoint will be considered unhealthy. Note that
    // this duration includes any in-between network
    // latency.
    ThresholdRTT time.Duration `json:"threshold_rtt,omitempty"`

    // Attempts is how many requests the client will
    // make to the endpoint in a single check.
    Attempts int `json:"attempts,omitempty"`

    // CertExpiryThreshold is how close to expiration
    // the TLS certificate must be before declaring
    // a degraded status. Default is 14 days.
    CertExpiryThreshold time.Duration `json:"cert_expiry_threshold,omitempty"`

    // TrustedRoots is a list of PEM files to load as
    // trusted root CAs when connecting to TLS remotes.
    TrustedRoots []string `json:"trusted_roots,omitempty"`
    // contains filtered or unexported fields
}

TLSChecker implements a Checker for TLS endpoints.

TODO: Implement more checks on the certificate and TLS configuration.

- Cipher suites
- Protocol versions
- OCSP stapling
- Multiple SNIs
- Other things that you might see at SSL Labs or other TLS health checks

func (TLSChecker) Check Uses

func (c TLSChecker) Check() (Result, error)

Check performs checks using c according to its configuration. An error is only returned if there is a configuration error.

Directories

PathSynopsis
cmd
cmd/checkup

Package checkup imports 32 packages (graph) and is imported by 11 packages. Updated 2018-04-11. Refresh now. Tools for package owners.