Documentation
¶
Overview ¶
Package collector provides the basics of a NEL-compliant report collector.
Index ¶
- Variables
- func DecodeRawReports(b []byte, reports *[]NelReport) error
- func EncodeRawBatch(batch *ReportBatch) ([]byte, error)
- func EncodeRawReports(reports []NelReport) ([]byte, error)
- func PrintBatchAsCLF(batch *ReportBatch, w io.Writer)
- func RegisterContextReportLoaderFunc(name string, ...)
- func RegisterReportLoader(name string, loader ReportLoader)
- func RegisterReportLoaderFunc(name string, loader func(config toml.Primitive) (ReportProcessor, error))
- type Annotations
- type Clock
- type ContextReportLoaderFunc
- type HandlerCloser
- type HotSwap
- type NelReport
- type Pipeline
- func (p *Pipeline) AddProcessor(processor ReportProcessor)
- func (p *Pipeline) Close()
- func (p *Pipeline) LoadFromConfig(ctx context.Context, configBytes []byte) error
- func (p *Pipeline) ProcessReports(ctx context.Context, w http.ResponseWriter, r *http.Request) error
- func (p *Pipeline) ServeHTTP(w http.ResponseWriter, r *http.Request)
- type ReportBatch
- type ReportLoader
- type ReportLoaderFunc
- type ReportProcessor
Constants ¶
This section is empty.
Variables ¶
var ErrDropped = errors.New("queue full, report dropped")
ErrDropped is returned from ProcessReports when the queue is full and the report is dropped.
Functions ¶
func DecodeRawReports ¶
DecodeRawReports unmarshals an array of NelReports without using our custom spec-aware JSON parsing rules. It's the inverse of EncodeRawReports.
func EncodeRawBatch ¶
func EncodeRawBatch(batch *ReportBatch) ([]byte, error)
EncodeRawBatch marshals a batch of NelReports, including any custom annotations, without using our custom spec-aware JSON parsing rules.
func EncodeRawReports ¶
EncodeRawReports marshals an array of NelReports without using our custom spec-aware JSON parsing rules, instead dumping out the content exactly as it looks in Go. This is used extensively in test cases to compare the results of parsing and annotating against golden files.
func PrintBatchAsCLF ¶
func PrintBatchAsCLF(batch *ReportBatch, w io.Writer)
PrintBatchAsCLF prints out a summary of each report in the batch using a format not unlike the format of an Apache access.log file.
func RegisterContextReportLoaderFunc ¶
func RegisterContextReportLoaderFunc(name string, loader func(ctx context.Context, config toml.Primitive) (ReportProcessor, error))
RegisterContextReportLoaderFunc registers a context-aware function that can load a particular kind of report processor.
func RegisterReportLoader ¶
func RegisterReportLoader(name string, loader ReportLoader)
RegisterReportLoader registers a ReportLoader for a particular kind of report processor.
func RegisterReportLoaderFunc ¶
func RegisterReportLoaderFunc(name string, loader func(config toml.Primitive) (ReportProcessor, error))
RegisterReportLoaderFunc registers a function that can load a particular kind of report processor.
Types ¶
type Annotations ¶
type Annotations struct {
Annotations map[string]interface{}
}
Annotations lets you attach an arbitrary collection of extra data to each individual report, and to each report batch. Each annotation has a name and an arbitrary type; it's up to you to make sure that your processors don't make conflicting assumptions about the type of an annotation with a particular name.
func (*Annotations) AnnotationWriter ¶
func (a *Annotations) AnnotationWriter(name string) io.Writer
AnnotationWriter returns an io.Writer that can be used to build up the content of a []byte annotation.
func (*Annotations) GetAnnotation ¶
func (a *Annotations) GetAnnotation(name string) interface{}
GetAnnotation returns the annotation with the given name, or nil if there isn't one.
func (*Annotations) GetOrAddAnnotation ¶
func (a *Annotations) GetOrAddAnnotation(name string, defaultValue interface{}) interface{}
GetOrAddAnnotation returns the annotation with the given name, if it exists. If it doesn't, then we save `defaultValue` as the new value for this annotation, and return it.
func (*Annotations) SetAnnotation ¶
func (a *Annotations) SetAnnotation(name string, value interface{})
SetAnnotation adds an annotation, overwriting any existing annotation with the same name.
type Clock ¶
Clock lets you override how a pipeline assigns timestamps to each report. The default is to use time.Now; you can provide a custom implementation to get reproducible timestamps in test cases.
type ContextReportLoaderFunc ¶
type ContextReportLoaderFunc func(ctx context.Context, config toml.Primitive) (ReportProcessor, error)
ContextReportLoaderFunc allows you to register a simple function (which needs access to a Context) as a ReportLoader.
func (ContextReportLoaderFunc) Load ¶
func (f ContextReportLoaderFunc) Load(ctx context.Context, config toml.Primitive) (ReportProcessor, error)
Load defers to a ContextReportLoaderFunc to load a ReportProcessor from the contents of a configuration file.
type HandlerCloser ¶
HandlerCloser is an interface for a http.Handler that processes data asynchronously and therefore must be closed. Once Close is called, the caller must ensure that no further calls to ServeHTTP are made.
type HotSwap ¶
type HotSwap struct {
// contains filtered or unexported fields
}
A HotSwap wraps a HandlerCloser and adds the support for a new handler to swapped in the middle of execution with no interruption to processing. In this manner, an external listener can load a new configuration, create a new Handler, and call Swap. In a threadsafe manner, the old Handler will be replaced and all future calls to ServeHTTP will use the new version of the handler.
func (*HotSwap) Close ¶
func (h *HotSwap) Close()
Close closes the contained HandlerCloser, first ensuring that all in-progress calls to ServeHTTP have completed. It is up to the caller to ensure that no calls to ServeHTTP are made after this function has started.
func (*HotSwap) ServeHTTP ¶
func (h *HotSwap) ServeHTTP(w http.ResponseWriter, r *http.Request)
ServeHTTP delegates incoming requests to the contained handler in a thread safe manner.
func (*HotSwap) Swap ¶
func (h *HotSwap) Swap(hc HandlerCloser)
Swap takes a new Pipeline and atomicly exchanges a new handler for all future processing. It ensures that all active calls to ServeHTTP are done and then closes the old Pipeline
type NelReport ¶
type NelReport struct {
// The number of milliseconds between when the report was generated by
// the user agent and when it was uploaded.
Age int
// The type of report. For NEL, this will be "network-error".
ReportType string
// The URL of the request that this report describes.
URL string
// UserAgent represents the value of the User-Agent header in the request that the report is about
UserAgent string
// The referrer information of the request, as determined by the
// referrer policy associated with its client.
Referrer string
// The active sampling rate for this request, expressed as a fraction
// between 0.0 and 1.0 (inclusive).
SamplingFraction float32
// The IP address of the host to which the user agent set the request.
ServerIP string
// The ALPN ID of the network protocol used to fetch the resource.
Protocol string
// The method of the HTTP request (e.g. GET, POST)
Method string
// The status code of the HTTP response, if available.
StatusCode int
// The elapsed number of milliseconds between the start of the resource
// fetch and when it was aborted by the user agent.
ElapsedTime int
// The phase of the request in which the failure occurred. One of
// {dns, connection, application}; a successful request always has a value of application
Phase string
// The description of the error type. For reports about successful
// requests, this will be "ok". See the NEL spec for the authoritative
// list of possible values for failed requests.
Type string
// For non-NEL reports, this will contain the unparsed JSON content of
// the report's `body` field.
RawBody []byte
// An arbitrary set of extra data that you can attach to your reports.
Annotations
}
A NelReport describes a single network error report.
(The name is a bit of a misnomer; it can also represent other report payloads uploaded via the Reporting API, in which case ReportType will tell you what kind of report it is, and RawBody will contain the unparsed JSON `body` field. It can also represent information about successful HTTP requests, collected and delivered via a user agent's NEL stack, in which case the Type field will be "ok".)
func (NelReport) MarshalJSON ¶
MarshalJSON marshals a NEL report into a JSON payload as defined by the Reporting and NEL specs. (It correctly handles the nested structure of the JSON, extracting the fields of the non-nested NelReport type.)
func (*NelReport) UnmarshalJSON ¶
UnmarshalJSON unmarshals the JSON payload as defined by the Reporting and NEL specs into a NelReport object. (It correctly handles the nested structure of the JSON, filling in the fields of the non-nested NelReport type.)
type Pipeline ¶
type Pipeline struct {
// contains filtered or unexported fields
}
Pipeline is a series of processors that should be applied to each report that the collector receives. It uses a fixed number of workers to process the reports and a fixed sized queue that the workers read from. If the queue fills, reports are dropped. Pipeline{} is not a usable instance, use NewPipeline for production and NewTestPipeline* in tests.
func NewPipeline ¶
NewPipeline creates a new Pipeline with a specified buffer size and number of workers.
func NewTestPipeline ¶
NewTestPipeline creates a new Pipeline with a specified clock. This should only be used for testing.
func NewTestPipelineWithBuffer ¶
NewTestPipelineWithBuffer creates a new Pipeline with a specified buffer size and clock. This should only be used for testing.
func (*Pipeline) AddProcessor ¶
func (p *Pipeline) AddProcessor(processor ReportProcessor)
AddProcessor adds a new processor to the pipeline.
func (*Pipeline) Close ¶
func (p *Pipeline) Close()
Close stops the processing, such that anything in the queue gets processed, but nothing is added. It then waits until all processing workers have completed. All calls to ProcessReports must complete before Close is called, otherwise it will cause a panic.
func (*Pipeline) LoadFromConfig ¶
LoadFromConfig loads pipeline processors based on the contents of a TOML configuration file, and adds them to the pipeline.
The configuration must have sections named `processor`, each of which defines one processor that should be added to the pipeline. For instance, this configuration could look like:
[[processor]] type = "KeepNelReports" [[processor]] type = "DumpReportsAsCLF" use_utc = true
The `type` field of each element identifies which kind of processor to add; any additional fields let you specify any processor-specific configuration.
func (*Pipeline) ProcessReports ¶
func (p *Pipeline) ProcessReports(ctx context.Context, w http.ResponseWriter, r *http.Request) error
ProcessReports extracts reports from a POST upload payload, as defined by the Reporting spec, and runs all of the processors in the pipeline against each report. Returns ErrDropped if the request was dropped due to a full queue and nil on success. All other errors indicate something wrong with the request.
type ReportBatch ¶
type ReportBatch struct {
Reports []NelReport
// When this batch was received by the collector
Time time.Time
// The URL that was used to upload the report.
CollectorURL url.URL
// The IP address of the client that uploaded the batch of reports. You can
// typically assume that's the same IP address that was used for the original
// requests. The IP address will be encoded as a string; for example,
// "192.0.2.1" or "2001:db8::2".
ClientIP string
// The user agent of the client that uploaded the batch of reports.
ClientUserAgent string
// The key-value pairs of the HTTP header that is received by the collector.
// This can be used to get additional information. One example is to get the
// remote address of the client when the collector runs behind a proxy.
Header http.Header
// An arbitrary set of extra data that you can attach to this batch of
// reports.
Annotations
}
ReportBatch is a collection of reports that should all be processed together. We will create a new batch for each upload that the collector receives. Certain processors might join batches together or split them up.
func NewReportBatch ¶
func NewReportBatch(r *http.Request, clock Clock) (*ReportBatch, error)
NewReportBatch takes a HTTP request and a clock and fills in a ReportBatch, returning an error if parsing fails.
type ReportLoader ¶
type ReportLoader interface {
Load(ctx context.Context, config toml.Primitive) (ReportProcessor, error)
}
ReportLoader is an interface that knows how to load a ReportProcessor at runtime via the contents of a TOML configuration file.
type ReportLoaderFunc ¶
type ReportLoaderFunc func(config toml.Primitive) (ReportProcessor, error)
ReportLoaderFunc allows you to register a simple function as a ReportLoader.
func (ReportLoaderFunc) Load ¶
func (f ReportLoaderFunc) Load(ctx context.Context, config toml.Primitive) (ReportProcessor, error)
Load defers to a ReportLoaderFunc to load a ReportProcessor from the contents of a configuration file.
type ReportProcessor ¶
type ReportProcessor interface {
// ProcessReports handles a single batch of reports. You have full control
// over the contents of the batch; for instance, you can remove elements or
// update their contents, if appropriate.
ProcessReports(ctx context.Context, batch *ReportBatch)
}
A ReportProcessor implements one discrete processing step for handling uploaded reports. There are several predefined processors, which you can use to filter or publish reports. You can also implement custom annotation steps if you want to add additional data to a report before publishing.
Source Files