beater

package
v5.2.1+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 9, 2017 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package beater provides the implementation of the libbeat Beater interface for Metricbeat and functions for running Metricbeat Modules on their own.

Metricbeat collects metrics from operating systems and services. The code for gathering metrics from a particular service is organized into a logical grouping called a Module. Each Module has one or more MetricSet implementations which do the work of collecting a particular set of metrics from the service.

The public interfaces used in implementing Modules and MetricSets are defined in the github.com/elastic/beats/metricbeat/mb package.

Event Format

Each event generated by Metricbeat has the same general structure. The example event below was generated by a MetricSet named "cpu" in the "system" Module.

{
	"@timestamp": "2016-05-23T08:05:34.853Z",
	"beat": {
		"hostname": "host.example.com",
		"name": "host.example.com"
	},
	"metricset": {
		"host": "localhost",
		"module": "system",
		"name": "cpu",
		"rtt": 115
	},
	"system": {
		"cpu": {
			"idle": {
				"pct": 0.852,
				"ticks": 44421033
			},
			"iowait": {
				"pct": 0,
				"ticks": 159735
			},
			"irq": {
				"pct": 0,
				"ticks": 0
			},
			"nice": {
				"pct": 0,
				"ticks": 0
			},
			"softirq": {
				"pct": 0,
				"ticks": 14070
			},
			"steal": {
				"pct": 0,
				"ticks": 0
			},
			"system": {
				"pct": 0.0408,
				"ticks": 305704
			},
			"user": {
				"pct": 0.1071,
				"ticks": 841974
			}
		}
	},
	"type": "metricsets"
}

All events are stored in one index called metricbeat by default. Each MetricSet's data format is potentially unique so the MetricSet data is added to event as a dictionary under a key that is unique to the MetricSet. The key is constructed from the Module name and MetricSet name to ensure uniqueness. All documents are stored under the same type called "metricsets".

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func New

func New(b *beat.Beat, rawConfig *common.Config) (beat.Beater, error)

New creates and returns a new Metricbeat instance.

func PublishChannels

func PublishChannels(client publisher.Client, cs ...<-chan common.MapStr)

PublishChannels publishes the events read from each channel to the given publisher client. If the publisher client blocks for any reason then events will not be read from the given channels.

This method blocks until all of the channels have been closed and are fully read. To stop the method immediately, close the channels and close the publisher client to ensure that publishing does not block. This may result is some events being discarded.

Types

type Config

type Config struct {
	// Modules is a list of module specific configuration data.
	Modules []*common.Config `config:"modules" validate:"required"`
}

Config is the root of the Metricbeat configuration hierarchy.

type EventBuilder

type EventBuilder struct {
	ModuleName    string
	MetricSetName string
	Host          string
	StartTime     time.Time
	FetchDuration time.Duration
	Event         common.MapStr
	// contains filtered or unexported fields
}

EventBuilder is used for building MetricSet events. MetricSets generate a data in the form of a common.MapStr. This builder transforms that data into a complete event and applies any Module-level filtering.

func (EventBuilder) Build

func (b EventBuilder) Build() (common.MapStr, error)

Build builds an event from MetricSet data and applies the Module-level filters.

type Metricbeat

type Metricbeat struct {
	// contains filtered or unexported fields
}

Metricbeat implements the Beater interface for metricbeat.

func (*Metricbeat) Run

func (bt *Metricbeat) Run(b *beat.Beat) error

Run starts the workers for Metricbeat and blocks until Stop is called and the workers complete. Each host associated with a MetricSet is given its own goroutine for fetching data. The ensures that each host is isolated so that a single unresponsive host cannot inadvertently block other hosts within the same Module and MetricSet from collection.

func (*Metricbeat) Stop

func (bt *Metricbeat) Stop()

Stop signals to Metricbeat that it should stop. It closes the "done" channel and closes the publisher client associated with each Module.

Stop should only be called a single time. Calling it more than once may result in undefined behavior.

type ModuleRunner

type ModuleRunner interface {
	// Start starts the Module. If Start is called more than once, only the
	// first will start the Module.
	Start()

	// Stop stops the Module and waits for module's MetricSets to exit. The
	// publisher.Client will be closed by Stop. If Stop is called more than
	// once, only the first stop the Module and wait for it to exit.
	Stop()
}

ModuleRunner is a facade for a ModuleWrapper that provides a simple interface for starting and stopping a Module.

Example

ExampleModuleRunner demonstrates how to use ModuleRunner to start and stop a module.

// A *beat.Beat is injected into a Beater when it runs and contains the
// Publisher used to publish events. This Beat pointer is created here only
// for demonstration purposes.
var b *beat.Beat

config, err := common.NewConfigFrom(map[string]interface{}{
	"module":     moduleName,
	"metricsets": []string{metricSetName},
})
if err != nil {
	return
}

// Create a new ModuleWrapper based on the configuration.
module, err := metricbeat.NewModuleWrapper(config, mb.Registry)
if err != nil {
	return
}

// Create the ModuleRunner facade.
runner := metricbeat.NewModuleRunner(b.Publisher.Connect, module)

// Start the module and have it publish to a new publisher.Client.
runner.Start()

// Stop the module. This blocks until all MetricSets in the Module have
// stopped and the publisher.Client is closed.
runner.Stop()
Output:

func NewModuleRunner

func NewModuleRunner(pubClientFactory func() publisher.Client, mod *ModuleWrapper) ModuleRunner

NewModuleRunner returns a ModuleRunner facade. The events generated by the Module will be published to a new publisher.Client generated from the pubClientFactory.

type ModuleWrapper

type ModuleWrapper struct {
	mb.Module
	// contains filtered or unexported fields
}

ModuleWrapper contains the Module and the private data associated with running the Module and its MetricSets.

Use NewModuleWrapper or NewModuleWrappers to construct new ModuleWrappers.

Example

ExampleModuleWrapper demonstrates how to create a single ModuleWrapper from configuration, start the module, and consume events generated by the module.

// Build a configuration object.
config, err := common.NewConfigFrom(map[string]interface{}{
	"module":     moduleName,
	"metricsets": []string{metricSetName},
})
if err != nil {
	fmt.Println("Error:", err)
	return
}

// Create a new ModuleWrapper based on the configuration.
module, err := metricbeat.NewModuleWrapper(config, mb.Registry)
if err != nil {
	fmt.Println("Error:", err)
	return
}

// Run the module until done is closed.
done := make(chan struct{})
output := module.Start(done)

// Process events from the output channel until it is closed.
var wg sync.WaitGroup
wg.Add(1)
go func() {
	defer wg.Done()
	for event := range output {
		// Make rtt a constant so that the output is constant.
		event["metricset"].(common.MapStr)["rtt"] = 111
		fmt.Println(event.StringToPrint())
	}
}()

// Simulate running for a while.
time.Sleep(50 * time.Millisecond)

// When finished with the module, close the done channel. When the Module
// stops it will automatically close its output channel so that the output
// for loop stops.
close(done)
wg.Wait()
Output:

{
  "@timestamp": "2016-05-10T23:27:58.485Z",
  "_event_metadata": {
    "Fields": null,
    "FieldsUnderRoot": false,
    "Tags": null
  },
  "fake": {
    "status": {
      "metric": 1
    }
  },
  "metricset": {
    "module": "fake",
    "name": "status",
    "rtt": 111
  },
  "type": "metricsets"
}

func NewModuleWrapper

func NewModuleWrapper(moduleConfig *common.Config, r *mb.Register) (*ModuleWrapper, error)

NewModuleWrapper create a new Module and its associated MetricSets based on the given configuration. It constructs the supporting filters and stores them in the ModuleWrapper.

func NewModuleWrappers

func NewModuleWrappers(modulesConfig []*common.Config, r *mb.Register) ([]*ModuleWrapper, error)

NewModuleWrappers creates new Modules and their associated MetricSets based on the given configuration. It constructs the supporting filters and stores them all in a ModuleWrapper.

func (*ModuleWrapper) Start

func (mw *ModuleWrapper) Start(done <-chan struct{}) <-chan common.MapStr

Start starts the Module's MetricSet workers which are responsible for fetching metrics. The workers will continue to periodically fetch until the done channel is closed. When the done channel is closed all MetricSet workers will stop and the returned output channel will be closed.

The returned channel is buffered with a length one one. It must drained to prevent blocking the operation of the MetricSets.

Start should be called only once in the life of a ModuleWrapper.

func (*ModuleWrapper) String

func (mw *ModuleWrapper) String() string

String returns a string representation of ModuleWrapper.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL