connector

package
v0.0.0-...-1543348 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 7, 2023 License: Apache-2.0 Imports: 5 Imported by: 0

README

Connectors

A connector is both an exporter and receiver. It consumes data as an exporter in one pipeline and emits data as a receiver in another pipeline. It may consume and emit data of the same data type, or of different data types. A connector may generate and emit data to summarize the consumed data, or it may simply replicate or route data.

Status

Connectors are currently in Alpha stage. To enable connectors, run the collector with the service.connectors feature gate. e.g. ./otelcol --config config.yaml --feature-gates connectors.

Supported Data Types

Each type of connector is designed to work with one or more pairs of data types and may only be used to connect pipelines accordingly. (Recall that every pipeline is associated with a single data type, either traces, metrics, or logs.)

For example, the count connector counts traces, metrics, and logs, and reports the counts as a metric. Therefore, it may be used to connect the following types of pipelines.

Exporter Pipeline Type Receiver Pipeline Type
traces metrics
metrics metrics
logs metrics

Another example, the router connector, is useful for routing data onto the appropriate pipeline so that it may be processed in distinct ways and/or exported to an appropriate backend. It does not alter the data it consumes in any ways, nor does it produce any additional data. Therefore, it may be used to connect the following types of pipelines.

Exporter Pipeline Type Receiver Pipeline Type
traces traces
metrics metrics
logs logs

Configuration

Declaration

Connectors are defined within a dedicated connectors section at the top level of the collector config.

The count connector may be used with default settings.

receivers:
  foo:
exporters:
  bar:
connectors:
  count:
  router:
Usage

Recall that a connector is an exporter and a receiver and that each connector MUST be used as both, in separate pipelines.

receivers:
  foo:
exporters:
  bar:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo]
      exporters: [count]
    metrics:
      receivers: [count]
      exporters: [bar]

Connectors can be used alongside traditional exporters.

receivers:
  foo:
exporters:
  bar/traces_backend:
  bar/metrics_backend:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo]
      exporters: [bar/traces_backend, count]
    metrics:
      receivers: [count]
      exporters: [bar/metrics_backend]

Connectors can be used alongside traditional receivers.

receivers:
  foo/traces:
  foo/metrics:
exporters:
  bar:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo/traces]
      exporters: [count]
    metrics:
      receivers: [foo/metrics, count]
      exporters: [bar]

A connector can be an exporter in multiple pipelines.

receivers:
  foo/traces:
  foo/metrics:
  foo/logs:
exporters:
  bar/traces_backend:
  bar/metrics_backend:
  bar/logs_backend:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo/traces]
      exporters: [bar/traces_backend, count]
    metrics:
      receivers: [foo/metrics]
      exporters: [bar/metrics_backend, count]
    logs:
      receivers: [foo/logs]
      exporters: [bar/logs_backend, count]
    metrics/counts:
      receivers: [count]
      exporters: [bar/metrics_backend]

A connector can be a receiver in multiple pipelines.

receivers:
  foo/traces:
  foo/metrics:
exporters:
  bar/traces_backend:
  bar/metrics_backend:
  bar/metrics_backend/2:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo/traces]
      exporters: [bar/traces_backend, count]
    metrics:
      receivers: [count]
      exporters: [bar/metrics_backend]
    metrics/2:
      receivers: [count]
      exporters: [bar/metrics_backend/2]

Multiple connectors can be used in sequence.

receivers:
  foo:
exporters:
  bar:
connectors:
  count:
  count/the_counts:
service:
  pipelines:
    traces:
      receivers: [foo]
      exporters: [count]
    metrics:
      receivers: [count]
      exporters: [bar/metrics_backend, count/the_counts]
    metrics/count_the_counts:
      receivers: [count/the_counts]
      exporters: [bar]

A connector can only be used in a pair of pipelines when it supports the combination of Exporter Pipeline Type and Receiver Pipeline Type.

receivers:
  foo:
exporters:
  bar:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo]
      exporters: [count]
    logs:
      receivers: [count] # Invalid. The count connector does not support traces -> logs.
      exporters: [bar]
Exporter Pipeline Type

The type of pipeline in which a connector is used as an exporter.

Receiver Pipeline Type

The type of pipeline in which the connector is used as a receiver.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func MakeFactoryMap

func MakeFactoryMap(factories ...Factory) (map[component.Type]Factory, error)

MakeFactoryMap takes a list of connector factories and returns a map with factory type as keys. It returns a non-nil error when there are factories with duplicate type.

Types

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

Builder processor is a helper struct that given a set of Configs and Factories helps with creating processors.

func NewBuilder

func NewBuilder(cfgs map[component.ID]component.Config, factories map[component.Type]Factory) *Builder

NewBuilder creates a new connector.Builder to help with creating components form a set of configs and factories.

func (*Builder) CreateLogsToLogs

func (b *Builder) CreateLogsToLogs(ctx context.Context, set CreateSettings, next consumer.Logs) (Logs, error)

CreateLogsToLogs creates a Logs connector based on the settings and config.

func (*Builder) CreateLogsToMetrics

func (b *Builder) CreateLogsToMetrics(ctx context.Context, set CreateSettings, next consumer.Metrics) (Logs, error)

CreateLogsToMetrics creates a Logs connector based on the settings and config.

func (*Builder) CreateLogsToTraces

func (b *Builder) CreateLogsToTraces(ctx context.Context, set CreateSettings, next consumer.Traces) (Logs, error)

CreateLogsToTraces creates a Logs connector based on the settings and config.

func (*Builder) CreateMetricsToLogs

func (b *Builder) CreateMetricsToLogs(ctx context.Context, set CreateSettings, next consumer.Logs) (Metrics, error)

CreateMetricsToLogs creates a Metrics connector based on the settings and config.

func (*Builder) CreateMetricsToMetrics

func (b *Builder) CreateMetricsToMetrics(ctx context.Context, set CreateSettings, next consumer.Metrics) (Metrics, error)

CreateMetricsToMetrics creates a Metrics connector based on the settings and config.

func (*Builder) CreateMetricsToTraces

func (b *Builder) CreateMetricsToTraces(ctx context.Context, set CreateSettings, next consumer.Traces) (Metrics, error)

CreateMetricsToTraces creates a Metrics connector based on the settings and config.

func (*Builder) CreateTracesToLogs

func (b *Builder) CreateTracesToLogs(ctx context.Context, set CreateSettings, next consumer.Logs) (Traces, error)

CreateTracesToLogs creates a Traces connector based on the settings and config.

func (*Builder) CreateTracesToMetrics

func (b *Builder) CreateTracesToMetrics(ctx context.Context, set CreateSettings, next consumer.Metrics) (Traces, error)

CreateTracesToMetrics creates a Traces connector based on the settings and config.

func (*Builder) CreateTracesToTraces

func (b *Builder) CreateTracesToTraces(ctx context.Context, set CreateSettings, next consumer.Traces) (Traces, error)

CreateTracesToTraces creates a Traces connector based on the settings and config.

func (*Builder) Factory

func (b *Builder) Factory(componentType component.Type) component.Factory

func (*Builder) IsConfigured

func (b *Builder) IsConfigured(componentID component.ID) bool

type CreateLogsToLogsFunc

type CreateLogsToLogsFunc func(context.Context, CreateSettings, component.Config, consumer.Logs) (Logs, error)

CreateLogsToLogsFunc is the equivalent of Factory.CreateLogsToLogs().

func (CreateLogsToLogsFunc) CreateLogsToLogs

func (f CreateLogsToLogsFunc) CreateLogsToLogs(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Logs,
) (Logs, error)

CreateLogsToLogs implements Factory.CreateLogsToLogs().

type CreateLogsToMetricsFunc

type CreateLogsToMetricsFunc func(context.Context, CreateSettings, component.Config, consumer.Metrics) (Logs, error)

CreateLogsToMetricsFunc is the equivalent of Factory.CreateLogsToMetrics().

func (CreateLogsToMetricsFunc) CreateLogsToMetrics

func (f CreateLogsToMetricsFunc) CreateLogsToMetrics(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Metrics,
) (Logs, error)

CreateLogsToMetrics implements Factory.CreateLogsToMetrics().

type CreateLogsToTracesFunc

type CreateLogsToTracesFunc func(context.Context, CreateSettings, component.Config, consumer.Traces) (Logs, error)

CreateLogsToTracesFunc is the equivalent of Factory.CreateLogsToTraces().

func (CreateLogsToTracesFunc) CreateLogsToTraces

func (f CreateLogsToTracesFunc) CreateLogsToTraces(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Traces,
) (Logs, error)

CreateLogsToTraces implements Factory.CreateLogsToTraces().

type CreateMetricsToLogsFunc

type CreateMetricsToLogsFunc func(context.Context, CreateSettings, component.Config, consumer.Logs) (Metrics, error)

CreateMetricsToLogsFunc is the equivalent of Factory.CreateMetricsToLogs().

func (CreateMetricsToLogsFunc) CreateMetricsToLogs

func (f CreateMetricsToLogsFunc) CreateMetricsToLogs(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Logs,
) (Metrics, error)

CreateMetricsToLogs implements Factory.CreateMetricsToLogs().

type CreateMetricsToMetricsFunc

type CreateMetricsToMetricsFunc func(context.Context, CreateSettings, component.Config, consumer.Metrics) (Metrics, error)

CreateMetricsToMetricsFunc is the equivalent of Factory.CreateMetricsToTraces().

func (CreateMetricsToMetricsFunc) CreateMetricsToMetrics

func (f CreateMetricsToMetricsFunc) CreateMetricsToMetrics(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Metrics,
) (Metrics, error)

CreateMetricsToMetrics implements Factory.CreateMetricsToTraces().

type CreateMetricsToTracesFunc

type CreateMetricsToTracesFunc func(context.Context, CreateSettings, component.Config, consumer.Traces) (Metrics, error)

CreateMetricsToTracesFunc is the equivalent of Factory.CreateMetricsToTraces().

func (CreateMetricsToTracesFunc) CreateMetricsToTraces

func (f CreateMetricsToTracesFunc) CreateMetricsToTraces(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Traces,
) (Metrics, error)

CreateMetricsToTraces implements Factory.CreateMetricsToTraces().

type CreateSettings

type CreateSettings struct {
	// ID returns the ID of the component that will be created.
	ID component.ID

	component.TelemetrySettings

	// BuildInfo can be used by components for informational purposes
	BuildInfo component.BuildInfo
}

CreateSettings configures Connector creators.

type CreateTracesToLogsFunc

type CreateTracesToLogsFunc func(context.Context, CreateSettings, component.Config, consumer.Logs) (Traces, error)

CreateTracesToLogsFunc is the equivalent of Factory.CreateTracesToLogs().

func (CreateTracesToLogsFunc) CreateTracesToLogs

func (f CreateTracesToLogsFunc) CreateTracesToLogs(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Logs,
) (Traces, error)

CreateTracesToLogs implements Factory.CreateTracesToLogs().

type CreateTracesToMetricsFunc

type CreateTracesToMetricsFunc func(context.Context, CreateSettings, component.Config, consumer.Metrics) (Traces, error)

CreateTracesToMetricsFunc is the equivalent of Factory.CreateTracesToMetrics().

func (CreateTracesToMetricsFunc) CreateTracesToMetrics

func (f CreateTracesToMetricsFunc) CreateTracesToMetrics(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Metrics,
) (Traces, error)

CreateTracesToMetrics implements Factory.CreateTracesToMetrics().

type CreateTracesToTracesFunc

type CreateTracesToTracesFunc func(context.Context, CreateSettings, component.Config, consumer.Traces) (Traces, error)

CreateTracesToTracesFunc is the equivalent of Factory.CreateTracesToTraces().

func (CreateTracesToTracesFunc) CreateTracesToTraces

func (f CreateTracesToTracesFunc) CreateTracesToTraces(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Traces) (Traces, error)

CreateTracesToTraces implements Factory.CreateTracesToTraces().

type Factory

type Factory interface {
	component.Factory

	// CreateDefaultConfig creates the default configuration for the Connector.
	// This method can be called multiple times depending on the pipeline
	// configuration and should not cause side-effects that prevent the creation
	// of multiple instances of the Connector.
	// The object returned by this method needs to pass the checks implemented by
	// 'configtest.CheckConfigStruct'. It is recommended to have these checks in the
	// tests of any implementation of the Factory interface.
	CreateDefaultConfig() component.Config

	CreateTracesToTraces(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Traces) (Traces, error)
	CreateTracesToMetrics(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Metrics) (Traces, error)
	CreateTracesToLogs(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Logs) (Traces, error)

	CreateMetricsToTraces(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Traces) (Metrics, error)
	CreateMetricsToMetrics(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Metrics) (Metrics, error)
	CreateMetricsToLogs(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Logs) (Metrics, error)

	CreateLogsToTraces(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Traces) (Logs, error)
	CreateLogsToMetrics(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Metrics) (Logs, error)
	CreateLogsToLogs(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Logs) (Logs, error)

	TracesToTracesStability() component.StabilityLevel
	TracesToMetricsStability() component.StabilityLevel
	TracesToLogsStability() component.StabilityLevel

	MetricsToTracesStability() component.StabilityLevel
	MetricsToMetricsStability() component.StabilityLevel
	MetricsToLogsStability() component.StabilityLevel

	LogsToTracesStability() component.StabilityLevel
	LogsToMetricsStability() component.StabilityLevel
	LogsToLogsStability() component.StabilityLevel
	// contains filtered or unexported methods
}

Factory is factory interface for connectors.

This interface cannot be directly implemented. Implementations must use the NewFactory to implement it.

func NewFactory

func NewFactory(cfgType component.Type, createDefaultConfig component.CreateDefaultConfigFunc, options ...FactoryOption) Factory

NewFactory returns a Factory.

type FactoryOption

type FactoryOption interface {
	// contains filtered or unexported methods
}

FactoryOption applies changes to Factory.

func WithLogsToLogs

func WithLogsToLogs(createLogsToLogs CreateLogsToLogsFunc, sl component.StabilityLevel) FactoryOption

WithLogsToLogs overrides the default "error not supported" implementation for WithLogsToLogs and the default "undefined" stability level.

func WithLogsToMetrics

func WithLogsToMetrics(createLogsToMetrics CreateLogsToMetricsFunc, sl component.StabilityLevel) FactoryOption

WithLogsToMetrics overrides the default "error not supported" implementation for WithLogsToMetrics and the default "undefined" stability level.

func WithLogsToTraces

func WithLogsToTraces(createLogsToTraces CreateLogsToTracesFunc, sl component.StabilityLevel) FactoryOption

WithLogsToTraces overrides the default "error not supported" implementation for WithLogsToTraces and the default "undefined" stability level.

func WithMetricsToLogs

func WithMetricsToLogs(createMetricsToLogs CreateMetricsToLogsFunc, sl component.StabilityLevel) FactoryOption

WithMetricsToLogs overrides the default "error not supported" implementation for WithMetricsToLogs and the default "undefined" stability level.

func WithMetricsToMetrics

func WithMetricsToMetrics(createMetricsToMetrics CreateMetricsToMetricsFunc, sl component.StabilityLevel) FactoryOption

WithMetricsToMetrics overrides the default "error not supported" implementation for WithMetricsToMetrics and the default "undefined" stability level.

func WithMetricsToTraces

func WithMetricsToTraces(createMetricsToTraces CreateMetricsToTracesFunc, sl component.StabilityLevel) FactoryOption

WithMetricsToTraces overrides the default "error not supported" implementation for WithMetricsToTraces and the default "undefined" stability level.

func WithTracesToLogs

func WithTracesToLogs(createTracesToLogs CreateTracesToLogsFunc, sl component.StabilityLevel) FactoryOption

WithTracesToLogs overrides the default "error not supported" implementation for WithTracesToLogs and the default "undefined" stability level.

func WithTracesToMetrics

func WithTracesToMetrics(createTracesToMetrics CreateTracesToMetricsFunc, sl component.StabilityLevel) FactoryOption

WithTracesToMetrics overrides the default "error not supported" implementation for WithTracesToMetrics and the default "undefined" stability level.

func WithTracesToTraces

func WithTracesToTraces(createTracesToTraces CreateTracesToTracesFunc, sl component.StabilityLevel) FactoryOption

WithTracesToTraces overrides the default "error not supported" implementation for WithTracesToTraces and the default "undefined" stability level.

type Logs

type Logs interface {
	component.Component
	consumer.Logs
}

A Logs connector acts as an exporter from a logs pipeline and a receiver to one or more traces, metrics, or logs pipelines. Logs feeds a consumer.Traces, consumer.Metrics, or consumer.Logs with data.

Examples:

  • Structured logs containing span information could be consumed and emitted as traces.
  • Metrics could be extracted from structured logs that contain numeric data.
  • Logs could be collected in one pipeline and routed to another logs pipeline based on criteria such as attributes or other content of the log. The second pipeline can then process and export the log to the appropriate backend.

type Metrics

type Metrics interface {
	component.Component
	consumer.Metrics
}

A Metrics connector acts as an exporter from a metrics pipeline and a receiver to one or more traces, metrics, or logs pipelines. Metrics feeds a consumer.Traces, consumer.Metrics, or consumer.Logs with data.

Examples:

  • Latency between related data points could be modeled and emitted as traces.
  • Metrics could be collected in one pipeline and routed to another metrics pipeline based on criteria such as attributes or other content of the metric. The second pipeline can then process and export the metric to the appropriate backend.
  • Metrics could be analyzed by a logs connector that emits events when particular criteria are met.

type Traces

type Traces interface {
	component.Component
	consumer.Traces
}

A Traces connector acts as an exporter from a traces pipeline and a receiver to one or more traces, metrics, or logs pipelines. Traces feeds a consumer.Traces, consumer.Metrics, or consumer.Logs with data.

Examples:

  • Traces could be collected in one pipeline and routed to another traces pipeline based on criteria such as attributes or other content of the trace. The second pipeline can then process and export the trace to the appropriate backend.
  • Traces could be summarized by a metrics connector that emits statistics describing the number of traces observed.
  • Traces could be analyzed by a logs connector that emits events when particular criteria are met.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL