receivercreator

Receiver Creator

Status
Stability	alpha: logs, traces
	beta: metrics
Distributions	contrib
Issues
Code Owners	@rmfitzpatrick

This receiver can instantiate other receivers at runtime based on whether observed endpoints match a configured rule. To use the receiver creator, you must first configure one or more observers that will discover networked endpoints that you may be interested in. The configured rules will be evaluated for each endpoint discovered. If the rule evaluates to true then the receiver for that rule will be started against the matched endpoint.

If you use the receiver creator in multiple pipelines of differing telemetry types, but a given dynamically instantiated receiver doesn't support one of the pipeline's type, it will effectively lead to a logged no-op that won't cause a collector service failure.

Configuration

watch_observers

A list of observers previously defined to be run in the extensions section. receiver_creator will watch for endpoints generated by these observers.

receivers

A map of receiver names (e.g. redis/1) to a template for when and how to instantiate that receiver.

receivers.<receiver_type/id>.rule

Rule expression using expvar syntax. Variables available are detailed below in Rule Expressions.

Note: The built-in type function introduced in v1.14.1 has been relocated to typeOf.

receivers.<receiver_type/id>.config

This is configuration that will be used when creating the receiver at runtime.

This option can use static and dynamic configuration values. Static values are normal YAML values. However, the value can also be dynamically constructed from the discovered endpoint object. Dynamic values are surrounded by backticks (`). If a literal backtick is needed use \` to escape it. Dynamic values can be used with static values in which case they are concatenated. For example:

config:
   secure_url: https://`pod.labels["secure_host"]`

The value of secure_url will be https:// concatenated with the value of the secure_host label.

This can also be used when the discovered endpoint needs to be changed dynamically. For instance, suppose the IP 1.2.3.4 is discovered without a port but the port needs to be set inside endpoint. You could do:

config:
   endpoint: '`endpoint`:8080'

If your target receiver provides an endpoint config field and you aren't manually setting it like the above example, the observer endpoint target value will automatically be sourced. If no endpoint field is available you are required to specify any necessary fields.

receivers.resource_attributes

resource_attributes:
  <endpoint type>:
    <attribute>: <attribute value>

This setting controls what resource attributes are set on telemetry emitted from the created receiver. These attributes can be set from values in the endpoint that was matched by the rule. These attributes vary based on the endpoint type. These defaults can be disabled by setting the attribute to be removed to an empty value. Note that the values can be dynamic and processed the same as in config.

Note that the backticks below are not typos--they indicate the value is set dynamically.

type == "pod"

type == "port"

type == "container"

type == "hostport"

None

type == "k8s.service"

type == "k8s.node"

See redis/2 in examples.

receivers.<receiver_type/id>.resource_attributes

receivers:
  <receiver_type>:
    resource_attributes:
      <attribute>: <attribute string value>

Similar to the per-endpoint type resource_attributes described above but for individual receiver instances. Duplicate attribute entries (including the empty string) in this receiver-specific mapping take precedence. These attribute values also support expansion from endpoint environment content. At this time their values must be strings.

Rule Expressions

Each rule must start with type == ("pod"|"port"|"hostport"|"container"|"k8s.service"|"k8s.node") && such that the rule matches only one endpoint type. Depending on the type of endpoint the rule is targeting it will have different variables available.

Pod

Port

Host Port

Container

Kubernetes Service

Kubernetes Node

Examples

extensions:
  # Configures the Kubernetes observer to watch for pod start and stop events.
  k8s_observer:
    observe_nodes: true
    observe_services: true
  host_observer:

receivers:
  receiver_creator/1:
    # Name of the extensions to watch for endpoints to start and stop.
    watch_observers: [k8s_observer]
    receivers:
      prometheus_simple:
        # Configure prometheus scraping if standard prometheus annotations are set on the pod.
        rule: type == "pod" && annotations["prometheus.io/scrape"] == "true"
        config:
          metrics_path: '`"prometheus.io/path" in annotations ? annotations["prometheus.io/path"] : "/metrics"`'
          endpoint: '`endpoint`:`"prometheus.io/port" in annotations ? annotations["prometheus.io/port"] : 9090`'
        resource_attributes:
          an.attribute: a.value
          # Dynamic configuration values
          app.version: '`labels["app_version"]`'

      redis/1:
        # If this rule matches an instance of this receiver will be started.
        rule: type == "port" && port == 6379
        config:
          # Static receiver-specific config.
          password: secret
          # Dynamic configuration value.
          collection_interval: '`pod.annotations["collection_interval"]`'

      redis/2:
        # Set a resource attribute based on endpoint value.
        rule: type == "port" && port == 6379

    resource_attributes:
      # Dynamic configuration values, overwriting default attributes`
      pod:
        service.name: '`labels["service_name"]`'
        app: '`labels["app"]`'
      port:
        service.name: '`pod.labels["service_name"]`'
        app: '`pod.labels["app"]`'
  receiver_creator/2:
    # Name of the extensions to watch for endpoints to start and stop.
    watch_observers: [host_observer]
    receivers:
      redis/on_host:
        # If this rule matches an instance of this receiver will be started.
        rule: type == "port" && port == 6379 && is_ipv6 == true
        resource_attributes:
          service.name: redis_on_host
  receiver_creator/3:
    watch_observers: [k8s_observer]
    receivers:
      kubeletstats:
        rule: type == "k8s.node"
        config:
          auth_type: serviceAccount
          collection_interval: 10s
          endpoint: '`endpoint`:`kubelet_endpoint_port`'
          extra_metadata_labels:
            - container.id
          metric_groups:
            - container
            - pod
            - node
      httpcheck:
        # Configure probing if standard prometheus annotations are set on the pod.
        rule: type == "k8s.service" && annotations["prometheus.io/probe"] == "true"
        config:
          targets:
          - endpoint: 'http://`endpoint`:`"prometheus.io/port" in annotations ? annotations["prometheus.io/port"] : 9090``"prometheus.io/path" in annotations ? annotations["prometheus.io/path"] : "/health"`'
            method: GET
          collection_interval: 10s

processors:
  exampleprocessor:

exporters:
  exampleexporter:

service:
  pipelines:
    metrics:
      receivers: [receiver_creator/1, receiver_creator/2, receiver_creator/3]
      processors: [exampleprocessor]
      exporters: [exampleexporter]
  extensions: [k8s_observer, host_observer]

The full list of settings exposed for this receiver are documented here with detailed sample configurations here.