datadogexporter

package module
v0.99.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 22, 2024 License: Apache-2.0 Imports: 52 Imported by: 10

README

Datadog Exporter

Status
Stability alpha: logs
beta: traces, metrics
Distributions contrib
Issues Open issues Closed issues
Code Owners @mx-psi, @dineshg13, @liustanley, @songy23, @mackjmr
Emeritus @gbbr

Please review the Collector's security documentation, which contains recommendations on securing sensitive information such as the API key required by this exporter.

The Datadog Exporter now skips APM stats computation by default. It is recommended to only use the Datadog Connector in order to compute APM stats. To temporarily revert to the previous behavior, disable the exporter.datadogexporter.DisableAPMStats feature gate. Example: otelcol --config=config.yaml --feature-gates=-exporter.datadogexporter.DisableAPMStats

Visit the official documentation for usage instructions.

FAQs

Why am I getting errors 413 - Request Entity Too Large, how do I fix it?

This error indicates the payload size sent by the Datadog exporter exceeds the size limit (see previous examples https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/16834, https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/17566).

This is usually caused by the pipeline batching too many telemetry data before sending to the Datadog exporter. To fix that, try lowering send_batch_size and send_batch_max_size in your batchprocessor config. You might want to have a separate batch processor dedicated for datadog exporter if other exporters expect a larger batch size, e.g.

processors:
  batch:  # To be used by other exporters
    timeout: 1s
    # Default value for send_batch_size is 8192
  batch/datadog:
    send_batch_max_size: 100
    send_batch_size: 10
    timeout: 10s
...
service:
  pipelines:
    metrics:
      receivers: ...
      processors: [batch/datadog]
      exporters: [datadog]

The exact values for send_batch_size and send_batch_max_size depends on your specific workload. Also note that, Datadog intake has different payload size limits for the 3 signal types:

Fall back to the Zorkian metric client with feature gate

Since v0.69.0, the Datadog exporter has switched to use the native metric client datadog-api-client-go for metric export instead of Zorkian client by default. While datadog-api-client-go fixed several issues that are present in Zorkian client, there is a performance regression with it compared to Zorkian client especially under high metric volume. If you observe memory or throughput issues in the Datadog exporter with datadog-api-client-go, you can configure the Datadog exporter to fall back to the Zorkian client by disabling the feature gate exporter.datadogexporter.metricexportnativeclient, e.g.

otelcol --config=config.yaml --feature-gates=-exporter.datadogexporter.metricexportnativeclient

Note that we are currently migrating the Datadog metrics exporter to use the metrics serializer instead. The feature flag exporter.datadogexporter.metricexportnativeclient will be deprecated and eventually removed in the future, following the feature lifecycle.

Remap OTel’s service.name attribute to service for logs

For Datadog Exporter versions 0.83.0 and later, the service field of OTel logs is populated as OTel semantic convention service.name. However, service.name is not one of the default service attributes in Datadog’s log preprocessing.

To get the service field correctly populated in your logs, you can specify service.name to be the source of a log’s service by setting a log service remapper processor.

Documentation

Index

Constants

View Source
const (
	// DefaultSite is the default site of the Datadog intake to send data to
	DefaultSite = "datadoghq.com"
)

Variables

This section is empty.

Functions

func NewFactory

func NewFactory() exporter.Factory

NewFactory creates a Datadog exporter factory

Types

type APIConfig

type APIConfig struct {
	// Key is the Datadog API key to associate your Agent's data with your organization.
	// Create a new API key here: https://app.datadoghq.com/account/settings
	Key configopaque.String `mapstructure:"key"`

	// Site is the site of the Datadog intake to send data to.
	// The default value is "datadoghq.com".
	Site string `mapstructure:"site"`

	// FailOnInvalidKey states whether to exit at startup on invalid API key.
	// The default value is false.
	FailOnInvalidKey bool `mapstructure:"fail_on_invalid_key"`
}

APIConfig defines the API configuration options

type Config

type Config struct {
	confighttp.ClientConfig      `mapstructure:",squash"` // squash ensures fields are correctly decoded in embedded struct.
	exporterhelper.QueueSettings `mapstructure:"sending_queue"`
	configretry.BackOffConfig    `mapstructure:"retry_on_failure"`

	TagsConfig `mapstructure:",squash"`

	// API defines the Datadog API configuration.
	API APIConfig `mapstructure:"api"`

	// Metrics defines the Metrics exporter specific configuration
	Metrics MetricsConfig `mapstructure:"metrics"`

	// Traces defines the Traces exporter specific configuration
	Traces TracesConfig `mapstructure:"traces"`

	// Logs defines the Logs exporter specific configuration
	Logs LogsConfig `mapstructure:"logs"`

	// HostMetadata defines the host metadata specific configuration
	HostMetadata HostMetadataConfig `mapstructure:"host_metadata"`

	// OnlyMetadata defines whether to only send metadata
	// This is useful for agent-collector setups, so that
	// metadata about a host is sent to the backend even
	// when telemetry data is reported via a different host.
	//
	// This flag is incompatible with disabling host metadata,
	// `use_resource_metadata`, or `host_metadata::hostname_source != first_resource`
	OnlyMetadata bool `mapstructure:"only_metadata"`
	// contains filtered or unexported fields
}

Config defines configuration for the Datadog exporter.

func (*Config) Unmarshal added in v0.55.0

func (c *Config) Unmarshal(configMap *confmap.Conf) error

Unmarshal a configuration map into the configuration struct.

func (*Config) Validate added in v0.55.0

func (c *Config) Validate() error

Validate the configuration for errors. This is required by component.Config.

type CumulativeMonotonicSumMode added in v0.55.0

type CumulativeMonotonicSumMode string

CumulativeMonotonicSumMode is the export mode for OTLP Sum metrics.

const (
	// CumulativeMonotonicSumModeToDelta calculates delta for
	// cumulative monotonic sum metrics in the client side and reports
	// them as Datadog counts.
	CumulativeMonotonicSumModeToDelta CumulativeMonotonicSumMode = "to_delta"

	// CumulativeMonotonicSumModeRawValue reports the raw value for
	// cumulative monotonic sum metrics as a Datadog gauge.
	CumulativeMonotonicSumModeRawValue CumulativeMonotonicSumMode = "raw_value"
)

func (*CumulativeMonotonicSumMode) UnmarshalText added in v0.55.0

func (sm *CumulativeMonotonicSumMode) UnmarshalText(in []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface.

type HistogramConfig added in v0.55.0

type HistogramConfig struct {
	// Mode for exporting histograms. Valid values are 'distributions', 'counters' or 'nobuckets'.
	//  - 'distributions' sends histograms as Datadog distributions (recommended).
	//  - 'counters' sends histograms as Datadog counts, one metric per bucket.
	//  - 'nobuckets' sends no bucket histogram metrics. Aggregation metrics will still be sent
	//    if `send_aggregation_metrics` is enabled.
	//
	// The current default is 'distributions'.
	Mode HistogramMode `mapstructure:"mode"`

	// SendCountSum states if the export should send .sum and .count metrics for histograms.
	// The default is false.
	// Deprecated: [v0.75.0] Use `send_aggregation_metrics` (HistogramConfig.SendAggregations) instead.
	SendCountSum bool `mapstructure:"send_count_sum_metrics"`

	// SendAggregations states if the exporter should send .sum, .count, .min and .max metrics for histograms.
	// The default is false.
	SendAggregations bool `mapstructure:"send_aggregation_metrics"`
}

HistogramConfig customizes export of OTLP Histograms.

type HistogramMode added in v0.55.0

type HistogramMode string
const (
	// HistogramModeNoBuckets reports no bucket histogram metrics. .sum and .count metrics will still be sent
	// if `send_count_sum_metrics` is enabled.
	HistogramModeNoBuckets HistogramMode = "nobuckets"
	// HistogramModeCounters reports histograms as Datadog counts, one metric per bucket.
	HistogramModeCounters HistogramMode = "counters"
	// HistogramModeDistributions reports histograms as Datadog distributions (recommended).
	HistogramModeDistributions HistogramMode = "distributions"
)

func (*HistogramMode) UnmarshalText added in v0.55.0

func (hm *HistogramMode) UnmarshalText(in []byte) error

type HostMetadataConfig added in v0.55.0

type HostMetadataConfig struct {
	// Enabled enables the host metadata functionality.
	Enabled bool `mapstructure:"enabled"`

	// HostnameSource is the source for the hostname of host metadata.
	// This hostname is used for identifying the infrastructure list, host map and host tag information related to the host where the Datadog exporter is running.
	// Changing this setting will not change the host used to tag your metrics, traces and logs in any way.
	// For remote hosts, see https://docs.datadoghq.com/opentelemetry/schema_semantics/host_metadata/.
	//
	// Valid values are 'first_resource' and 'config_or_system':
	// - 'first_resource' picks the host metadata hostname from the resource
	//    attributes on the first OTLP payload that gets to the exporter.
	//    If the first payload lacks hostname-like attributes, it will fallback to 'config_or_system'.
	//    **Do not use this hostname source if receiving data from multiple hosts**.
	// - 'config_or_system' picks the host metadata hostname from the 'hostname' setting,
	//    If this is empty it will use available system APIs and cloud provider endpoints.
	//
	// The default is 'config_or_system'.
	HostnameSource HostnameSource `mapstructure:"hostname_source"`

	// Tags is a list of host tags.
	// These tags will be attached to telemetry signals that have the host metadata hostname.
	// To attach tags to telemetry signals regardless of the host, use a processor instead.
	Tags []string `mapstructure:"tags"`
}

HostMetadataConfig defines the host metadata related configuration. Host metadata is the information used for populating the infrastructure list, the host map and providing host tags functionality.

The exporter will send host metadata for a single host, whose name is chosen according to `host_metadata::hostname_source`.

type HostnameSource added in v0.55.0

type HostnameSource string

HostnameSource is the source for the hostname of host metadata.

const (
	// HostnameSourceFirstResource picks the host metadata hostname from the resource
	// attributes on the first OTLP payload that gets to the exporter. If it is lacking any
	// hostname-like attributes, it will fallback to 'config_or_system' behavior (see below).
	//
	// Do not use this hostname source if receiving data from multiple hosts.
	HostnameSourceFirstResource HostnameSource = "first_resource"

	// HostnameSourceConfigOrSystem picks the host metadata hostname from the 'hostname' setting,
	// and if this is empty, from available system APIs and cloud provider endpoints.
	HostnameSourceConfigOrSystem HostnameSource = "config_or_system"
)

func (*HostnameSource) UnmarshalText added in v0.55.0

func (sm *HostnameSource) UnmarshalText(in []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface.

type InitialValueMode added in v0.82.0

type InitialValueMode string

InitialValueMode defines what the exporter should do with the initial value of a time series when transforming from cumulative to delta.

const (
	// InitialValueModeAuto reports the initial value if its start timestamp
	// is set and it happens after the process was started.
	InitialValueModeAuto InitialValueMode = "auto"

	// InitialValueModeDrop always drops the initial value.
	InitialValueModeDrop InitialValueMode = "drop"

	// InitialValueModeKeep always reports the initial value.
	InitialValueModeKeep InitialValueMode = "keep"
)

func (*InitialValueMode) UnmarshalText added in v0.82.0

func (iv *InitialValueMode) UnmarshalText(in []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface.

type LogsConfig added in v0.61.0

type LogsConfig struct {
	// TCPAddr.Endpoint is the host of the Datadog intake server to send logs to.
	// If unset, the value is obtained from the Site.
	confignet.TCPAddrConfig `mapstructure:",squash"`

	// DumpPayloads report whether payloads should be dumped when logging level is debug.
	DumpPayloads bool `mapstructure:"dump_payloads"`
}

LogsConfig defines logs exporter specific configuration

type MetricsConfig

type MetricsConfig struct {
	// DeltaTTL defines the time that previous points of a cumulative monotonic
	// metric are kept in memory to calculate deltas
	DeltaTTL int64 `mapstructure:"delta_ttl"`

	// TCPAddr.Endpoint is the host of the Datadog intake server to send metrics to.
	// If unset, the value is obtained from the Site.
	confignet.TCPAddrConfig `mapstructure:",squash"`

	ExporterConfig MetricsExporterConfig `mapstructure:",squash"`

	// HistConfig defines the export of OTLP Histograms.
	HistConfig HistogramConfig `mapstructure:"histograms"`

	// SumConfig defines the export of OTLP Sums.
	SumConfig SumConfig `mapstructure:"sums"`

	// SummaryConfig defines the export for OTLP Summaries.
	SummaryConfig SummaryConfig `mapstructure:"summaries"`
}

MetricsConfig defines the metrics exporter specific configuration options

type MetricsExporterConfig added in v0.55.0

type MetricsExporterConfig struct {
	// ResourceAttributesAsTags, if set to true, will use the exporterhelper feature to transform all
	// resource attributes into metric labels, which are then converted into tags
	ResourceAttributesAsTags bool `mapstructure:"resource_attributes_as_tags"`

	// InstrumentationScopeMetadataAsTags, if set to true, adds the name and version of the
	// instrumentation scope that created a metric to the metric tags
	InstrumentationScopeMetadataAsTags bool `mapstructure:"instrumentation_scope_metadata_as_tags"`
}

MetricsExporterConfig provides options for a user to customize the behavior of the metrics exporter

type SumConfig added in v0.55.0

type SumConfig struct {
	// CumulativeMonotonicMode is the mode for exporting OTLP Cumulative Monotonic Sums.
	// Valid values are 'to_delta' or 'raw_value'.
	//  - 'to_delta' calculates delta for cumulative monotonic sums and sends it as a Datadog count.
	//  - 'raw_value' sends the raw value of cumulative monotonic sums as Datadog gauges.
	//
	// The default is 'to_delta'.
	// See https://docs.datadoghq.com/metrics/otlp/?tab=sum#mapping for details and examples.
	CumulativeMonotonicMode CumulativeMonotonicSumMode `mapstructure:"cumulative_monotonic_mode"`

	// InitialCumulativeMonotonicMode defines the behavior of the exporter when receiving the first value
	// of a cumulative monotonic sum.
	InitialCumulativeMonotonicMode InitialValueMode `mapstructure:"initial_cumulative_monotonic_value"`
}

SumConfig customizes export of OTLP Sums.

type SummaryConfig added in v0.55.0

type SummaryConfig struct {
	// Mode is the the mode for exporting OTLP Summaries.
	// Valid values are 'noquantiles' or 'gauges'.
	//  - 'noquantiles' sends no `.quantile` metrics. `.sum` and `.count` metrics will still be sent.
	//  - 'gauges' sends `.quantile` metrics as gauges tagged by the quantile.
	//
	// The default is 'gauges'.
	// See https://docs.datadoghq.com/metrics/otlp/?tab=summary#mapping for details and examples.
	Mode SummaryMode `mapstructure:"mode"`
}

SummaryConfig customizes export of OTLP Summaries.

type SummaryMode added in v0.55.0

type SummaryMode string

SummaryMode is the export mode for OTLP Summary metrics.

const (
	// SummaryModeNoQuantiles sends no `.quantile` metrics. `.sum` and `.count` metrics will still be sent.
	SummaryModeNoQuantiles SummaryMode = "noquantiles"
	// SummaryModeGauges sends `.quantile` metrics as gauges tagged by the quantile.
	SummaryModeGauges SummaryMode = "gauges"
)

func (*SummaryMode) UnmarshalText added in v0.55.0

func (sm *SummaryMode) UnmarshalText(in []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface.

type TagsConfig

type TagsConfig struct {
	// Hostname is the fallback hostname used for payloads without hostname-identifying attributes.
	// This option will NOT change the hostname applied to your metrics, traces and logs if they already have hostname-identifying attributes.
	// If unset, the hostname will be determined automatically. See https://docs.datadoghq.com/opentelemetry/schema_semantics/hostname/?tab=datadogexporter#fallback-hostname-logic for details.
	//
	// Prefer using the `datadog.host.name` resource attribute over using this setting.
	// See https://docs.datadoghq.com/opentelemetry/schema_semantics/hostname/?tab=datadogexporter#general-hostname-semantic-conventions for details.
	Hostname string `mapstructure:"hostname"`
}

TagsConfig defines the tag-related configuration It is embedded in the configuration

type TracesConfig

type TracesConfig struct {
	// TCPAddr.Endpoint is the host of the Datadog intake server to send traces to.
	// If unset, the value is obtained from the Site.
	confignet.TCPAddrConfig `mapstructure:",squash"`

	// ignored resources
	// A blacklist of regular expressions can be provided to disable certain traces based on their resource name
	// all entries must be surrounded by double quotes and separated by commas.
	// ignore_resources: ["(GET|POST) /healthcheck"]
	IgnoreResources []string `mapstructure:"ignore_resources"`

	// SpanNameRemappings is the map of datadog span names and preferred name to map to. This can be used to
	// automatically map Datadog Span Operation Names to an updated value. All entries should be key/value pairs.
	// span_name_remappings:
	//   io.opentelemetry.javaagent.spring.client: spring.client
	//   instrumentation:express.server: express
	//   go.opentelemetry.io_contrib_instrumentation_net_http_otelhttp.client: http.client
	SpanNameRemappings map[string]string `mapstructure:"span_name_remappings"`

	// If set to true the OpenTelemetry span name will used in the Datadog resource name.
	// If set to false the resource name will be filled with the instrumentation library name + span kind.
	// The default value is `false`.
	SpanNameAsResourceName bool `mapstructure:"span_name_as_resource_name"`

	// If set to true, enables an additional stats computation check on spans to see they have an eligible `span.kind` (server, consumer, client, producer).
	// If enabled, a span with an eligible `span.kind` will have stats computed. If disabled, only top-level and measured spans will have stats computed.
	// NOTE: For stats computed from OTel traces, only top-level spans are considered when this option is off.
	ComputeStatsBySpanKind bool `mapstructure:"compute_stats_by_span_kind"`

	// If set to true, enables `peer.service` aggregation in the exporter. If disabled, aggregated trace stats will not include `peer.service` as a dimension.
	// For the best experience with `peer.service`, it is recommended to also enable `compute_stats_by_span_kind`.
	// If enabling both causes the datadog exporter to consume too many resources, try disabling `compute_stats_by_span_kind` first.
	// If the overhead remains high, it will be due to a high cardinality of `peer.service` values from the traces. You may need to check your instrumentation.
	// Deprecated: Please use PeerTagsAggregation instead
	PeerServiceAggregation bool `mapstructure:"peer_service_aggregation"`

	// If set to true, enables aggregation of peer related tags (e.g., `peer.service`, `db.instance`, etc.) in the datadog exporter.
	// If disabled, aggregated trace stats will not include these tags as dimensions on trace metrics.
	// For the best experience with peer tags, Datadog also recommends enabling `compute_stats_by_span_kind`.
	// If you are using an OTel tracer, it's best to have both enabled because client/producer spans with relevant peer tags
	// may not be marked by the datadog exporter as top-level spans.
	// If enabling both causes the datadog exporter to consume too many resources, try disabling `compute_stats_by_span_kind` first.
	// A high cardinality of peer tags or APM resources can also contribute to higher CPU and memory consumption.
	// You can check for the cardinality of these fields by making trace search queries in the Datadog UI.
	// The default list of peer tags can be found in https://github.com/DataDog/datadog-agent/blob/main/pkg/trace/stats/concentrator.go.
	PeerTagsAggregation bool `mapstructure:"peer_tags_aggregation"`

	// [BETA] Optional list of supplementary peer tags that go beyond the defaults. The Datadog backend validates all tags
	// and will drop ones that are unapproved. The default set of peer tags can be found at
	// https://github.com/DataDog/datadog-agent/blob/505170c4ac8c3cbff1a61cf5f84b28d835c91058/pkg/trace/stats/concentrator.go#L55.
	PeerTags []string `mapstructure:"peer_tags"`

	// TraceBuffer specifies the number of Datadog Agent TracerPayloads to buffer before dropping.
	// The default value is 0, meaning the Datadog Agent TracerPayloads are unbuffered.
	TraceBuffer int `mapstructure:"trace_buffer"`
	// contains filtered or unexported fields
}

TracesConfig defines the traces exporter specific configuration options

Directories

Path Synopsis
internal
clientutil
Package clientutil contains the Datadog API client util functions.
Package clientutil contains the Datadog API client util functions.
hostmetadata
Package metadata is responsible for collecting host metadata from different providers such as EC2, ECS, AWS, etc and pushing it to Datadog.
Package metadata is responsible for collecting host metadata from different providers such as EC2, ECS, AWS, etc and pushing it to Datadog.
hostmetadata/internal/azure
Package azure contains the Azure hostname provider
Package azure contains the Azure hostname provider
hostmetadata/internal/ec2
Package ec2 contains the AWS EC2 hostname provider
Package ec2 contains the AWS EC2 hostname provider
hostmetadata/internal/ecs
Package ecs contains the ECS Fargate hostname provider
Package ecs contains the ECS Fargate hostname provider
hostmetadata/internal/gcp
Package gcp contains the GCP hostname provider
Package gcp contains the GCP hostname provider
hostmetadata/internal/gohai
package gohai provides system information like cpu, memory in a format that is expected by the datadog intake
package gohai provides system information like cpu, memory in a format that is expected by the datadog intake
hostmetadata/internal/k8s
Package k8s contains the Kubernetes hostname provider
Package k8s contains the Kubernetes hostname provider
hostmetadata/internal/system
Package system contains the system hostname provider
Package system contains the system hostname provider
hostmetadata/provider
Package provider contains the cluster name provider
Package provider contains the cluster name provider
hostmetadata/valid
Package valid contains functions that validate hostnames
Package valid contains functions that validate hostnames
logs
Package logs provides utils for transforming OTLP LogRecord to Datadog format it also provides sender for submitting transformed logs to datadog backend This uses datadog-api-client-go for submitting logs
Package logs provides utils for transforming OTLP LogRecord to Datadog format it also provides sender for submitting transformed logs to datadog backend This uses datadog-api-client-go for submitting logs
metrics
Package metrics contains helper functions for Datadog metric export
Package metrics contains helper functions for Datadog metric export
metrics/sketches
Package sketches is a copy of part from github.com/DataDog/datadog-agent/pkg/metrics.
Package sketches is a copy of part from github.com/DataDog/datadog-agent/pkg/metrics.
scrub
Package scrub contains a Scrubber that scrubs error from sensitive details
Package scrub contains a Scrubber that scrubs error from sensitive details
testutil
Package testutil contains the test util functions
Package testutil contains the test util functions

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL