cloudflarereceiver

Cloudflare Receiver

Status
Stability	alpha: logs
Distributions	contrib
Issues
Code Owners	@dehaansa, @djaglowski | Seeking more code owners!

This Cloudflare receiver allows Cloudflare's LogPush Jobs to send logs over HTTPS from the Cloudflare logs aggregation system to an OpenTelemetry collector.

Getting Started

To successfully operate this receiver, you must follow these steps in order:

1. Have a Cloudflare site at the Enterprise plan level.
   • At the time the receiver was written, LogPush was available only for Enterprise sites.
2. Create a LogPush HTTP destination job following the directions provided by Cloudflare. When the job is created, it will attempt to validate the connection to the receiver.
   • If you've configured the receiver with a secret to validate requests, ensure you add the value to the destination_conf parameter of the LogPush job by adding its value as a query parameter under the header_X-CF-Secret parameter. For example, "destination_conf": "https://example.com?header_X-CF-Secret=abcd1234".
   • If you want the receiver to parse one of the fields as the log record's timestamp (EdgeStartTimestamp is the default), the timestamp should be formatted RFC3339. This is not the default format, and must be explicitly specified in your job config.
     • If using the deprecated logpull_options parameter to configure your job, this can be explicitly specified by adding &timestamps=rfc3339 to the logpull_options string when creating your LogPush job.
     • If using the output_options parameter to configure your job, this can be explicitly specified by setting the timestamp_format field of output_options to "rfc3339"
   • The receiver expects the uploaded logs to be in ndjson format with no template, prefix, suffix, or delimiter changes based on the options in output_options. The only settings supported by this receiver in output_options are field_names, CVE-2021-44228, and sample_rate.
3. If the LogPush job creates successfully, the receiver is correctly configured and the LogPush job was able to send it a "test" message. If the job failed to create, the most likely issue is with the SSL configuration. Check both the LogPush API response and the receiver's logs for more details.

Optional

If the receiver will be handling TLS termination:

1. Receive a properly CA signed SSL certificate for use on the collector host.
2. Configure the receiver using the previously acquired SSL certificate, and then start the collector.

Configuration

• tls (Optional - Cloudflare requires TLS, and self-signed will not be sufficient)
  • cert_file
    • You may need to append your CA certificate to the server's certificate, if it is not a CA known to the LogPush API.
  • key_file
• endpoint
  • The endpoint on which the receiver will await requests from Cloudflare
• secret
  • If this value is set, the receiver expects to see it in any valid requests under the X-CF-Secret header
• timestamp_field (default: EdgeStartTimestamp)
  • This receiver was built with the Cloudflare http_requests dataset in mind, but should be able to support any Cloudflare dataset. If using another dataset, you will need to set the timestamp_field appropriately in order to have the log record be associated with the correct timestamp. the timestamp must be formatted RFC3339, as stated in the Getting Started section.
• attributes
  • This parameter allows the receiver to be configured to set log record attributes based on fields found in the log message. The fields are not removed from the log message when set in this way. Only string, boolean, integer or float fields can be mapped using this parameter.

Example:

receivers:
  cloudflare:
    logs:
      tls:
        key_file: some_key_file
        cert_file: some_cert_file
      endpoint: 0.0.0.0:12345
      secret: 1234567890abcdef1234567890abcdef
      timestamp_field: EdgeStartTimestamp
      attributes:
        ClientIP: http_request.client_ip
        ClientRequestURI: http_request.uri