clickhouse_20200328

package module
v0.0.0-...-00f639f Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 14, 2023 License: MIT Imports: 14 Imported by: 0

README

filebeat-output-clickhouse

本项目为filebeat输出插件,支持将事件推送到clickhouse数据表,您需要在filebeat中引入本项目并且重新编译filebeat。

一、安装编译

1、下载beats
git clone git@github.com:elastic/beats.git
2、更改beats outputs includes文件, 添加filebeat-output-clickhouse

includes所在目录:beats/libbeat/publisher/includes/includes.go
添加如下代码:

import (
...
_ "gitee.com/superqy/filebeat-output-clickhouse"
)
3、构建filebeat

filebeat所在目录:beats/filebeat
执行如下命令:

make

二、配置

filebeat-out-clickhouse插件配置
#----------------------------- ClickHouse output --------------------------------
output.clickHouse:
  # clickhouse数据库配置
  # https://github.com/ClickHouse/clickhouse-go
  # 示例 tcp://host1:9000?username=user&password=qwerty&database=clicks&read_timeout=10&write_timeout=20&alt_hosts=host2:9000,host3:9000
  url: "tcp://127.0.0.1:9000?debug=true&read_timeout=10&write_timeout=20"
  # 接收数据的表名
  table: ck_test
  # 数据过滤器的表列,匹配日志文件中对应的键
  columns: [ "id", "name", "created_date" ]
  # 异常重试休眠时间 单位:秒
  retry_interval: 60
  # 是否跳过异常事件推送 true-表示跳过执行异常实践 false-会一直重试,重试间隔为retry_interval
  skip_unexpected_type_row: false
  # 批处理数据量,影响filebeat每次抓取数据行
  bulk_max_size: 1000

Documentation

The Go Gopher

There is no documentation for this package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL