es_util

package module

v0.3.3 Latest Latest Go to latest Published: Mar 7, 2023 License: MIT Imports: 6 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/haxii/es-util

Links

Open Source Insights

Documentation ¶

Index ¶

func Int64ArrToInterfaceArray(a []int64) []interface{}
func MakeBulkESRequest(es *elastic.Client, bulkSize, limit int, reqGap time.Duration, ...) (int, error)
func MakeBulkESRequestIgnoreConflict(es *elastic.Client, bulkSize, limit int, reqGap time.Duration, ...) (int, error)
func MakeRequest(es *elastic.Client, bulkSize int, makeSingleRequest SingleReqMaker) (successCount int, err error)
func MakeRequestWithOpt(es *elastic.Client, bulkSize int, opt BulkOpt, ...) (successCount int, err error)
func ParseBulkResponse(resp *elastic.BulkResponse) (int, error)
func ParseBulkResponseIgnoreConflict(resp *elastic.BulkResponse) (int, error)
func ParseBulkResponseIgnoreDocMiss(resp *elastic.BulkResponse) (int, error)
func SearchWithExport(makeService func() *elastic.SearchService, export bool) (*elastic.SearchResult, error)
func StrArrToInterfaceArray(a []string) []interface{}
type BulkOpt
type SingleReqMaker

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func Int64ArrToInterfaceArray ¶

func Int64ArrToInterfaceArray(a []int64) []interface{}

func MakeBulkESRequest ¶

func MakeBulkESRequest(es *elastic.Client, bulkSize, limit int,
	reqGap time.Duration, makeSingleRequest SingleReqMaker) (int, error)

MakeBulkESRequest 分批执行 ES 的批量请求，避免 payload 过大造成集群宕机

bulkSize 最终插入的总数据量, limit 为每批批量插入的数据量, reqGap 为每批插入的时间间隔, makeSingleRequest 应按数据的序号返回对应的每个批量请求的具体内容, 返回成功的数量

func MakeBulkESRequestIgnoreConflict ¶ added in v0.1.1

func MakeBulkESRequestIgnoreConflict(es *elastic.Client, bulkSize, limit int,
	reqGap time.Duration, makeSingleRequest SingleReqMaker) (int, error)

MakeBulkESRequestIgnoreConflict 分批执行 ES 的批量请求，忽略版本冲突, 避免 payload 过大造成集群宕机

bulkSize 最终插入的总数据量, limit 为每批批量插入的数据量, reqGap 为每批插入的时间间隔, makeSingleRequest 应按数据的序号返回对应的每个批量请求的具体内容, 返回成功的数量

func MakeRequest ¶ added in v0.3.0

func MakeRequest(es *elastic.Client, bulkSize int, makeSingleRequest SingleReqMaker) (successCount int, err error)

MakeRequest 分批执行 ES 的批量请求，避免 payload 过大造成集群宕机，每批 1k, 间隔 1s

func MakeRequestWithOpt ¶ added in v0.3.0

func MakeRequestWithOpt(es *elastic.Client, bulkSize int, opt BulkOpt,
	makeSingleRequest SingleReqMaker) (successCount int, err error)

func ParseBulkResponse ¶

func ParseBulkResponse(resp *elastic.BulkResponse) (int, error)

func ParseBulkResponseIgnoreConflict ¶ added in v0.1.1

func ParseBulkResponseIgnoreConflict(resp *elastic.BulkResponse) (int, error)

func ParseBulkResponseIgnoreDocMiss ¶ added in v0.3.3

func ParseBulkResponseIgnoreDocMiss(resp *elastic.BulkResponse) (int, error)

func SearchWithExport ¶ added in v0.3.1

func SearchWithExport(makeService func() *elastic.SearchService, export bool) (*elastic.SearchResult, error)

func StrArrToInterfaceArray ¶

func StrArrToInterfaceArray(a []string) []interface{}

Types ¶

type BulkOpt ¶ added in v0.3.0

type BulkOpt struct {
	Limit          int           // 每批数据量 默认 1000
	Gap            time.Duration // 每批插入的时间间隔 默认 1s
	Refresh        string        // 是否 refresh 默认 false
	IgnoreConflict bool          // 是否忽略冲突 默认 false
	IgnoreDocMiss  bool          // 是否忽略文档不存在 默认 false
}

type SingleReqMaker ¶ added in v0.3.0

type SingleReqMaker func(int) elastic.BulkableRequest

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL