inserter

package

v0.0.0-...-2ecc45d Latest Latest Go to latest Published: Jan 10, 2023 License: MIT Imports: 16 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/edwvee/dbatcher

Links

Open Source Insights

Documentation ¶

Index ¶

Variables
type ClickHouseInserter
- func (ci *ClickHouseInserter) Init(config Config) error
- func (ci ClickHouseInserter) Insert(t *table.Table) error
type Config
type DummyInserter
- func (ci *DummyInserter) Init(config Config) error
- func (ci DummyInserter) Insert(t *table.Table) error
type InsertErrorLogger
- func NewInsertErrorLogger(w io.Writer, prettyPrint bool) *InsertErrorLogger
- func NewInsertErrorLoggerFromConfig(config InsertErrorLoggerConfig) (*InsertErrorLogger, error)
type InsertErrorLoggerConfig
type Inserter
type MysqlInserter
- func (mi *MysqlInserter) Init(config Config) error
- func (mi MysqlInserter) Insert(t *table.Table) error

Constants ¶

This section is empty.

Variables ¶

View Source

var (
	//ErrNoDatabaseInDsnOrInTableName is returned when inserter tries to get table's structure
	//and there is need in database's name and it can't be found
	ErrNoDatabaseInDsnOrInTableName = errors.New("no database in dsn or in table name")
	//ErrNoSuchTableStructure means there are no column configuration for such database and table
	ErrNoSuchTableStructure = errors.New("no column info for a table")
)

View Source

var ErrCantParseToClickhouseType = errors.New("can't parse clickhouse type")

ErrCantParseToClickhouseType means type of value from JSON can't be converted into clickhouse's column type

Functions ¶

This section is empty.

Types ¶

type ClickHouseInserter ¶

type ClickHouseInserter struct {
	// contains filtered or unexported fields
}

ClickHouseInserter inserts rows into ClickHouse

func (*ClickHouseInserter) Init ¶

func (ci *ClickHouseInserter) Init(config Config) error

Init setups ClickHouseInserter and connects to ClickHouse

func (ClickHouseInserter) Insert ¶

func (ci ClickHouseInserter) Insert(t *table.Table) error

Insert gets table structure and inserts

type Config ¶

type Config struct {
	Type            string `toml:"type"`
	Dsn             string `toml:"dsn"`
	MaxConnections  int    `toml:"max_connections"`
	InsertTimeoutMs int    `toml:"insert_timeout_ms"`
}

Config is a config for inserter

type DummyInserter ¶

type DummyInserter struct{}

DummyInserter doesn't write data, only reports

func (*DummyInserter) Init ¶

func (ci *DummyInserter) Init(config Config) error

Init does nothing

func (DummyInserter) Insert ¶

func (ci DummyInserter) Insert(t *table.Table) error

Insert reports about table's rows count

type InsertErrorLogger ¶

type InsertErrorLogger struct {
	// contains filtered or unexported fields
}

func NewInsertErrorLogger ¶

func NewInsertErrorLogger(w io.Writer, prettyPrint bool) *InsertErrorLogger

func NewInsertErrorLoggerFromConfig ¶

func NewInsertErrorLoggerFromConfig(config InsertErrorLoggerConfig) (*InsertErrorLogger, error)

func (*InsertErrorLogger) Close ¶

func (l *InsertErrorLogger) Close() error

func (*InsertErrorLogger) Log ¶

func (l *InsertErrorLogger) Log(insertError error, t *table.Table) error

func (*InsertErrorLogger) MakeData ¶

func (l *InsertErrorLogger) MakeData(insertError error, t *table.Table) insertErrorLoggerData

type InsertErrorLoggerConfig ¶

type InsertErrorLoggerConfig struct {
	Path        string `toml:"path"`
	PrettyPrint bool   `toml:"pretty_print"`
}

type Inserter ¶

type Inserter interface {
	Init(config Config) error
	Insert(t *table.Table) error
}

Inserter inserts table's rows to a specific DMBS or other destination

type MysqlInserter ¶

type MysqlInserter struct {
	// contains filtered or unexported fields
}

MysqlInserter inserts rows into MySQL

func (*MysqlInserter) Init ¶

func (mi *MysqlInserter) Init(config Config) error

Init setups MysqlInserter and connects to mysql

func (MysqlInserter) Insert ¶

func (mi MysqlInserter) Insert(t *table.Table) error

Insert inserts rows to mysql

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL