Documentation ¶
Overview ¶
Package dbsync contains the code for transferring data in and out of external PostgreSQL and MySQL databases, primarily for Maestro import tables.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type PrimitiveSelect ¶
A primitive select is an SQL statement which only supports SELECT, FROM, WHERE, LIMIT and ORDER BY.
func (*PrimitiveSelect) String ¶
func (s *PrimitiveSelect) String() string
Convet a PrimitiveSelect to an actual SQL statement string.
type TableReader ¶
type TableReader interface { Read(p []byte) (n int, err error) Bytes() int64 Rows() int64 LastId() string Statement() string CountReport(fn func(int64, int64), every int) GetBQSchema(dbConn *sqlx.DB, stmt PrimitiveSelect) (*bigquery.TableSchema, error) }
This interface should be satisfied by all the drivers. It is a Reader, as well as provides a bunch of transfer stats and the schema converted from driver-specific dialect to BigQuery.
func NewTableReader ¶
func NewTableReader(dbConn *sqlx.DB, driver, table string, stmt *PrimitiveSelect, idColumn, lastId string, compress bool) (TableReader, error)
Create a new reader. The driver should be "potgres" or "mysql". The idColumn is the column used for incremental imports and should be indexed. lastId is the greatest id of the previous import. If compress is true, the data will be gzip compressed.
type TableWriter ¶
This interface should be satisfied by all writer drivers.
func NewTableWriter ¶
func NewTableWriter(dbConn *sqlx.DB, driver string, bqTable *bigquery.Table, table string) (TableWriter, error)
Create a new writer. Driver should be "postgres" or "mysql" (NIY). The table argument allows giving the table an alternative name in the target external database.