Documentation
¶
Index ¶
- Variables
- func CreateIncrementalTable(db *sql.DB, columns []cloudstorage.TableCol, tableName string) error
- func CreateSchema(db *sql.DB, schemaName string) error
- func CreateTable(sourceDatabase string, sourceTable string, sourceTiDBConn, redConn *sql.DB) error
- func DeleteQuery(db *sql.DB, tableDef cloudstorage.TableDefinition, incurmentalTableName string) error
- func DropTableIfExists(db *sql.DB, tableName string) error
- func GenDDLViaColumnsDiff(prevColumns []cloudstorage.TableCol, curTableDef cloudstorage.TableDefinition) ([]string, error)
- func GetRedshiftColumnString(column cloudstorage.TableCol) (string, error)
- func GetRedshiftTypeString(column cloudstorage.TableCol) (string, error)
- func InsertQuery(db *sql.DB, tableDef cloudstorage.TableDefinition, incurmentalTableName string) error
- func LoadSnapshotFromS3(db *sql.DB, targetTable, filePath string, credential *credentials.Value) error
- type RedshiftConfig
- type RedshiftConnector
- func (rc *RedshiftConnector) Close()
- func (rc *RedshiftConnector) CopyTableSchema(sourceDatabase string, sourceTable string, sourceTiDBConn *sql.DB) error
- func (rc *RedshiftConnector) ExecDDL(tableDef cloudstorage.TableDefinition) error
- func (rc *RedshiftConnector) InitSchema(columns []cloudstorage.TableCol) error
- func (rc *RedshiftConnector) LoadIncrement(tableDef cloudstorage.TableDefinition, filePath string) error
- func (rc *RedshiftConnector) LoadSnapshot(targetTable, filePath string) error
Constants ¶
This section is empty.
Variables ¶
View Source
var TiDB2RedshiftTypeMap map[string]string = map[string]string{
"text": "TEXT",
"tinytext": "TEXT",
"mediumtext": "TEXT",
"longtext": "TEXT",
"blob": "VARBYTE",
"tinyblob": "VARBYTE",
"varchar": "VARCHAR",
"char": "CHAR",
"binary": "VARBYTE",
"varbinary": "VARBYTE",
"tinyint": "SMALLINT",
"tinyint unsigned": "SMALLINT",
"smallint": "SMALLINT",
"smallint unsigned": "INT",
"int": "INT",
"int unsigned": "BIGINT",
"mediumint": "INT",
"mediumint unsigned": "BIGINT",
"bigint": "BIGINT",
"bigint unsigned": "DECIMAL(20,0)",
"float": "REAL",
"float unsigned": "REAL",
"double": "DOUBLE PRECISION",
"double unsigned": "DOUBLE PRECISION",
"decimal": "DECIMAL",
"numeric": "NUMERIC",
"bool": "BOOLEAN",
"boolean": "BOOLEAN",
"date": "DATE",
"datetime": "TIMESTAMP",
"timestamp": "TIMESTAMP",
"time": "TIME",
}
TiDB2RedshiftTypeMap is a map from TiDB type to Redshift type.
Functions ¶
func CreateIncrementalTable ¶ added in v0.0.4
CreateIncrementalTable create a temp table to store incremental data Note: We do not use external table here, because redshift external table has a lot of limitations: 1. Does not support NOT NULL or PRIMARY KEY 2. Does not support specifying null value 3. Need a manifest file to specify the file location 4. ...
func CreateTable ¶
func DeleteQuery ¶
func DeleteQuery(db *sql.DB, tableDef cloudstorage.TableDefinition, incurmentalTableName string) error
func GenDDLViaColumnsDiff ¶
func GenDDLViaColumnsDiff(prevColumns []cloudstorage.TableCol, curTableDef cloudstorage.TableDefinition) ([]string, error)
func GetRedshiftColumnString ¶
func GetRedshiftColumnString(column cloudstorage.TableCol) (string, error)
GetRedshiftColumnString returns a string describing the column in Redshift, e.g. "id INT NOT NULL DEFAULT '0'" Refer to: https://dev.mysql.com/doc/refman/8.0/en/data-types.html https://docs.aws.amazon.com/redshift/latest/dg/c_Supported_data_types.html
func GetRedshiftTypeString ¶
func GetRedshiftTypeString(column cloudstorage.TableCol) (string, error)
GetRedshiftColumnString returns the column string for Redshift. For example, "id INT", "name VARCHAR(255) and "age INT".
func InsertQuery ¶
func InsertQuery(db *sql.DB, tableDef cloudstorage.TableDefinition, incurmentalTableName string) error
func LoadSnapshotFromS3 ¶
func LoadSnapshotFromS3(db *sql.DB, targetTable, filePath string, credential *credentials.Value) error
LoadSnapshotFromS3 load snapshot data from s3 to redshift use csv file path for storageUri, like s3://tidbbucket/snapshot/stock.csv
Types ¶
type RedshiftConfig ¶
type RedshiftConnector ¶
type RedshiftConnector struct {
// contains filtered or unexported fields
}
func NewRedshiftConnector ¶
func NewRedshiftConnector(rsConfig *RedshiftConfig, increTableName string, storageURI *url.URL, s3Credentials *credentials.Value) (*RedshiftConnector, error)
func (*RedshiftConnector) Close ¶
func (rc *RedshiftConnector) Close()
func (*RedshiftConnector) CopyTableSchema ¶
func (*RedshiftConnector) ExecDDL ¶
func (rc *RedshiftConnector) ExecDDL(tableDef cloudstorage.TableDefinition) error
func (*RedshiftConnector) InitSchema ¶
func (rc *RedshiftConnector) InitSchema(columns []cloudstorage.TableCol) error
func (*RedshiftConnector) LoadIncrement ¶
func (rc *RedshiftConnector) LoadIncrement(tableDef cloudstorage.TableDefinition, filePath string) error
func (*RedshiftConnector) LoadSnapshot ¶
func (rc *RedshiftConnector) LoadSnapshot(targetTable, filePath string) error
Source Files
Click to show internal directories.
Click to hide internal directories.