qlbridge: github.com/araddon/qlbridge/schema Index | Files

package schema

import "github.com/araddon/qlbridge/schema"

Package schema implements core Relational Algrebra schema objects such as Table, Schema, DataSource, Fields, Headers, Index.

Package schema is a generated protocol buffer package.

It is generated from these files:

schema.proto

It has these top-level messages:

TablePartition
Partition
TablePb
FieldPb
Index

Index

Package Files

apply_schema.go datasource.go message.go registry.go schema.go schema.pb.go

Constants

const (
    // NoNulls defines if we allow nulls
    NoNulls = false
    // AllowNulls ?
    AllowNulls = true
)

Variables

var (
    // ErrNotFound is error expressing sought item was not found.
    ErrNotFound = fmt.Errorf("Not Found")
    // ErrNotImplemented this feature is not implemented for this source.
    ErrNotImplemented = fmt.Errorf("Not Implemented")
)
var (
    // SchemaRefreshInterval default schema Refresh Interval
    SchemaRefreshInterval = -time.Minute * 5

    // Static list of common field names for describe header on Show, Describe
    EngineFullCols       = []string{"Engine", "Support", "Comment", "Transactions", "XA", "Savepoints"}
    ProdedureFullCols    = []string{"Db", "Name", "Type", "Definer", "Modified", "Created", "Security_type", "Comment", "character_set_client ", "collation_connection", "Database Collation"}
    DescribeFullCols     = []string{"Field", "Type", "Collation", "Null", "Key", "Default", "Extra", "Privileges", "Comment"}
    DescribeFullColMap   = map[string]int{"Field": 0, "Type": 1, "Collation": 2, "Null": 3, "Key": 4, "Default": 5, "Extra": 6, "Privileges": 7, "Comment": 8}
    DescribeCols         = []string{"Field", "Type", "Null", "Key", "Default", "Extra"}
    DescribeColMap       = map[string]int{"Field": 0, "Type": 1, "Null": 2, "Key": 3, "Default": 4, "Extra": 5}
    ShowTableColumns     = []string{"Table", "Table_Type"}
    ShowVariablesColumns = []string{"Variable_name", "Value"}
    ShowDatabasesColumns = []string{"Database"}
    ShowTableColumnMap   = map[string]int{"Table": 0}
    ShowIndexCols        = []string{"Table", "Non_unique", "Key_name", "Seq_in_index", "Column_name", "Collation", "Cardinality", "Sub_part", "Packed", "Null", "Index_type", "Index_comment"}
    DescribeFullHeaders  = NewDescribeFullHeaders()
    DescribeHeaders      = NewDescribeHeaders()
)
var (

    // DisableRecover If true, we will not capture/suppress panics.
    // Test only feature hopefully
    DisableRecover bool
)

func CreateDefaultRegistry Uses

func CreateDefaultRegistry(applyer Applyer)

CreateDefaultRegistry create the default registry.

func RegisterSchema Uses

func RegisterSchema(schema *Schema) error

RegisterSchema makes a named schema available by the provided @name If Register is called twice with the same name or if source is nil, it panics.

Sources are specific schemas of type csv, elasticsearch, etc containing multiple tables.

func RegisterSourceAsSchema Uses

func RegisterSourceAsSchema(name string, source Source) error

RegisterSourceAsSchema means you have a datasource, that is going to act as a named schema. ie, this will not be a nested schema with sub-schemas and the source will not be re-useable as a source-type.

func RegisterSourceType Uses

func RegisterSourceType(sourceType string, source Source)

RegisterSourceType makes a datasource type available by the provided @sourceType If Register is called twice with the same name or if source is nil, it panics.

Sources are specific schemas of type csv, elasticsearch, etc containing multiple tables.

type Alter Uses

type Alter interface {
    // DropTable drop given table
    DropTable(table string) error
}

Alter interface for schema storage sources

type Applyer Uses

type Applyer interface {
    // Init initialize the applyer with registry.
    Init(r *Registry)
    // AddOrUpdateOnSchema Add or Update object (Table, Index)
    AddOrUpdateOnSchema(s *Schema, obj interface{}) error
    // Drop an object from schema
    Drop(s *Schema, obj interface{}) error
}

Applyer takes schema writes and applies them. This is used both as a database is being loaded, and schema is loaded by store as well as responsible for applying schema changes such as Alters. In distributed db's this is very, very huge part of work so is a very important interface that is under flux.

func NewApplyer Uses

func NewApplyer(sp SchemaSourceProvider) Applyer

NewApplyer new in memory applyer. For distributed db's we would need a different applyer (Raft).

type ConfigNode Uses

type ConfigNode struct {
    Name     string       `json:"name"`     // Name of this Node optional
    Source   string       `json:"source"`   // Name of source this node belongs to
    Address  string       `json:"address"`  // host/ip
    Settings u.JsonHelper `json:"settings"` // Arbitrary settings
}

ConfigNode are Servers/Services, ie a running instance of said Source - each must represent a single source type - normal use is a server, describing partitions of servers - may have arbitrary config info in Settings.

type ConfigSchema Uses

type ConfigSchema struct {
    Name       string   `json:"name"`    // Virtual Schema Name, must be unique
    Sources    []string `json:"sources"` // List of sources , the names of the "Db" in source
    ConfigNode []string `json:"-"`       // List of backend Servers
}

ConfigSchema is the json/config block for Schema, the data-sources that make up this Virtual Schema. Must have a name and list of sources to include.

type ConfigSource Uses

type ConfigSource struct {
    Name         string            `json:"name"`            // Name
    Schema       string            `json:"schema"`          // Schema Name if different than Name, will join existing schema
    SourceType   string            `json:"type"`            // [mysql,elasticsearch,csv,etc] Name in DataSource Registry
    TablesToLoad []string          `json:"tables_to_load"`  // if non empty, only load these tables
    TableAliases map[string]string `json:"table_aliases"`   // if non empty, only load these tables
    Nodes        []*ConfigNode     `json:"nodes"`           // List of nodes
    Hosts        []string          `json:"hosts"`           // List of hosts, replaces older "nodes"
    Settings     u.JsonHelper      `json:"settings"`        // Arbitrary settings specific to each source type
    Partitions   []*TablePartition `json:"partitions"`      // List of partitions per table (optional)
    PartitionCt  uint32            `json:"partition_count"` // Instead of array of per table partitions, raw partition count
}

ConfigSource are backend datasources ie : storage/database/csvfiles Each represents a single source type/config. May belong to more than one schema.

func NewSourceConfig Uses

func NewSourceConfig(name, sourceType string) *ConfigSource

func (*ConfigSource) String Uses

func (m *ConfigSource) String() string

type Conn Uses

type Conn interface {
    Close() error
}

Conn A Connection/Session to a file, api, backend database. Depending on the features of the backing source, it may optionally implement different portions of this interface.

Minimum Read Features to provide Sql Select

- Scanning:   iterate through messages/rows
- Schema Tables:  at a minium list of tables available, the column level data
                  can be introspected so is optional

Planning:

- CreateMutator(ctx *plan.Context) :  execute a mutation task insert, delete, update

Non Select based Sql DML Operations for Mutator:

- Deletion:    (sql delete)
    Delete()
    DeleteExpression()
- Upsert Interface   (sql Update, Upsert, Insert)
    Put()
    PutMulti()

DataSource Connection Session that is Stateful. this is really a generic interface, will actually implement features below: SchemaColumns, Scanner, Seeker, Mutator

func OpenConn Uses

func OpenConn(schemaName, table string) (Conn, error)

OpenConn a schema-source Connection, Global open connection function using default schema registry.

type ConnAll Uses

type ConnAll interface {
    Close() error
    ConnColumns
    Iterator
    ConnSeeker
    ConnUpsert
    ConnDeletion
}

ConnAll interface describes the FULL set of features a connection can implement.

type ConnColumns Uses

type ConnColumns interface {
    Columns() []string
}

ConnColumns Interface for a data source connection exposing column positions for []driver.Value iteration

type ConnDeletion Uses

type ConnDeletion interface {
    // Delete using this key
    Delete(driver.Value) (int, error)
    // Delete with given expression
    DeleteExpression(p interface{}, n expr.Node) (int, error)
}

ConnDeletion deletion interface for data sources

type ConnMutation Uses

type ConnMutation interface {
    CreateMutator(pc interface{}) (ConnMutator, error)
}

ConnMutation creates a Mutator connection similar to Open() connection for select - accepts the plan context used in this upsert/insert/update - returns a connection which must be closed

type ConnMutator Uses

type ConnMutator interface {
    ConnUpsert
    ConnDeletion
}

ConnMutator Mutator Connection

type ConnPatchWhere Uses

type ConnPatchWhere interface {
    PatchWhere(ctx context.Context, where expr.Node, patch interface{}) (int64, error)
}

ConnPatchWhere pass through where expression to underlying datasource Used for update statements WHERE x = y

type ConnScanner Uses

type ConnScanner interface {
    Conn
    Iterator
}

ConnScanner is the primary basis for reading data sources. It exposes an interface to scan through rows. If the Source supports Predicate Push Down (ie, push the where/sql down to underlying store) this is just the resulting rows. Otherwise, Qlbridge engine must polyfill.

type ConnSeeker Uses

type ConnSeeker interface {
    Get(key driver.Value) (Message, error)
}

ConnSeeker is a conn that is Key-Value store, allows relational implementation to be faster for Seeking row values instead of scanning

type ConnUpsert Uses

type ConnUpsert interface {
    Put(ctx context.Context, key Key, value interface{}) (Key, error)
    PutMulti(ctx context.Context, keys []Key, src interface{}) ([]Key, error)
}

ConnUpsert Mutation interface for Put

- assumes datasource understands key(s?)

type DialectWriter Uses

type DialectWriter interface {
    // Dialect ie "mysql", "postgres", "cassandra", "bigquery"
    Dialect() string
    Table(tbl *Table) string
    FieldType(t value.ValueType) string
}

DialectWriter knows how to format the schema output specific to a dialect such as postgres, mysql, bigquery all have different identity, value escape characters.

type Field Uses

type Field struct {
    FieldPb
    Context map[string]interface{} // During schema discovery of underlying source, may need to store additional info
    // contains filtered or unexported fields
}

Field Describes the column info, name, data type, defaults, index, null - dialects (mysql, mongo, cassandra) have their own descriptors for these,

so this is generic meant to be converted to Frontend at runtime

func NewDescribeFullHeaders Uses

func NewDescribeFullHeaders() []*Field

func NewDescribeHeaders Uses

func NewDescribeHeaders() []*Field

func NewField Uses

func NewField(name string, valType value.ValueType, size int, allowNulls bool, defaultVal driver.Value, key, collation, description string) *Field

func NewFieldBase Uses

func NewFieldBase(name string, valType value.ValueType, size int, desc string) *Field

func (*Field) AddContext Uses

func (m *Field) AddContext(key string, value interface{})

func (*Field) AsRow Uses

func (m *Field) AsRow() []driver.Value

func (*Field) Body Uses

func (m *Field) Body() interface{}

func (*Field) Id Uses

func (m *Field) Id() uint64

func (*Field) String Uses

func (m *Field) String() string

func (*Field) ValueType Uses

func (m *Field) ValueType() value.ValueType

type FieldData Uses

type FieldData []byte

FieldData is the byte value of a "Described" field ready to write to the wire so we don't have to continually re-serialize it.

type FieldPb Uses

type FieldPb struct {
    Name        string   `protobuf:"bytes,1,opt,name=name" json:"name,omitempty"`
    Description string   `protobuf:"bytes,2,opt,name=description" json:"description,omitempty"`
    Key         string   `protobuf:"bytes,3,opt,name=key" json:"key,omitempty"`
    Extra       string   `protobuf:"bytes,4,opt,name=extra" json:"extra,omitempty"`
    Data        string   `protobuf:"bytes,5,opt,name=data" json:"data,omitempty"`
    Length      uint32   `protobuf:"varint,6,opt,name=length" json:"length,omitempty"`
    Type        uint32   `protobuf:"varint,7,opt,name=type" json:"type,omitempty"`
    NativeType  uint32   `protobuf:"varint,8,opt,name=nativeType" json:"nativeType,omitempty"`
    DefLength   uint64   `protobuf:"varint,9,opt,name=defLength" json:"defLength,omitempty"`
    DefVal      []byte   `protobuf:"bytes,11,opt,name=defVal,proto3" json:"defVal,omitempty"`
    Indexed     bool     `protobuf:"varint,13,opt,name=indexed" json:"indexed,omitempty"`
    NoNulls     bool     `protobuf:"varint,14,opt,name=noNulls" json:"noNulls,omitempty"`
    Collation   string   `protobuf:"bytes,15,opt,name=collation" json:"collation,omitempty"`
    Roles       []string `protobuf:"bytes,16,rep,name=roles" json:"roles,omitempty"`
    Indexes     []*Index `protobuf:"bytes,17,rep,name=indexes" json:"indexes,omitempty"`
    ContextJson []byte   `protobuf:"bytes,18,opt,name=contextJson,proto3" json:"contextJson,omitempty"`
}

func (*FieldPb) Descriptor Uses

func (*FieldPb) Descriptor() ([]byte, []int)

func (*FieldPb) GetCollation Uses

func (m *FieldPb) GetCollation() string

func (*FieldPb) GetContextJson Uses

func (m *FieldPb) GetContextJson() []byte

func (*FieldPb) GetData Uses

func (m *FieldPb) GetData() string

func (*FieldPb) GetDefLength Uses

func (m *FieldPb) GetDefLength() uint64

func (*FieldPb) GetDefVal Uses

func (m *FieldPb) GetDefVal() []byte

func (*FieldPb) GetDescription Uses

func (m *FieldPb) GetDescription() string

func (*FieldPb) GetExtra Uses

func (m *FieldPb) GetExtra() string

func (*FieldPb) GetIndexed Uses

func (m *FieldPb) GetIndexed() bool

func (*FieldPb) GetIndexes Uses

func (m *FieldPb) GetIndexes() []*Index

func (*FieldPb) GetKey Uses

func (m *FieldPb) GetKey() string

func (*FieldPb) GetLength Uses

func (m *FieldPb) GetLength() uint32

func (*FieldPb) GetName Uses

func (m *FieldPb) GetName() string

func (*FieldPb) GetNativeType Uses

func (m *FieldPb) GetNativeType() uint32

func (*FieldPb) GetNoNulls Uses

func (m *FieldPb) GetNoNulls() bool

func (*FieldPb) GetRoles Uses

func (m *FieldPb) GetRoles() []string

func (*FieldPb) GetType Uses

func (m *FieldPb) GetType() uint32

func (*FieldPb) ProtoMessage Uses

func (*FieldPb) ProtoMessage()

func (*FieldPb) Reset Uses

func (m *FieldPb) Reset()

func (*FieldPb) String Uses

func (m *FieldPb) String() string

type InMemApplyer Uses

type InMemApplyer struct {
    // contains filtered or unexported fields
}

InMemApplyer applies schema changes in memory. As changes to schema come in (such as ALTER statements, new tables, new databases) we need to apply them to the underlying schema.

func (*InMemApplyer) AddOrUpdateOnSchema Uses

func (m *InMemApplyer) AddOrUpdateOnSchema(s *Schema, v interface{}) error

AddOrUpdateOnSchema we have a schema change to apply. A schema change is a new table, index, or whole new schema being registered. We provide the first argument which is which schema it is being applied to (ie, add table x to schema y).

func (*InMemApplyer) Drop Uses

func (m *InMemApplyer) Drop(s *Schema, v interface{}) error

Drop we have a schema change to apply.

func (*InMemApplyer) Init Uses

func (m *InMemApplyer) Init(r *Registry)

Init store the registry as part of in-mem applyer which needs it.

type Index Uses

type Index struct {
    Name          string   `protobuf:"bytes,1,opt,name=name" json:"name,omitempty"`
    Fields        []string `protobuf:"bytes,2,rep,name=fields" json:"fields,omitempty"`
    PrimaryKey    bool     `protobuf:"varint,3,opt,name=primaryKey" json:"primaryKey,omitempty"`
    HashPartition []string `protobuf:"bytes,4,rep,name=hashPartition" json:"hashPartition,omitempty"`
    PartitionSize int32    `protobuf:"varint,5,opt,name=partitionSize" json:"partitionSize,omitempty"`
}

Index a description of how field(s) should be indexed for a table.

func (*Index) Descriptor Uses

func (*Index) Descriptor() ([]byte, []int)

func (*Index) GetFields Uses

func (m *Index) GetFields() []string

func (*Index) GetHashPartition Uses

func (m *Index) GetHashPartition() []string

func (*Index) GetName Uses

func (m *Index) GetName() string

func (*Index) GetPartitionSize Uses

func (m *Index) GetPartitionSize() int32

func (*Index) GetPrimaryKey Uses

func (m *Index) GetPrimaryKey() bool

func (*Index) ProtoMessage Uses

func (*Index) ProtoMessage()

func (*Index) Reset Uses

func (m *Index) Reset()

func (*Index) String Uses

func (m *Index) String() string

type Iterator Uses

type Iterator interface {
    // Next returns the next message.  If none remain, returns nil.
    Next() Message
}

Iterator is simple iterator for paging through a datastore Message(rows) to be used for scanning. Building block for Tasks that process part of a DAG of tasks to process data.

type Key Uses

type Key interface {
    Key() driver.Value
}

Key interface is the Unique Key identifying a row.

type KeyUint Uses

type KeyUint struct {
    ID uint64
}

KeyUint implements Key interface and is simple uint64 key

func NewKeyUint Uses

func NewKeyUint(key uint64) *KeyUint

NewKeyUint simple new uint64 key

func (*KeyUint) Key Uses

func (m *KeyUint) Key() driver.Value

Key is key interface

type Message Uses

type Message interface {
    Id() uint64
    Body() interface{}
}

Message is an interface to describe a Row being processed by query engine/vm or it is a message between distributed parts of the system. It provides a Id() method which can be used by consistent-hash algorithms for routing a message consistently to different processes/servers.

Body() returns interface allowing this to be generic structure for routing

see http://github.com/lytics/grid

type MessageValues Uses

type MessageValues interface {
    Values() []driver.Value
}

MessageValues describes a message with array of driver.Value.

type Partition Uses

type Partition struct {
    Id    string `protobuf:"bytes,1,opt,name=id" json:"id,omitempty"`
    Left  string `protobuf:"bytes,2,opt,name=left" json:"left,omitempty"`
    Right string `protobuf:"bytes,3,opt,name=right" json:"right,omitempty"`
}

Partition describes a range of data the left-key is contained in this partition the right key is not contained in this partition, in the next one

func (*Partition) Descriptor Uses

func (*Partition) Descriptor() ([]byte, []int)

func (*Partition) GetId Uses

func (m *Partition) GetId() string

func (*Partition) GetLeft Uses

func (m *Partition) GetLeft() string

func (*Partition) GetRight Uses

func (m *Partition) GetRight() string

func (*Partition) ProtoMessage Uses

func (*Partition) ProtoMessage()

func (*Partition) Reset Uses

func (m *Partition) Reset()

func (*Partition) String Uses

func (m *Partition) String() string

type Registry Uses

type Registry struct {
    // contains filtered or unexported fields
}

Registry is a global or namespace registry of datasources and schema. Datasources have a "sourcetype" and define somewhat the driver. Schemas are made up of one or more underlying source-types and have normal schema info about tables etc.

func DefaultRegistry Uses

func DefaultRegistry() *Registry

DefaultRegistry get access to the shared/global registry of all datasource implementations

func NewRegistry Uses

func NewRegistry(applyer Applyer) *Registry

NewRegistry create schema registry.

func (*Registry) GetSource Uses

func (m *Registry) GetSource(sourceType string) (Source, error)

GetSource Find a DataSource by SourceType

func (*Registry) Init Uses

func (m *Registry) Init()

Init pre-schema load call any sources that need pre-schema init

func (*Registry) Schema Uses

func (m *Registry) Schema(schemaName string) (*Schema, bool)

Schema Get schema for given name.

func (*Registry) SchemaAdd Uses

func (m *Registry) SchemaAdd(s *Schema) error

SchemaAdd Add a new Schema

func (*Registry) SchemaAddChild Uses

func (m *Registry) SchemaAddChild(name string, child *Schema) error

SchemaAddChild Add a new Child Schema

func (*Registry) SchemaAddFromConfig Uses

func (m *Registry) SchemaAddFromConfig(conf *ConfigSource) error

SchemaAddFromConfig means you have a Schema-Source you want to add

func (*Registry) SchemaDrop Uses

func (m *Registry) SchemaDrop(schema, name string, objectType lex.TokenType) error

SchemaDrop removes a schema

func (*Registry) SchemaRefresh Uses

func (m *Registry) SchemaRefresh(name string) error

SchemaRefresh means reload the schema from underlying store. Possibly requires introspection.

func (*Registry) Schemas Uses

func (m *Registry) Schemas() []string

Schemas returns a list of schema names

func (*Registry) String Uses

func (m *Registry) String() string

String describe contents of registry.

type Schema Uses

type Schema struct {
    Name       string        // Name of schema
    Conf       *ConfigSource // source configuration
    DS         Source        // This datasource Interface
    InfoSchema *Schema       // represent this Schema as sql schema like "information_schema"
    SchemaRef  *Schema       // IF this is infoschema, the schema it refers to
    // contains filtered or unexported fields
}

Schema is a "Virtual" Schema and may have multiple different backing sources. - Multiple DataSource(s) (each may be discrete source type such as mysql, elasticsearch, etc) - each schema supplies tables to the virtual table pool - each table name across schemas must be unique (or aliased)

func NewInfoSchema Uses

func NewInfoSchema(schemaName string, s *Schema) *Schema

NewInfoSchema create a new empty schema with given name.

func NewSchema Uses

func NewSchema(schemaName string) *Schema

NewSchema create a new empty schema with given name.

func NewSchemaSource Uses

func NewSchemaSource(schemaName string, ds Source) *Schema

NewSchemaSource create a new empty schema with given name and source.

func (*Schema) Current Uses

func (m *Schema) Current() bool

Current Is this schema up to date?

func (*Schema) OpenConn Uses

func (m *Schema) OpenConn(tableName string) (Conn, error)

OpenConn get a connection from this schema by table name.

func (*Schema) Schema Uses

func (m *Schema) Schema(schemaName string) (*Schema, error)

Schema Find a child Schema for given schema name,

func (*Schema) SchemaForTable Uses

func (m *Schema) SchemaForTable(tableName string) (*Schema, error)

SchemaForTable Find a Schema for given Table

func (*Schema) Since Uses

func (m *Schema) Since(dur time.Duration) bool

Since Is this schema object been refreshed within time window described by @dur time ago ?

func (*Schema) Table Uses

func (m *Schema) Table(tableIn string) (*Table, error)

Table gets Table definition for given table name

func (*Schema) Tables Uses

func (m *Schema) Tables() []string

Tables gets list of all tables for this schema.

type SchemaSourceProvider Uses

type SchemaSourceProvider func(s *Schema) Source

SchemaSourceProvider is factory for creating schema storage

type Source Uses

type Source interface {
    // Init provides opportunity for those sources that require/ no configuration and
    // introspect schema from their environment time to load pre-schema discovery
    Init()
    // Setup optional interface for getting the Schema injected during creation/starup.
    // Since the Source is a singleton, stateful manager,  it has a startup/shutdown process.
    Setup(*Schema) error
    // Close this source, ensure connections, underlying resources are closed.
    Close() error
    // Open create a connection (not thread safe) to this source.
    Open(source string) (Conn, error)
    // Tables is a list of table names provided by this source.
    Tables() []string
    // Table get table schema for given table name.
    Table(table string) (*Table, error)
}

Source is an interface describing a datasource such as a database, file, api, in-mem data etc. It is thread-safe, singleton, responsible for creating connections and exposing schema. It also exposes partition information optionally if a distributed source.

Sources are registered in a registry, to be dynamically created as schema demands.

Lifecycle:

Init()
Setup()
// running ....  Open() , Table(name)  etc .....
Close()

type SourcePartitionable Uses

type SourcePartitionable interface {
    // Partitions list of partitions.
    Partitions() []*Partition
    PartitionSource(p *Partition) (Conn, error)
}

SourcePartitionable is an optional interface a source may implement that announces it (source) as partitionable into ranges for splitting reads, writes onto different nodes of a cluster.

Many databases's already have internal Partition schemas this allow's those to be exposed for use in our partitioning, so the query-planner can distributed work across nodes.

type SourceTableColumn Uses

type SourceTableColumn interface {
    // Underlying data type of column
    Column(col string) (value.ValueType, bool)
}

SourceTableColumn is a partial source that just provides access to Column schema info, used in Generators.

type SourceTableSchema Uses

type SourceTableSchema interface {
    Table(table string) (*Table, error)
}

SourceTableSchema Partial interface from Source to define just Table()

type Table Uses

type Table struct {
    TablePb
    Fields         []*Field               // List of Fields, in order
    Context        map[string]interface{} // During schema discovery of underlying source, may need to store additional info
    FieldPositions map[string]int         // Maps name of column to ordinal position in array of []driver.Value's
    FieldMap       map[string]*Field      // Map of Field-name -> Field
    Schema         *Schema                // The schema this is member of
    Source         Source                 // The source
    // contains filtered or unexported fields
}

Table represents traditional definition of Database Table. It belongs to a Schema and can be used to create a Datasource used to read this table.

func NewTable Uses

func NewTable(table string) *Table

NewTable create a new table for a schema.

func (*Table) AddContext Uses

func (m *Table) AddContext(key string, value interface{})

AddContext add key/value pairs to context (settings, metatadata).

func (*Table) AddField Uses

func (m *Table) AddField(fld *Field)

AddField register a new field

func (*Table) AddFieldType Uses

func (m *Table) AddFieldType(name string, valType value.ValueType)

AddFieldType describe and register a new column

func (*Table) AsRows Uses

func (m *Table) AsRows() [][]driver.Value

AsRows return all fields suiteable as list of values for Describe/Show statements.

func (*Table) Body Uses

func (m *Table) Body() interface{}

Body satisifies Message Interface

func (*Table) Column Uses

func (m *Table) Column(col string) (value.ValueType, bool)

Column get the Underlying data type.

func (*Table) Columns Uses

func (m *Table) Columns() []string

Columns list of all column names.

func (*Table) Current Uses

func (m *Table) Current() bool

Current Is this schema object current? ie, have we refreshed it from source since refresh interval.

func (*Table) FieldNamesPositions Uses

func (m *Table) FieldNamesPositions() map[string]int

FieldNamesPositions List of Field Names and ordinal position in Column list

func (*Table) FieldsAsMessages Uses

func (m *Table) FieldsAsMessages() []Message

FieldsAsMessages get list of all fields as interface Message used in schema as sql "describe table"

func (*Table) HasField Uses

func (m *Table) HasField(name string) bool

HasField does this table have given field/column?

func (*Table) Id Uses

func (m *Table) Id() uint64

Id satisifieds Message Interface

func (*Table) Marshal Uses

func (m *Table) Marshal() ([]byte, error)

func (*Table) SetColumns Uses

func (m *Table) SetColumns(cols []string)

SetColumns Explicityly set column names.

func (*Table) SetColumnsFromFields Uses

func (m *Table) SetColumnsFromFields()

SetColumnsFromFields Explicityly set column names from fields.

func (*Table) SetRefreshed Uses

func (m *Table) SetRefreshed()

SetRefreshed update the refreshed date to now.

func (*Table) SetRows Uses

func (m *Table) SetRows(rows [][]driver.Value)

SetRows set rows aka values for this table. Used for schema/testing.

func (*Table) Since Uses

func (m *Table) Since(dur time.Duration) bool

Since Is this schema object within time window described by @dur time ago ?

type TablePartition Uses

type TablePartition struct {
    Table      string       `protobuf:"bytes,1,opt,name=table" json:"table,omitempty"`
    Keys       []string     `protobuf:"bytes,2,rep,name=keys" json:"keys,omitempty"`
    Partitions []*Partition `protobuf:"bytes,3,rep,name=partitions" json:"partitions,omitempty"`
}

Partition describes a range of data (in a Table). left-key is contained in this partition right key is not contained in this partition, in the next partition. So any value >= left-key, and < right-key is contained herein.

func (*TablePartition) Descriptor Uses

func (*TablePartition) Descriptor() ([]byte, []int)

func (*TablePartition) GetKeys Uses

func (m *TablePartition) GetKeys() []string

func (*TablePartition) GetPartitions Uses

func (m *TablePartition) GetPartitions() []*Partition

func (*TablePartition) GetTable Uses

func (m *TablePartition) GetTable() string

func (*TablePartition) ProtoMessage Uses

func (*TablePartition) ProtoMessage()

func (*TablePartition) Reset Uses

func (m *TablePartition) Reset()

func (*TablePartition) String Uses

func (m *TablePartition) String() string

type TablePb Uses

type TablePb struct {
    // Name of table lowercased
    Name string `protobuf:"bytes,1,opt,name=name" json:"name,omitempty"`
    // Name of table (not lowercased)
    NameOriginal string `protobuf:"bytes,2,opt,name=nameOriginal" json:"nameOriginal,omitempty"`
    // some dbs are more hiearchical (table-column-family)
    Parent string `protobuf:"bytes,3,opt,name=parent" json:"parent,omitempty"`
    // Character set, default = utf8
    Charset uint32 `protobuf:"varint,4,opt,name=Charset" json:"Charset,omitempty"`
    // Partitions in this table, optional may be empty
    Partition *TablePartition `protobuf:"bytes,5,opt,name=partition" json:"partition,omitempty"`
    // Partition Count
    PartitionCt uint32 `protobuf:"varint,6,opt,name=PartitionCt" json:"PartitionCt,omitempty"`
    // List of indexes for this table
    Indexes []*Index `protobuf:"bytes,7,rep,name=indexes" json:"indexes,omitempty"`
    // context json bytes
    ContextJson []byte `protobuf:"bytes,8,opt,name=contextJson,proto3" json:"contextJson,omitempty"`
    // List of Fields, in order
    Fieldpbs []*FieldPb `protobuf:"bytes,9,rep,name=fieldpbs" json:"fieldpbs,omitempty"`
}

func (*TablePb) Descriptor Uses

func (*TablePb) Descriptor() ([]byte, []int)

func (*TablePb) GetCharset Uses

func (m *TablePb) GetCharset() uint32

func (*TablePb) GetContextJson Uses

func (m *TablePb) GetContextJson() []byte

func (*TablePb) GetFieldpbs Uses

func (m *TablePb) GetFieldpbs() []*FieldPb

func (*TablePb) GetIndexes Uses

func (m *TablePb) GetIndexes() []*Index

func (*TablePb) GetName Uses

func (m *TablePb) GetName() string

func (*TablePb) GetNameOriginal Uses

func (m *TablePb) GetNameOriginal() string

func (*TablePb) GetParent Uses

func (m *TablePb) GetParent() string

func (*TablePb) GetPartition Uses

func (m *TablePb) GetPartition() *TablePartition

func (*TablePb) GetPartitionCt Uses

func (m *TablePb) GetPartitionCt() uint32

func (*TablePb) ProtoMessage Uses

func (*TablePb) ProtoMessage()

func (*TablePb) Reset Uses

func (m *TablePb) Reset()

func (*TablePb) String Uses

func (m *TablePb) String() string

type TimeMessage Uses

type TimeMessage interface {
    Ts() time.Time
}

TimeMessage describes a message with a timestamp.

Package schema imports 15 packages (graph) and is imported by 65 packages. Updated 2019-10-29. Refresh now. Tools for package owners.