README
¶
Lytics Source
Query Lytics Api Or more specifically the Segment Scan api.
Use SQL To query the api and via the mysql protocol, uses Secure HTTPS for transport to Lytics Api.
Example
The default example dataux docker container
includes a single table appearances in database baseball
that is made from a single csv file stored in the container.
We are going to add an additional source (Lytics) via a SQL CREATE statement.
docker pull gcr.io/dataux-io/dataux:latest
docker run --rm -e "LOGGING=debug" -p 4000:4000 --name dataux gcr.io/dataux-io/dataux:latest
mysql -h 127.0.0.1 -P4000
-- Get api key from Account in Lytics
CREATE source lytics WITH {
"type":"lytics",
"schema":"lytics",
"settings" : {
"apikey":"YOUR_API_KEY"
}
};
use lytics;
show tables;
describe user;
select * from user WHERE EXISTS email limit 10;
TODO
- handle ending paging/limits (ie, don't open cursors)
- paging segment scans efficiently
- Select INTO (mytable)
Documentation
¶
Index ¶
- Constants
- Variables
- type Generator
- type ResultReader
- type ResultReaderNext
- type Source
- func (m *Source) Close() error
- func (m *Source) DataSource() schema.Source
- func (m *Source) Init()
- func (m *Source) Open(schemaName string) (schema.Conn, error)
- func (m *Source) Setup(ss *schema.SchemaSource) error
- func (m *Source) Table(table string) (*schema.Table, error)
- func (m *Source) Tables() []string
Constants ¶
const (
// SourceType defines the "source" type from qlbridge datasource
SourceType = "lytics"
)
Variables ¶
var (
DefaultLimit = 1000
)
Functions ¶
This section is empty.
Types ¶
type Generator ¶
type Generator struct {
// contains filtered or unexported fields
}
Generator Generate Lytics api requests from Sql Statements - convert sql WHERE into SegmentQL api statements - fetch entities from entity scan api
func (*Generator) WalkExecSource ¶
type ResultReader ¶
type ResultReader struct {
*exec.TaskBase
Docs []u.JsonHelper
Vals [][]driver.Value
Total int
Aggs u.JsonHelper
ScrollId string
Req *Generator
// contains filtered or unexported fields
}
ResultReader, adapts the lytics http json response
func NewResultReader ¶
func NewResultReader(req *Generator) *ResultReader
func (*ResultReader) Close ¶
func (m *ResultReader) Close() error
type ResultReaderNext ¶
type ResultReaderNext struct {
*ResultReader
}
A wrapper, allowing us to implement sql/driver Next() interface
which is different than qlbridge/datasource Next()
type Source ¶
type Source struct {
// contains filtered or unexported fields
}
Source is the Lytics data source provider responsible for schema management, and connection management to Lytics
Source Files