README
¶
Lytics Source
Query Lytics Api Or more specifically the Segment Scan api.
Use SQL To query the api and via the mysql protocol, uses Secure HTTPS for transport to Lytics Api.
Example
The default example dataux docker container
includes a single table appearances in database baseball
that is made from a single csv file stored in the container.
We are going to add an additional source (Lytics) via a SQL CREATE statement.
docker pull gcr.io/dataux-io/dataux:latest
docker run --rm -e "LOGGING=debug" -p 4000:4000 --network host --name dataux gcr.io/dataux-io/dataux:latest
mysql -h 127.0.0.1 -P4000
-- Create api key from Api-Tokens in Lytics
CREATE source lytics WITH {
"type":"lytics",
"schema":"lytics",
"settings" : {
"apikey":"YOUR_API_KEY"
}
};
use lytics;
show tables;
describe user;
select * from user WHERE EXISTS email limit 10;
TODO
- SELECT single entities.
- handle ending paging/limits (ie, don't open cursors)
- paging segment scans efficiently
- Select INTO (mytable)
Documentation
¶
Index ¶
Constants ¶
const (
// SourceType defines the "source" type from qlbridge datasource
SourceType = "lytics"
)
Variables ¶
var (
// DefaultLimit is default limit page size.
DefaultLimit = 1000
)
Functions ¶
This section is empty.
Types ¶
type Generator ¶
type Generator struct {
// contains filtered or unexported fields
}
Generator Generate Lytics api requests from Sql Statements - convert sql WHERE into SegmentQL api statements - fetch entities from entity scan api
func NewGenerator ¶
NewGenerator create a new generator to generate lytics query from request.
func (*Generator) WalkExecSource ¶
WalkExecSource allow this to do its own Exec planning.
type ResultReader ¶
type ResultReader struct {
*exec.TaskBase
Docs []u.JsonHelper
Vals [][]driver.Value
Total int
Aggs u.JsonHelper
ScrollId string
Req *Generator
// contains filtered or unexported fields
}
ResultReader, adapts the lytics http json response
func NewResultReader ¶
func NewResultReader(req *Generator) *ResultReader
NewResultReader create a new lytics reader that will handle translation of lytics rest api content into dataux/qlbridge types.
func (*ResultReader) Close ¶
func (m *ResultReader) Close() error
func (*ResultReader) Run ¶
func (m *ResultReader) Run() error
Run() Fetch api response, page through result continuing to send messages.
type ResultReaderNext ¶
type ResultReaderNext struct {
*ResultReader
}
ResultReaderNext is wrapper, allowing us to implement sql/driver Next() interface which is different than qlbridge/datasource Next()
Source Files