enigma

package module
v0.0.0-...-8c11eb5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 2, 2015 License: MIT Imports: 8 Imported by: 0

README

Enigma Build Status GoDoc

Enigma.io lets you quickly search and analyze billions of public records published by governments, companies and organizations.

Enigma is a simple go client for the enigma.io API.

Documentation

Full documentation and examples are available in the godoc package index GoDoc

Examples

Client
package main

import (
	enigma "github.com/mohamedattahri/enigma"
)

func main() {
	client := enigma.NewClient("some_api_key")
}
Metadata
Parent
response, err := client.Meta().Parent("us.gov.whitehouse")
if err != nil {
	fmt.Println(err)
	return
}
fmt.Println(response.Info.ChildrenTablesTotal)
Table
response, err := client.Meta().Table("us.gov.whitehouse.visitor-list")
if err != nil {
	fmt.Println(err)
	return
}
fmt.Println(response.Result.DbBoundaryLabel)
Data
response, err := client.Data("us.gov.whitehouse.visitor-list").Select("namefull", "appt_made_date").Sort("namefirst", enigma.Desc).Results()
if err != nil {
	fmt.Println(err)
	return
}
fmt.Println(string(response.Result))
Stats
response, err := client.Stats("us.gov.whitehouse.visitor-list", "total_people").Operation(enigma.Sum).Results()
if err != nil {
	fmt.Println(err)
	return
}

var obj map[string]string
json.Unmarshal(response.Result, &obj)
fmt.Println(obj["sum"])
Export
url, err := client.Export("us.gov.whitehouse.visitor-list").FileURL(nil)
if err != nil {
	fmt.Println(err)
	return
}
fmt.Println(url)

TODO:

More tests.

Documentation

Overview

Package enigma provides developers with a Go client for the Enigma.io API.

The Enigma API allows users to download datasets, query metadata, or perform server side operations on tables in Enigma. All calls to the API are made through a RESTful protocol and require an API key. The Enigma API is served over HTTPS.

About Enigma

Enigma.io (http://enigma.io) lets you quickly search and analyze billions of public records published by governments, companies and organizations.

Reference

Please refer to the official documentation of the API http://app.enigma.io/api for more info.

Example (Data) 
client := enigma.NewClient("some_api_key")
response, err := client.Data("us.gov.whitehouse.visitor-list").Select("namefull", "appt_made_date").Sort("namefirst", enigma.Desc).Results()
if err != nil {
	fmt.Println(err)
	return
}
fmt.Println(string(response.Result))

Output:
Example (Export) 
ready := make(chan string)

client := enigma.NewClient("some_api_key")
_, err := client.Export("us.gov.whitehouse.visitor-list").FileURL(ready)
if err != nil {
	fmt.Println(err)
	return
}

url := <-ready
fmt.Println(url)
// url now points to a ready to download file.

Output:
Example (Meta) 
client := enigma.NewClient("some_api_key")
response, err := client.Meta().Table("us.gov.whitehouse.visitor-list")
if err != nil {
	fmt.Println(err)
	return
}
fmt.Println(response.Result.DbBoundaryLabel)

Output:
Example (Stats) 
client := enigma.NewClient("some_api_key")
response, err := client.Stats("us.gov.whitehouse.visitor-list", "total_people").Operation(enigma.Sum).Results()
if err != nil {
	fmt.Println(err)
	return
}

var obj map[string]string
json.Unmarshal(response.Result, &obj)
fmt.Println(obj["sum"])

Output:

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client of the Enigma API. Use NewClient to instantiate a new instance as in the following example: 

client := enigma.NewClient("some_api_key")

func NewClient 

func NewClient(key string) *Client

NewClient instantiates a new Client instance with a given API key.

func (*Client) Data 

func (client *Client) Data(datapath string) *DataQuery

Data queries the content of table datapaths. Data queries may be filtered, sorted and paginated using the returned query object.

For large tables and tables with a large number of columns, data API calls may take some time to complete. API users are advised to make use of the Select() and/or Limit() whenever possible to improve performance.

Build a query by chaining up parameters, then call Results() to actually perform the query. 

client.Data("us.gov.whitehouse.visitor-list").Select("namefull", "appt_made_date").Sort("namefirst", enigma.Desc).Results()

func (*Client) Export 

func (client *Client) Export(datapath string) *ExportQuery

Export requests exports of table datapaths as GZiped files. When the export API is called, an export is queued and the API immediately returns a URL pointing to the future location of the exported file.

Build a query by chaining up parameters, then call FileURL() to perform the query and get the Url of the file to download. 

client.Export("us.gov.whitehouse.visitor-list").Select("namefull").Sort("namefull", Asc).FileURL(nil)

func (*Client) Meta 

func (client *Client) Meta() *MetaQuery

Meta can be used to query all datapaths for their metadata.

func (*Client) Stats 

func (client *Client) Stats(datapath, column string) *StatsQuery

Stats queries table datapaths by column for statistics on the data they contain. Like data queries, stats queries may be filtered, sorted and paginated using the returned query object.

Build a query by chaining up parameters, then call Results() to actually perform the query. 

client.Stats("us.gov.whitehouse.visitor-list", "total_people").Operation(enigma.Sum).Results()

type Conjunction 

type Conjunction string

Conjunction represents the logical link between multiple search or where parameters. 

const (
	Or  Conjunction = "or"
	And Conjunction = "and"
)

Valid conjunctions

type DataQuery 

type DataQuery query

DataQuery queries table datapaths for the data they contain. Data queries may be filtered, sorted and paginated using the provided URL parameters.

func (*DataQuery) Conjunction 

func (q *DataQuery) Conjunction(conjunction Conjunction) *DataQuery

Conjunction is only applicable when more than one Search() or Where() parameter is provided. Defaults to And.

func (*DataQuery) Limit 

func (q *DataQuery) Limit(number int) *DataQuery

Limit the number of rows returned (max. 500). Defaults to 500.

func (*DataQuery) Page 

func (q *DataQuery) Page(number int) *DataQuery

Page paginates row results and return the nth page of results. Pages are calculated based on the current limit, which defaults to 500.

func (*DataQuery) Results 

func (q *DataQuery) Results() (response DataResponse, err error)

Results or error returned by the server.

func (*DataQuery) Search 

func (q *DataQuery) Search(query string) *DataQuery

Search filters the results by only returning rows that match a query. Multiple search parameters may be provided.

By default this searches the entire table for matching text.

To search particular fields only, use the query format "@fieldname query".

To match multiple queries within a single search parameter, the | (or) operator can be used eg. "DataQuery1|DataQuery2".

func (*DataQuery) Select 

func (q *DataQuery) Select(columns ...string) *DataQuery

Select the columns to be returned with each row. Default is to return all columns.

func (*DataQuery) Sort 

func (q *DataQuery) Sort(column string, direction SortDirection) *DataQuery

Sort rows by a particular column in a given direction.

func (*DataQuery) Where 

func (q *DataQuery) Where(query string) *DataQuery

Where filters results with a SQL-style "where" clause. Only applies to numerical and date columns – use the "search" parameter for strings. Multiple where parameters may be provided.

Query format: <column><operator><value>

Valid operators: >=, >, =, !=, <, and <=.

<column> [not] in (<value>,<value>,...) Match rows where column matches one of the provided values.

<column> [not] between <value> and <value> Match rows where column lies within range provided (inclusive).

type DataResponse 

type DataResponse struct {
	DataPath string          `json:"data_path"`
	Result   json.RawMessage `json:"result"`
	Info     struct {
		RowsLimit    int `json:"rows_limit"`
		CurrentPage  int `json:"current_page"`
		TotalPages   int `json:"total_pages"`
		TotalResults int `json:"total_results"`
	} `json:"info"`
}

DataResponse attributes

type ExportQuery 

type ExportQuery query

ExportQuery queries data tables to produce a file that can be downloaded.

func (*ExportQuery) Conjunction 

func (q *ExportQuery) Conjunction(conjunction Conjunction) *ExportQuery

Conjunction is only applicable when more than one Search() or Where() parameter is provided. Defaults to And.

func (*ExportQuery) FileURL 

func (q *ExportQuery) FileURL(ready chan string) (url string, err error)

FileURL returns the URL of the GZip file containing the exported data.

Passing the ready chan will poll the returned URL until the file is ready for take out. The url pushed down the channel should be used to download the file.

Passing nil will simply return the url of the file to download. 

ready := make(chan string)
_, err := client.Export("us.gov.whitehouse.visitor-list").FileURL(ready)
if err != nil {
	fmt.Println(err)
	return
}
downloadUrl := <- ready

func (*ExportQuery) Page 

func (q *ExportQuery) Page(number int) *ExportQuery

Page paginates row results and returns the nth page of results. Pages are calculated based on the current limit, which defaults to 500.

func (*ExportQuery) Search 

func (q *ExportQuery) Search(query string) *ExportQuery

Search filters results by only returning rows that match a search query. Multiple search parameters may be provided.

By default this searches the entire table for matching text. To search particular fields only, use the DataQuery format "@fieldname DataQuery".

To match multiple queries within a single search parameter, the | (or) operator can be used eg. "query1|query2".

func (*ExportQuery) Select 

func (q *ExportQuery) Select(columns ...string) *ExportQuery

Select the list of columns to be returned with each row. Default is to return all columns.

func (*ExportQuery) Sort 

func (q *ExportQuery) Sort(column string, direction SortDirection) *ExportQuery

Sort rows by a particular column in a given direction. Asc denotes ascending order, Desc denotes descending.

func (*ExportQuery) Where 

func (q *ExportQuery) Where(query string) *ExportQuery

Where filters results with a SQL-style "where" clause. Only applies to numerical and date columns – use the "search" parameter for strings. Multiple where parameters may be provided.

Query format: <column><operator><value>

Valid operators: >=, >, =, !=, <, and <=.

<column> [not] in (<value>,<value>,...) Match rows where column matches one of the provided values.

<column> [not] between <value> and <value> Match rows where column lies within range provided (inclusive).

type MetaParentNodeResponse 

type MetaParentNodeResponse struct {
	DataPath string `json:"data_path"`
	Result   struct {
		Path []struct {
			Level       string `json:"level"`
			Label       string `json:"label"`
			Description string `json:"description"`
		} `json:"path"`
		ImmediateNodes []struct {
			Datapath    string `json:"datapath"`
			Label       string `json:"label"`
			Description string `json:"description"`
		} `json:"immediate_nodes"`
		ChildrenTables []struct {
			Datapath         string `json:"datapath"`
			Label            string `json:"label"`
			Description      string `json:"description"`
			DbBoundaryLabel  string `json:"db_boundary_label"`
			DbBoundaryTables string `json:"db_boundary_tables"`
		} `json:"children_tables"`
	} `json:"result"`
	Info struct {
		ResultType          string `json:"result_type"`
		ChildrenTablesLimit int    `json:"children_tables_limit"`
		ChildrenTablesTotal int    `json:"children_tables_total"`
		CurrentPage         int    `json:"current_page"`
		TotalPages          int    `json:"total_pages"`
	} `json:"info"`
}

MetaParentNodeResponse represents the structure of a metadata response describing a parent node.

type MetaQuery 

type MetaQuery query

MetaQuery can be used on all datapaths to query their metadata.

func (*MetaQuery) Parent 

func (q *MetaQuery) Parent(datapath string) (response *MetaParentNodeResponse, err error)

Parent metadata request for the given datapath.

func (*MetaQuery) Table 

func (q *MetaQuery) Table(datapath string) (response *MetaTableNodeResponse, err error)

Table metadata request for the given datapath.

type MetaTableNodeResponse 

type MetaTableNodeResponse struct {
	DataPath string `json:"datapath"`
	Result   struct {
		Path []struct {
			Level       string `json:"level"`
			Label       string `json:"label"`
			Description string `json:"description"`
		} `json:"path"`
		Columns []struct {
			ID          string `json:"id"`
			Label       string `json:"label"`
			Description string `json:"description"`
			Type        string `json:"type"`
			Index       int    `json:"index"`
		} `json:"columns"`
		DbBoundaryDatapath string `json:"db_boundary_datapath"`
		DbBoundaryLabel    string `json:"db_boundary_label"`
		DbBoundaryTables   []struct {
			Datapath string `json:"datapath"`
			Label    string `json:"label"`
		} `json:"db_boundary_tables"`
		AncestorDatapaths []string `json:"ancestor_datapaths"`
		Documents         []struct {
			URL   string `json:"url"`
			Title string `json:"title"`
			Type  string `json:"type"`
		} `json:"documents"`
		Metadata []struct {
			Value string `json:"value"`
			Label string `json:"label"`
		} `json:"metadata"`
	} `json:"result"`
	Info struct {
		ResultType string `json:"result_type"`
	} `json:"info"`
}

MetaTableNodeResponse represents the structure of a metadata response describing a table.

type Operation 

type Operation string

Operation represents a calculation that a stats request can perform on a selected column. 

const (
	Sum       Operation = "sum"
	Avg       Operation = "avg"
	StdDev    Operation = "stddev"
	Variance  Operation = "variance"
	Max       Operation = "max"
	Min       Operation = "min"
	Frequency Operation = "frequency"
)

Valid stat operations

type SortDirection 

type SortDirection string

SortDirection represents the direction in which a selected column or calculation result should be sorted. 

const (
	// Asc for ascending order
	Asc SortDirection = "+"
	// Desc for descending order
	Desc SortDirection = "-"
)

type StatsQuery 

type StatsQuery query

StatsQuery can be used to query columns of tables for statistics on the data they contain. Like data queries, stats queries may be filtered, sorted and paginated using the provided URL parameters.

func (*StatsQuery) By 

func (q *StatsQuery) By(operation Operation) *StatsQuery

By indicates the compound operation to run on a given pair of columns. Valid compound operations are sum and avg.

When running a compound operation query, the Of() parameter is required (see below).

func (*StatsQuery) Conjunction 

func (q *StatsQuery) Conjunction(conjunction Conjunction) *StatsQuery

Conjunction is only applicable when more than one Search() or Where() parameter is provided. Defaults to And.

func (*StatsQuery) Limit 

func (q *StatsQuery) Limit(limit int) *StatsQuery

Limit the number of frequency, compound sum, or compound average results returned (max. 500). Defaults to 500.

func (*StatsQuery) Of 

func (q *StatsQuery) Of(column string) *StatsQuery

Of indicates the numerical column to compare against when running a compound operation.

Required when using the By() parameter. Must be a numerical column.

func (*StatsQuery) Operation 

func (q *StatsQuery) Operation(operation Operation) *StatsQuery

Operation to run the given column.

For a numerical column, valid operations are Sum, Avg, StdDev, Variance, Max, Min and Frequency.

For a date column, valid operations are max,min and frequency.

For all other columns, the only valid operation is frequency.

Defaults to all available operations based on the column's type.

func (*StatsQuery) Page 

func (q *StatsQuery) Page(number int) *StatsQuery

Page paginates row results and returns the nth page of results. Pages are calculated based on the current limit, which defaults to 500.

func (*StatsQuery) Results 

func (q *StatsQuery) Results() (response *StatsResponse, err error)

Results or error returned by the server.

func (*StatsQuery) Search 

func (q *StatsQuery) Search(query string) *StatsQuery

Search filters results by only returning rows that match a search query. Multiple search parameters may be provided. By default this searches the entire table for matching text.

To search particular fields only, use the StatsQuery format "@fieldname StatsQuery".

To match multiple queries within a single search parameter, the | (or) operator can be used eg. "StatsQuery1|StatsQuery2".

func (*StatsQuery) Sort 

func (q *StatsQuery) Sort(direction SortDirection) *StatsQuery

Sort rows by a particular column in a given direction. Asc denotes ascending order, Desc denotes descending.

func (*StatsQuery) Where 

func (q *StatsQuery) Where(query string) *StatsQuery

Where filters results with a SQL-style "where" clause. Only applies to numerical and date columns – use the Search() for strings. Multiple where parameters may be provided.

Query format: <column><operator><value>

Valid operators: >=, >, =, !=, <, and <=.

<column> [not] in (<value>,<value>,...) Match rows where column matches one of the provided values.

<column> [not] between <value> and <value> Match rows where column lies within range provided (inclusive).

type StatsResponse 

type StatsResponse struct {
	DataPath string          `json:"data_path"`
	Result   json.RawMessage `json:"result"`
	Info     struct {
		Column       interface{} `json:"column"`
		Operations   []Operation `json:"operations"`
		RowsLimit    int         `json:"rows_limit"`
		CurrentPage  int         `json:"current_page"`
		TotalPages   int         `json:"total_pages"`
		TotalResults int         `json:"total_results"`
	} `json:"info"`
}

StatsResponse represents the response returned from a Stats endpoint.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.