go-soda: github.com/SebastiaanKlippert/go-soda Index | Files

package soda

import "github.com/SebastiaanKlippert/go-soda"

Package soda provides HTTP GET tools for SODA (Socrata Open Data API) webservices, see http://dev.socrata.com/


Package Files

metadata.go soda.go


var ErrDone = errors.New("Done")

ErrDone is returned by OffsetGetRequest.Next when done

type Column Uses

type Column struct {
    DataTypeName   string `json:"dataTypeName"`
    FieldName      string `json:"fieldName"`
    Format         Format `json:"format"`
    ID             int    `json:"id"`
    Name           string `json:"name"`
    Position       int    `json:"position"`
    RenderTypeName string `json:"renderTypeName"`
    TableColumnID  int    `json:"tableColumnId"`
    Width          int    `json:"width"`

Column describes one data column

type Direction Uses

type Direction bool

Direction is used to set the sort direction to ascending or descending

const (
    // DirAsc is used to set ascending sort order
    DirAsc Direction = false

    // DirDesc is used to set descending sort order
    DirDesc Direction = true

type Format Uses

type Format struct {
    PrecisionStyle string `json:"precisionStyle"`
    Align          string `json:"align"`
    NoCommas       string `json:"noCommas"`

Format describes column formats

type GetRequest Uses

type GetRequest struct {
    Format     string //json, csv etc
    Filters    SimpleFilters
    Query      SoSQL
    Metadata   metadata
    HTTPClient *http.Client //For clients who need a custom HTTP client
    // contains filtered or unexported fields

GetRequest is a wrapper/container for SODA requests. This is NOT safe for use by multiple goroutines as Format, Filters and Query will be overwritten. Create a new GetRequest in each goroutine you use or use an OffsetGetRequest

func NewGetRequest Uses

func NewGetRequest(endpoint, apptoken string) *GetRequest

NewGetRequest creates a new GET request, the endpoint must be specified without the format. For example https://data.ct.gov/resource/hma6-9xbg

func (*GetRequest) Count Uses

func (r *GetRequest) Count() (uint, error)

Count gets the total number of records in the dataset by executing a SODA request

func (*GetRequest) Fields Uses

func (r *GetRequest) Fields() ([]string, error)

Fields returns all the fields present in the dataset (ignores select fields). Spaces in fieldnames are replaced by underscores.

func (*GetRequest) Get Uses

func (r *GetRequest) Get() (*http.Response, error)

Get executes the HTTP GET request

func (*GetRequest) GetEndpoint Uses

func (r *GetRequest) GetEndpoint() string

GetEndpoint returns the complete SODA URL with format

func (*GetRequest) Modified Uses

func (r *GetRequest) Modified() (time.Time, error)

Modified returns when the dataset was last updated

func (*GetRequest) URLValues Uses

func (r *GetRequest) URLValues() url.Values

URLValues returns the url.Values for the GetRequest

type Metadata Uses

type Metadata struct {
    AverageRating int       `json:"averageRating"`
    Category      string    `json:"category"`
    Columns       []Column  `json:"columns"`
    CreatedAt     Timestamp `json:"createdAt"`
    DisplayType   string    `json:"displayType"`
    DownloadCount int       `json:"downloadCount"`
    Flags         []string  `json:"flags"`
    Grants        []struct {
        Flags     []string `json:"flags"`
        Inherited bool     `json:"inherited"`
        Type      string   `json:"type"`
    }   `json:"grants"`
    ID             string    `json:"id"`
    IndexUpdatedAt Timestamp `json:"indexUpdatedAt"`
    License        struct {
        Name string `json:"name"`
    }   `json:"license"`
    LicenseID string `json:"licenseId"`
    Metadata  struct {
        AvailableDisplayTypes []string `json:"availableDisplayTypes"`
        CustomFields          struct {
            Licentie struct {
                Licentie string `json:"Licentie"`
            } `json:"Licentie"`
        }   `json:"custom_fields"`
        RdfSubject       string `json:"rdfSubject"`
        RenderTypeConfig struct {
            Visible struct {
                Table bool `json:"table"`
            } `json:"visible"`
        }   `json:"renderTypeConfig"`
        RowLabel string `json:"rowLabel"`
    }   `json:"metadata"`
    Name             string `json:"name"`
    NewBackend       bool   `json:"newBackend"`
    NumberOfComments int    `json:"numberOfComments"`
    Oid              int    `json:"oid"`
    Owner            struct {
        DisplayName string   `json:"displayName"`
        ID          string   `json:"id"`
        Rights      []string `json:"rights"`
        RoleName    string   `json:"roleName"`
        ScreenName  string   `json:"screenName"`
    }   `json:"owner"`
    PublicationAppendEnabled bool      `json:"publicationAppendEnabled"`
    PublicationDate          Timestamp `json:"publicationDate"`
    PublicationGroup         int       `json:"publicationGroup"`
    PublicationStage         string    `json:"publicationStage"`
    Query                    struct{}  `json:"-"` //TODO
    Ratings                  struct {
        Rating int `json:"rating"`
    }   `json:"ratings"`
    Rights        []string  `json:"rights"`
    RowsUpdatedAt Timestamp `json:"rowsUpdatedAt"`
    RowsUpdatedBy string    `json:"rowsUpdatedBy"`
    TableAuthor   struct {
        DisplayName string   `json:"displayName"`
        ID          string   `json:"id"`
        Rights      []string `json:"rights"`
        RoleName    string   `json:"roleName"`
        ScreenName  string   `json:"screenName"`
    }   `json:"tableAuthor"`
    TableID          int       `json:"tableId"`
    Tags             []string  `json:"tags"`
    TotalTimesRated  int       `json:"totalTimesRated"`
    ViewCount        int       `json:"viewCount"`
    ViewLastModified Timestamp `json:"viewLastModified"`
    ViewType         string    `json:"viewType"`

Metadata contains the resource metadata

type OffsetGetRequest Uses

type OffsetGetRequest struct {
    // contains filtered or unexported fields

OffsetGetRequest is a request getter that gets all the records using the filters and limits from gr and is safe to use by multiple goroutines, use Next(number) to get the next number of records. A sync.WaitGroup is embedded for easy concurrency.

func NewOffsetGetRequest Uses

func NewOffsetGetRequest(gr *GetRequest) (*OffsetGetRequest, error)

NewOffsetGetRequest creates a new OffsetGetRequest from gr and does a count request to determine the number of records to get

func (*OffsetGetRequest) Count Uses

func (o *OffsetGetRequest) Count() uint

Count returns the number of records from memory

func (*OffsetGetRequest) IsDone Uses

func (o *OffsetGetRequest) IsDone() bool

IsDone returns if we have gotten all records

func (*OffsetGetRequest) Next Uses

func (o *OffsetGetRequest) Next(number uint) (*http.Response, error)

Next gets the next number of records

type SimpleFilters Uses

type SimpleFilters map[string]string

SimpleFilters is the easiest way to filter columns for equality. Add the column to filter on a map key and the filter value as map value. If you include multiple filters, the filters will be combined using a boolean AND. See http://dev.socrata.com/docs/filtering.html

func (SimpleFilters) URLValues Uses

func (sf SimpleFilters) URLValues() url.Values

URLValues returns the url.Values for the SimpleFilters

type SoSQL Uses

type SoSQL struct {
    Select []string //The set of columns to be returned. Default: All columns, equivalent to $select=*
    Where  string   //Filters the rows to be returned. Default: No filter, and returning a max of $limit values
    Order  []struct {
        Column string //Column name
        Desc   bool   //Descending. Default: false = Ascending
    }   //Specifies the order of results. Default: Unspecified order, but it will be consistent across paging
    Group  string //Column to group results on, similar to SQL Grouping. Default: No grouping
    Limit  uint   //Maximum number of results to return. Default: 1000 (with a maximum of 50,000)
    Offset uint   //Offset count into the results to start at, used for paging. Default: 0
    Q      string //Performs a full text search for a value. Default: No search


SoSQL implements the Socrata Query Language and is used to build more complex queries. See http://dev.socrata.com/docs/queries.html

func (*SoSQL) AddOrder Uses

func (sq *SoSQL) AddOrder(column string, dir Direction)

AddOrder can be called for each field you want to sort the result on. If parameter descending is true, the column will be sorted descending, or ascending if false.

func (*SoSQL) ClearOrder Uses

func (sq *SoSQL) ClearOrder()

ClearOrder removes all order fields

func (*SoSQL) URLValues Uses

func (sq *SoSQL) URLValues() url.Values

URLValues returns the url.Values for the SoSQL query

type Timestamp Uses

type Timestamp time.Time

Timestamp is a time.Time struct unmarshalled from a unix epoch time

func (Timestamp) Time Uses

func (t Timestamp) Time() time.Time

Time returns t as time.Time

func (*Timestamp) UnmarshalJSON Uses

func (t *Timestamp) UnmarshalJSON(b []byte) error

UnmarshalJSON sets t from a timestamp

Package soda imports 12 packages (graph). Updated 2020-07-24. Refresh now. Tools for package owners.