Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CSVColumnStreamer ¶
func CSVColumnStreamer(defs map[string]EventColumnDef, records <-chan mixpanel.EventData)
CSVColumnStreamer writes CSVs with explicitly defined events and properties. This is useful if only a subset of the properties attached to an event type are useful or the data needs to be stored in a traditional SQL table with columns known ahead of time.
This will write to a unique io.Writer for each specified event.
The `defs` map contains a mapping of the event names to capture to their EventColumnDefs. Any event received that is not in this map will simply be dropped.
func CSVStreamer ¶
CSVStreamer writes the records passed on the given chan in a schema-less way. An initial header row containing the names of the columns is written first.
Format is:
event_id,key,value
This way, it is possible to GROUP BY event_id to get the full view of a single event.
The reason for this format is because it is not possible to know all of the column names beforehand, and making multiple passes over the data to find a common set of columns is a nonstarter because of the time and memory requirements this requires.
Types ¶
type EventColumnDef ¶
type EventColumnDef struct {
// contains filtered or unexported fields
}
EventColumnDef represents the definition of an event's CSV columns to be passed on to the `CSVColumnStreamer` function.
- `columns` contains the names of the columns.
- `values` represents a row, in the same order as specified by `columns`. This is to avoid creating excessive garbage by allocating and destroying the array on each iteration.
func NewEventColumnDef ¶
func NewEventColumnDef(w io.Writer, columns []string) EventColumnDef
NewEventColumnDef oddly enough creates an instance of the EventColumnDef struct from the given io.Writer and list of column names.
Columns in the output will be in the same order as they passed in here.
Source Files