Documentation
¶
Overview ¶
const CSV = "foo,bar,10,01/02/2006"
type Row struct {
Foo string
Bar string
Amount float44
Date time.Time `csv:"01/02/2006"`
}
reader := csv.NewReader(strings.NewReader(CSV))
var rows []Row err := NewDecoder(reader).Decode(&rows)
Package to interact with CSV files, finding, reading, processing. Uses pipeline methodoligy to implement funcations that can be composed together to create complex pipelines
Index ¶
- Constants
- func IsCSV(filename string) bool
- func IsTSV(filename string) bool
- func ParseS3Path(s3Path string) (string, string, error)
- func Stream(reader io.Reader, header bool, terminator rune, logger *zl.Logger) (<-chan []string, error)
- func StreamCSV(reader io.Reader, header bool, logger *zl.Logger) (<-chan []string, error)
- func StreamDirectory(path string, header bool, logger *zl.Logger) (<-chan []string, error)
- func StreamTSV(reader io.Reader, header bool, logger *zl.Logger) (<-chan []string, error)
- func WalkDirectory(directory string, logger *zl.Logger) (<-chan string, error)
- func WalkDirectoryFilter(directory string, filter FilterFunc, logger *zl.Logger) (<-chan string, error)
- type Decoder
- type FilterFunc
- type PipelineFunc
- type S3Reader
- func (t *S3Reader) List(bucket, prefix string) (<-chan string, error)
- func (t *S3Reader) Stream(bucket, path string, header bool) (<-chan []string, error)
- func (t *S3Reader) WithFilter(fileFilter FilterFunc) *S3Reader
- func (t *S3Reader) WithPipeline(fns ...PipelineFunc) *S3Reader
- func (t *S3Reader) WithTerminator(terminator rune) *S3Reader
Constants ¶
View Source
const ( Comma = ',' Tab = '\t' )
Variables ¶
This section is empty.
Functions ¶
func Stream ¶
func Stream(reader io.Reader, header bool, terminator rune, logger *zl.Logger) (<-chan []string, error)
Stream streams a csv file row by row attempts to close the reader if possible
func StreamDirectory ¶
func WalkDirectory ¶
WalkDirectory looking for CSV files as defined by IsCSV function. Use WalkDirectoryFilter if you need to specify a custom extension.
func WalkDirectoryFilter ¶
func WalkDirectoryFilter(directory string, filter FilterFunc, logger *zl.Logger) (<-chan string, error)
WalkDirectoryFilter walks directory looking for files that match the FilterFunc
Types ¶
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
func NewDecoder ¶
func (*Decoder) SetDateFormat ¶
type PipelineFunc ¶
PipelineFunc process a row to add information based on filename
type S3Reader ¶
type S3Reader struct {
// contains filtered or unexported fields
}
func (*S3Reader) WithFilter ¶
func (t *S3Reader) WithFilter(fileFilter FilterFunc) *S3Reader
func (*S3Reader) WithPipeline ¶
func (t *S3Reader) WithPipeline(fns ...PipelineFunc) *S3Reader
func (*S3Reader) WithTerminator ¶
Source Files
Click to show internal directories.
Click to hide internal directories.