csv

type OpenStringer ¶

type OpenStringer interface {
	fmt.Stringer
	Opener
}

OpenStringer is an Opener which also has a String method which should return the name of the resource being opened (e.g. a file or URL).

type Opener ¶

type Opener interface {
	Open() (io.ReadCloser, error)
}

Opener is an interface to a resource which can be repeatedly Opened (and the returned ReadCloser can be subsequently read). Each call to Open should return a ReadCloser which reads from the beginning of the resource. In the case of an error while reading, Open will be called again to retry reading the entire resource.

type Option ¶

type Option func(*Source)

Option is a functional option to pass to NewSource.

func WithConcurrency(c int) Option

func WithMaxRetries(maxRetries int) Option

func WithOpenStringers(os []OpenStringer) Option

func WithURLs(urls []string) Option

type Source ¶

type Source struct {
	// contains filtered or unexported fields
}

Source satisfies the PDK.Source interface for CSV data. Each line in a CSV file will be returned by a call to record as a map[string]string where the keys are taken from the first line of the CSV. Source is safe for concurrent use.

The Source takes care of retrying failed reads/downloads and making sure not to return duplicate data. TODO: this functionality needs more testing.

func NewSource(options ...Option) *Source

func (c *Source) Record() (interface{}, error)