Documentation
¶
Index ¶
- func NewS3FileReader(ctx context.Context, bucket string, key string, cfgs ...*aws.Config) (source.ParquetFile, error)
- func NewS3FileReaderVersioned(ctx context.Context, bucket string, key string, version *string, ...) (source.ParquetFile, error)
- func NewS3FileReaderVersionedWithClient(ctx context.Context, s3Client s3iface.S3API, bucket string, key string, ...) (source.ParquetFile, error)
- func NewS3FileReaderWithClient(ctx context.Context, s3Client s3iface.S3API, bucket string, key string) (source.ParquetFile, error)
- func NewS3FileWriter(ctx context.Context, bucket string, key string, acl string, ...) (source.ParquetFile, error)
- func NewS3FileWriterWithClient(ctx context.Context, s3Client s3iface.S3API, bucket string, key string, ...) (source.ParquetFile, error)
- func SetActiveSession(sess *session.Session)
- type S3File
- func (s *S3File) Close() error
- func (s *S3File) Create(key string) (source.ParquetFile, error)
- func (s *S3File) Open(name string) (source.ParquetFile, error)
- func (s *S3File) Read(p []byte) (n int, err error)
- func (s *S3File) Seek(offset int64, whence int) (int64, error)
- func (s *S3File) Write(p []byte) (n int, err error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewS3FileReader ¶
func NewS3FileReader(ctx context.Context, bucket string, key string, cfgs ...*aws.Config) (source.ParquetFile, error)
NewS3FileReader creates an S3 FileReader, to be used with NewParquetReader
func NewS3FileReaderVersioned ¶
func NewS3FileReaderVersioned(ctx context.Context, bucket string, key string, version *string, cfgs ...*aws.Config) (source.ParquetFile, error)
NewS3FileReaderVersioned creates an S3 FileReader for a versioned of S3 object, to be used with NewParquetReader
func NewS3FileReaderVersionedWithClient ¶
func NewS3FileReaderVersionedWithClient(ctx context.Context, s3Client s3iface.S3API, bucket string, key string, version *string) (source.ParquetFile, error)
NewS3FileReaderVersionedWithClient is the same as NewS3FileReaderVersioned but allows passing your own S3 client
func NewS3FileReaderWithClient ¶
func NewS3FileReaderWithClient(ctx context.Context, s3Client s3iface.S3API, bucket string, key string) (source.ParquetFile, error)
NewS3FileReaderWithClient is the same as NewS3FileReader but allows passing your own S3 client
func NewS3FileWriter ¶
func NewS3FileWriter( ctx context.Context, bucket string, key string, acl string, uploaderOptions []func(*s3manager.Uploader), cfgs ...*aws.Config, ) (source.ParquetFile, error)
NewS3FileWriter creates an S3 FileWriter, to be used with NewParquetWriter
func NewS3FileWriterWithClient ¶
func NewS3FileWriterWithClient( ctx context.Context, s3Client s3iface.S3API, bucket string, key string, acl string, uploaderOptions []func(*s3manager.Uploader), ) (source.ParquetFile, error)
NewS3FileWriterWithClient is the same as NewS3FileWriter but allows passing your own S3 client.
func SetActiveSession ¶
SetActiveSession sets the current session. If this is unset, the functions of this package will implicitly create a new session.Session and use that. This allows you to control what session is used, particularly useful for testing with a system like localstack.
Types ¶
type S3File ¶
type S3File struct {
BucketName string
Key string
VersionId *string
ACL string
// contains filtered or unexported fields
}
S3File is ParquetFile for AWS S3
func (*S3File) Close ¶
Close signals write completion and cleans up any open streams. Will block until pending uploads are complete.
func (*S3File) Create ¶
func (s *S3File) Create(key string) (source.ParquetFile, error)
Create creates a new S3 File instance to perform writes
func (*S3File) Open ¶
func (s *S3File) Open(name string) (source.ParquetFile, error)
Open creates a new S3 File instance to perform concurrent reads
Source Files
Directories