curconvert

package
v0.0.0-...-f227794 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 3, 2020 License: MIT Imports: 21 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CurColumn

type CurColumn struct {
	Name string
	Type string
}

type CurConvert

type CurConvert struct {
	CurColumns     []string
	CurFiles       []string
	CurParqetFiles map[string]bool
	CurColumnTypes map[string]string
	// contains filtered or unexported fields
}

CurConvert class and functions

func NewCurConvert

func NewCurConvert(sBucket string, sObject string, dBucket string, dObject string) *CurConvert

NewCurConvert - Init struct

func (*CurConvert) CheckCURExists

func (c *CurConvert) CheckCURExists() error

CheckCURExists - Attempts to fetch manifest file to confirm existence of CUR

func (*CurConvert) CleanCur

func (c *CurConvert) CleanCur() error

CleanCUr

func (*CurConvert) ConvertCur

func (c *CurConvert) ConvertCur() error

ConvertCur - Performs Download, Conversion

func (*CurConvert) DownloadCur

func (c *CurConvert) DownloadCur(curObject string) (string, error)

DownloadCur -

func (*CurConvert) GetCURColumns

func (c *CurConvert) GetCURColumns() ([]CurColumn, error)

GetCURColumns - Converts processed CUR columns into map and returns it

func (*CurConvert) ParquetCur

func (c *CurConvert) ParquetCur(inputFile string) (string, error)

ParquetCur -

func (*CurConvert) ParseCur

func (c *CurConvert) ParseCur() error

ParseCur - Reads JSON manifest file from S3 and adds needed data into struct

func (*CurConvert) SetDestKMSKey

func (c *CurConvert) SetDestKMSKey(key string) error

SetDestKMSKey - sets the KMS Master key arn to use for

func (*CurConvert) SetDestPath

func (c *CurConvert) SetDestPath(path string) error

SetDestPath - configures the dest pat for the converted CUR

func (*CurConvert) SetDestRole

func (c *CurConvert) SetDestRole(arn string, externalID string) error

SetDestRole - configures the source profile data for retrieving CUR from different AWS account

func (*CurConvert) SetFileConcurrency

func (c *CurConvert) SetFileConcurrency(concurrency int) error

SetFileConcurrency - Allows for over-ride of number of CUR files processed concurrently

func (*CurConvert) SetSourceManifest

func (c *CurConvert) SetSourceManifest(manifest string) error

SetCURManifest - configures the source manifest object for retrieving CUR from different AWS account

func (*CurConvert) SetSourceRole

func (c *CurConvert) SetSourceRole(arn string, externalID string) error

SetSourceRole - configures the source profile data for retrieving CUR from different AWS account

func (*CurConvert) SetTmpLocation

func (c *CurConvert) SetTmpLocation(path string) error

SetTmpLocation - sets the temp directory for CUR files to be downloaded to, and parquet files to be written too

func (*CurConvert) UploadCur

func (c *CurConvert) UploadCur(parquetFile string) error

UploadCur -

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL