Documentation
¶
Index ¶
- type CurColumn
- type CurConvert
- func (c *CurConvert) CheckCURExists() error
- func (c *CurConvert) CleanCur() error
- func (c *CurConvert) ConvertCur() error
- func (c *CurConvert) DownloadCur(curObject string) (string, error)
- func (c *CurConvert) GetCURColumns() ([]CurColumn, error)
- func (c *CurConvert) ParquetCur(inputFile string) (string, error)
- func (c *CurConvert) ParseCur() error
- func (c *CurConvert) SetDestKMSKey(key string) error
- func (c *CurConvert) SetDestPath(path string) error
- func (c *CurConvert) SetDestRole(arn string, externalID string) error
- func (c *CurConvert) SetFileConcurrency(concurrency int) error
- func (c *CurConvert) SetSourceManifest(manifest string) error
- func (c *CurConvert) SetSourceRole(arn string, externalID string) error
- func (c *CurConvert) SetTmpLocation(path string) error
- func (c *CurConvert) UploadCur(parquetFile string) error
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CurConvert ¶
type CurConvert struct {
CurColumns []string
CurFiles []string
CurParqetFiles map[string]bool
CurColumnTypes map[string]string
// contains filtered or unexported fields
}
CurConvert class and functions
func NewCurConvert ¶
func NewCurConvert(sBucket string, sObject string, dBucket string, dObject string) *CurConvert
NewCurConvert - Init struct
func (*CurConvert) CheckCURExists ¶
func (c *CurConvert) CheckCURExists() error
CheckCURExists - Attempts to fetch manifest file to confirm existence of CUR
func (*CurConvert) ConvertCur ¶
func (c *CurConvert) ConvertCur() error
ConvertCur - Performs Download, Conversion
func (*CurConvert) DownloadCur ¶
func (c *CurConvert) DownloadCur(curObject string) (string, error)
DownloadCur -
func (*CurConvert) GetCURColumns ¶
func (c *CurConvert) GetCURColumns() ([]CurColumn, error)
GetCURColumns - Converts processed CUR columns into map and returns it
func (*CurConvert) ParquetCur ¶
func (c *CurConvert) ParquetCur(inputFile string) (string, error)
ParquetCur -
func (*CurConvert) ParseCur ¶
func (c *CurConvert) ParseCur() error
ParseCur - Reads JSON manifest file from S3 and adds needed data into struct
func (*CurConvert) SetDestKMSKey ¶
func (c *CurConvert) SetDestKMSKey(key string) error
SetDestKMSKey - sets the KMS Master key arn to use for
func (*CurConvert) SetDestPath ¶
func (c *CurConvert) SetDestPath(path string) error
SetDestPath - configures the dest pat for the converted CUR
func (*CurConvert) SetDestRole ¶
func (c *CurConvert) SetDestRole(arn string, externalID string) error
SetDestRole - configures the source profile data for retrieving CUR from different AWS account
func (*CurConvert) SetFileConcurrency ¶
func (c *CurConvert) SetFileConcurrency(concurrency int) error
SetFileConcurrency - Allows for over-ride of number of CUR files processed concurrently
func (*CurConvert) SetSourceManifest ¶
func (c *CurConvert) SetSourceManifest(manifest string) error
SetCURManifest - configures the source manifest object for retrieving CUR from different AWS account
func (*CurConvert) SetSourceRole ¶
func (c *CurConvert) SetSourceRole(arn string, externalID string) error
SetSourceRole - configures the source profile data for retrieving CUR from different AWS account
func (*CurConvert) SetTmpLocation ¶
func (c *CurConvert) SetTmpLocation(path string) error
SetTmpLocation - sets the temp directory for CUR files to be downloaded to, and parquet files to be written too
Source Files