awsmgr

func (*AwsManager) CancelSpotRequests ¶

func (awsm *AwsManager) CancelSpotRequests(
	clt *ec2.Client,
	spotRequestTag string,
) error

TerminateSpotRequests will search for spot request(s) with the given tag and cancel them

func (*AwsManager) DownloadFromS3 ¶

func (am *AwsManager) DownloadFromS3(d common.S3Download) error

DownloadFromS3 downloads an object from an S3 bucket

func (*AwsManager) DownloadMultipleFromS3 ¶

func (am *AwsManager) DownloadMultipleFromS3(
	downloads []common.S3Download,
) error

DownloadMultipleFromS3 will run multiple downloads in parallel to speed up the process when downloading many files. The number of CPUs in the system is used as a parallelism limit. If an error occurred with one of the downloads, the function will return the first error that occurred.

func (*AwsManager) FileExistsOnS3 ¶

func (am *AwsManager) FileExistsOnS3(
	region, bucket, path string,
) (bool, error)

FileExistsOnS3 checks if a file in the given region, bucket and path exists in S3. The return value indicates existence (true/false). In case an error occurs during checking the file's existence, it will be returned

func (*AwsManager) ForceRefreshLaunchTemplates ¶

func (am *AwsManager) ForceRefreshLaunchTemplates()

ForceRefreshLaunchTemplates is a method to force refreshing the launch templates. This is currently unused but could be hooked up to a REST API to allow refreshing this by force from the UI

func (*AwsManager) ForceRefreshSubnets ¶

func (am *AwsManager) ForceRefreshSubnets()

ForceRefreshSubnets is a method to force refreshing the subnets This is currently unused but could be hooked up to a REST API to allow refreshing this by force from the UI

func (*AwsManager) GenerateSeed ¶

func (am *AwsManager) GenerateSeed(seed ShardSeed) error

func (*AwsManager) GetAllRegions ¶

func (am *AwsManager) GetAllRegions() []string

GetAllRegions returns a list of available regions. This now returns the hard coded array, but in the future the array could be periodically re-read using whatever discovery method we find for the regions we want enabled.

func (*AwsManager) GetAvailableSeeds ¶

func (am *AwsManager) GetAvailableSeeds() []*ShardSeed

func (*AwsManager) GetLaunchTemplate ¶

func (am *AwsManager) GetLaunchTemplate(
	templateID string,
) (AwsLaunchTemplate, error)

GetLaunchTemplate returns a single launch template identified by its ID or an error if it was not found in the set of available launch templates

func (*AwsManager) GetLaunchTemplateRegion ¶

func (am *AwsManager) GetLaunchTemplateRegion(launchTemplateID string) string

GetLaunchTemplateRegion will return the AWS region for a given launch template

func (*AwsManager) GetSubnetsForRegion ¶

func (am *AwsManager) GetSubnetsForRegion(region string) []AwsSubnet

GetSubnetsForRegion returns the subnets available in the given region

func (*AwsManager) GetVCPULimit ¶

func (am *AwsManager) GetVCPULimit(key string) int32

GetVCPULimit will return the limit of number of VCPUs for a certain `key` where key is `<region>-[spot|ondem]`, for instance 'us-east-1-spot' describes how many vcpus we can launch in spot markets in region us-east-1

func (*AwsManager) HasSeed ¶

func (am *AwsManager) HasSeed(seed ShardSeed, pending bool) (bool, error)

func (*AwsManager) KillAllInstances ¶

func (am *AwsManager) KillAllInstances() error

KillAllInstances will terminate all known running instances in EC2

func (*AwsManager) LaunchTemplates ¶

func (am *AwsManager) LaunchTemplates() []AwsLaunchTemplate

LaunchTemplates will return the available launch templates or an empty array if AWS was not enabled

func (*AwsManager) ListObjectsInS3 ¶

func (am *AwsManager) ListObjectsInS3(
	region, bucket, prefix string,
) ([]string, error)

ListObjectsInS3 will scan a bucket with a particular prefix and return all the objects that match the prefix

func (*AwsManager) ReadFromS3 ¶

func (am *AwsManager) ReadFromS3(
	d common.S3Download,
	tail int,
) ([]byte, error)

ReadFromS3 fetches an object as byte slice from an S3 bucket. If tail is -1 it iwll read the entire object - if it is higher than -1 it will only read the last tail bytes of the object

func (*AwsManager) RunEC2ForAllRegions ¶

func (am *AwsManager) RunEC2ForAllRegions(
	f func(e *ec2.Client, r string) error,
) error

RunEC2ForAllRegions will execute a function for each enabled region, passing in an EC2 client configured for that region and wait for the function to complete for all regions. If an error occurs for any region, it is returned to the caller

func (*AwsManager) RunSQForAllRegions ¶

func (am *AwsManager) RunSQForAllRegions(
	f func(e *servicequotas.Client, r string) error,
) error

RunSQForAllRegions will execute a function for each enabled region, passing in an SQ client configured for that region and wait for the function to complete for all regions. If an error occurs for any region, it is returned to the caller

func (*AwsManager) RunningInstances ¶

func (am *AwsManager) RunningInstances() []*AwsInstance

RunningInstances returns the currently running instances in EC2

func (*AwsManager) StartNewAgents ¶

func (am *AwsManager) StartNewAgents(
	templateIDs []string,
	testRunID string,
) ([]*AwsInstance, []error)

StartNewAgents is the main logic that spawns our new instances. We pass it an array of templateIDs to launch. For each instance we want to launch there is one entry in the array. So there can be many entries of the same launch template in this array - each entry in the list will result in an instance launched. The return array will contain the instances that we launched in the exact same sequence as the templateIDs that were passed.

func (*AwsManager) StopAgents ¶

func (am *AwsManager) StopAgents(a []*AwsInstance) error

StopAgents will terminate the EC2 instances by the instance objects passed

func (*AwsManager) StopAgentsByInstanceIds ¶

func (am *AwsManager) StopAgentsByInstanceIds(ids []string) error

StopAgentsByInstanceIds will terminate the EC2 instances specified in the ids array. It will enumerate the runningInstances and match their ID to the IDs in the passed array, and then call StopAgents()

func (*AwsManager) UploadToS3 ¶

func (am *AwsManager) UploadToS3(d common.S3Upload) error

UploadToS3 will upload a local file to a bucket and key in S3

func (*AwsManager) UploadToS3IfNotExists ¶

func (am *AwsManager) UploadToS3IfNotExists(d common.S3Upload) error

UploadToS3IfNotExists will check a file's existence and skip uploading if the file already exists

func (*ShardSeed) Equals ¶

func (s *ShardSeed) Equals(s2 *ShardSeed) bool