sharding

package
v2.10.9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 30, 2024 License: Apache-2.0 Imports: 22 Imported by: 3

Documentation

Index

Constants

View Source
const ShardControllerMappingKey = "shardControllerMapping"

Variables

View Source
var (
	HeartbeatDuration = env.ParseNumFromEnv(common.EnvControllerHeartbeatTime, 10, 10, 60)
	HeartbeatTimeout  = 3 * HeartbeatDuration
)

Functions

func GetOrUpdateShardFromConfigMap added in v2.9.0

func GetOrUpdateShardFromConfigMap(kubeClient kubernetes.Interface, settingsMgr *settings.SettingsManager, replicas, shard int) (int, error)

GetOrUpdateShardFromConfigMap finds the shard number from the shard mapping configmap. If the shard mapping configmap does not exist, the function creates the shard mapping configmap. The function takes the shard number from the environment variable (default value -1, if not set) and passes it to this function. If the shard value passed to this function is -1, that is, the shard was not set as an environment variable, we default the shard number to 0 for computing the default config map.

func InferShard

func InferShard() (int, error)

InferShard extracts the shard index based on its hostname.

Types

type ClusterFilterFunction added in v2.8.0

type ClusterFilterFunction func(c *v1alpha1.Cluster) bool

func GetClusterFilter

func GetClusterFilter(db db.ArgoDB, distributionFunction DistributionFunction, replicas, shard int) ClusterFilterFunction

GetClusterFilter returns a ClusterFilterFunction which is a function taking a cluster as a parameter and returns wheter or not the cluster should be processed by a given shard. It calls the distributionFunction to determine which shard will process the cluster, and if the given shard is equal to the calculated shard the function will return true.

type ClusterSharding added in v2.10.0

type ClusterSharding struct {
	Shard    int
	Replicas int
	Shards   map[string]int
	Clusters map[string]*v1alpha1.Cluster
	// contains filtered or unexported fields
}

func (*ClusterSharding) Add added in v2.10.0

func (sharding *ClusterSharding) Add(c *v1alpha1.Cluster)

func (*ClusterSharding) Delete added in v2.10.0

func (sharding *ClusterSharding) Delete(clusterServer string)

func (*ClusterSharding) GetClusterAccessor added in v2.10.0

func (d *ClusterSharding) GetClusterAccessor() clusterAccessor

func (*ClusterSharding) GetDistribution added in v2.10.0

func (sharding *ClusterSharding) GetDistribution() map[string]int

func (*ClusterSharding) Init added in v2.10.0

func (sharding *ClusterSharding) Init(clusters *v1alpha1.ClusterList)

func (*ClusterSharding) IsManagedCluster added in v2.10.0

func (s *ClusterSharding) IsManagedCluster(c *v1alpha1.Cluster) bool

IsManagedCluster returns wheter or not the cluster should be processed by a given shard.

func (*ClusterSharding) Update added in v2.10.0

func (sharding *ClusterSharding) Update(oldCluster *v1alpha1.Cluster, newCluster *v1alpha1.Cluster)

type ClusterShardingCache added in v2.10.0

type ClusterShardingCache interface {
	Init(clusters *v1alpha1.ClusterList)
	Add(c *v1alpha1.Cluster)
	Delete(clusterServer string)
	Update(oldCluster *v1alpha1.Cluster, newCluster *v1alpha1.Cluster)
	IsManagedCluster(c *v1alpha1.Cluster) bool
	GetDistribution() map[string]int
}

func GetClusterSharding added in v2.10.1

func GetClusterSharding(kubeClient kubernetes.Interface, settingsMgr *settings.SettingsManager, shardingAlgorithm string, enableDynamicClusterDistribution bool) (ClusterShardingCache, error)

func NewClusterSharding added in v2.10.0

func NewClusterSharding(_ db.ArgoDB, shard, replicas int, shardingAlgorithm string) ClusterShardingCache

type DistributionFunction added in v2.8.0

type DistributionFunction func(c *v1alpha1.Cluster) int

func GetDistributionFunction added in v2.8.0

func GetDistributionFunction(clusters clusterAccessor, shardingAlgorithm string, replicasCount int) DistributionFunction

GetDistributionFunction returns which DistributionFunction should be used based on the passed algorithm and the current datas.

func LegacyDistributionFunction added in v2.8.0

func LegacyDistributionFunction(replicas int) DistributionFunction

LegacyDistributionFunction returns a DistributionFunction using a stable distribution algorithm: for a given cluster the function will return the shard number based on the cluster id. This function is lightweight and can be distributed easily, however, it does not ensure an homogenous distribution as some shards may get assigned more clusters than others. It is the legacy function distribution that is kept for compatibility reasons

func NoShardingDistributionFunction added in v2.10.0

func NoShardingDistributionFunction() DistributionFunction

NoShardingDistributionFunction returns a DistributionFunction that will process all cluster by shard 0 the function is created for API compatibility purposes and is not supposed to be activated.

func RoundRobinDistributionFunction added in v2.8.0

func RoundRobinDistributionFunction(clusters clusterAccessor, replicas int) DistributionFunction

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL