assigners

type Assigner ¶

type Assigner interface {
	Assign(
		topic string,
		currAssignments []admin.PartitionAssignment,
	) ([]admin.PartitionAssignment, error)
}

Assigner is an interface for structs that figure out how to reassign replicas in existing topic partitions.

type BalancedLeaderAssigner ¶

type BalancedLeaderAssigner struct {
	// contains filtered or unexported fields
}

BalancedLeaderAssigner is an Assigner that tries to ensure that the leaders of each partition are balanced across all of the broker racks. The general goals are to do this in a way that:

1. Does in-partition leader swaps whenever possible as opposed to changing the set of brokers in a partition
2. Prevents particular brokers from being overrepresented among the leaders

The algorithm currently used is as follows:

while not balanced:
  find racks with fewest and most leaders (i.e., the overrepresented and underrepresented)
  improve balance by doing a single leader replacement:
    use the picker to rank the partitions that have an overrepresented leader
    for each leader:
      for each partition with the leader:
        swap the leader with one of its followers if possible, then stop
    otherwise, use the picker to replace the leader in the top-ranked partition with
      a new broker from the target rack

func NewBalancedLeaderAssigner(
	brokers []admin.BrokerInfo,
	picker pickers.Picker,
) *BalancedLeaderAssigner

func (b *BalancedLeaderAssigner) Assign(
	topic string,
	curr []admin.PartitionAssignment,
) ([]admin.PartitionAssignment, error)

type CrossRackAssigner ¶ added in v1.2.0

type CrossRackAssigner struct {
	// contains filtered or unexported fields
}

CrossRackAssigner is an assigner that ensures that the replicas of each partition are on different racks than each other. The algorithm is:

https://segment.atlassian.net/browse/DRES-922?focusedCommentId=237288

for each partition:

for each non-leader replica:
  if replica is in same rack as leader:
    change replica to a placeholder (-1)

then:

for each partition:

for each non-leader replica:
  if replica is set to placeholder:
    use picker to replace it with a broker in a different rack than the leader and any other replicas

Note that this assigner doesn't make any leader changes. Thus, the assignments need to already be leader balanced before we make the changes with this assigner.

func NewCrossRackAssigner(
	brokers []admin.BrokerInfo,
	picker pickers.Picker,
) *CrossRackAssigner

func (s *CrossRackAssigner) Assign(
	topic string,
	curr []admin.PartitionAssignment,
) ([]admin.PartitionAssignment, error)

type SingleRackAssigner ¶

type SingleRackAssigner struct {
	// contains filtered or unexported fields
}

SingleRackAssigner is an assigner that ensures that the replicas of each partition are in the same rack as the leader. The algorithm is:

for each partition:

for each non-leader replica:
  if replica not in same rack as leader:
    change replica to a placeholder (-1)

then:

for each partition:

for each non-leader replica:
  if replica is set to placeholder:
    use picker to replace it with a broker in the target rack

Note that this assigner doesn't make any leader changes. Thus, the assignments need to already be leader balanced before we make the changes with this assigner.

func NewSingleRackAssigner(
	brokers []admin.BrokerInfo,
	picker pickers.Picker,
) *SingleRackAssigner

func (s *SingleRackAssigner) Assign(
	topic string,
	curr []admin.PartitionAssignment,
) ([]admin.PartitionAssignment, error)

type StaticAssigner ¶

type StaticAssigner struct {
	Assignments []admin.PartitionAssignment
}

StaticAssigner is an Assigner that ignores the current state and assigns based on the value of the Assignments field. Generally intended for testing purposes.

func (s *StaticAssigner) Assign(
	topic string,
	curr []admin.PartitionAssignment,
) ([]admin.PartitionAssignment, error)

type StaticSingleRackAssigner ¶

type StaticSingleRackAssigner struct {
	// contains filtered or unexported fields
}

StaticSingleRackAssigner is an Assigner that assigns replicas within a static rack per partition. This might be useful for cases where we need particular partitions in particular racks, but don't care about the per-replica placement (assuming that the rack is ok).

The following algorithm is used:

for each partition:

for each replica:
  if replica not in the desired (static) rack:
    change the replica to a placeholder (-1)

then:

for each partition:

for each replica:
  if replica set to the placeholder:
    use picker to pick a broker from the set of all brokers in the target rack

In the case of ties, the lowest indexed broker is picked (if randomize is false) or a repeatably random choice (if randomize is true).

func NewStaticSingleRackAssigner(
	brokers []admin.BrokerInfo,
	rackAssignments []string,
	picker pickers.Picker,
) *StaticSingleRackAssigner

func (s *StaticSingleRackAssigner) Assign(
	topic string,
	curr []admin.PartitionAssignment,
) ([]admin.PartitionAssignment, error)