Marton Greber has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/24476 )
Change subject: KUDU-3767 Add global leader awareness to auto leader rebalancer
(Part 2)
......................................................................
KUDU-3767 Add global leader awareness to auto leader rebalancer (Part 2)
Part 1 threaded a global leader count map through the per-table loop to
break ties when scoring destinations. That steers individual transfers
but can't fix skew that spans tables: since each table is balanced
independently, a tserver can keep getting the ceiling ('+1') allocation
across many tables and end up holding far more leaders than its peers,
even though every table is locally balanced. A typical case is many
single-tablet tables whose lone leaders all land on the same tserver.
This patch adds the post-loop global pass promised in Part 1. Once every
table is balanced on its own, it moves leaders off tservers that are above
the global ceiling onto ones with room below it, but only when doing so
keeps the affected table within its own floor/ceil. The pass runs only in
a round where per-table balancing scheduled nothing (to avoid racing its
in-flight, asynchronous moves) and is best-effort, logging and skipping
transient step-down failures.
Change-Id: I5d44761f6444ca1d1bf4fcee405d65d029e41980
Reviewed-on: http://gerrit.cloudera.org:8080/24476
Reviewed-by: Alexey Serbin <[email protected]>
Reviewed-by: Marton Greber <[email protected]>
Tested-by: Marton Greber <[email protected]>
---
M src/kudu/master/auto_leader_rebalancer-test.cc
M src/kudu/master/auto_leader_rebalancer.cc
M src/kudu/master/auto_leader_rebalancer.h
3 files changed, 705 insertions(+), 15 deletions(-)
Approvals:
Alexey Serbin: Looks good to me, approved
Marton Greber: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/24476
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I5d44761f6444ca1d1bf4fcee405d65d029e41980
Gerrit-Change-Number: 24476
Gerrit-PatchSet: 6
Gerrit-Owner: Gabriella Lotz <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Ashwani Raina <[email protected]>
Gerrit-Reviewer: Gabriella Lotz <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <[email protected]>
Gerrit-Reviewer: Zoltan Chovan <[email protected]>
