somandal commented on code in PR #15617:
URL: https://github.com/apache/pinot/pull/15617#discussion_r2059080890
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/TableRebalancer.java:
##########
@@ -479,6 +482,16 @@ private RebalanceResult doRebalance(TableConfig
tableConfig, RebalanceConfig reb
externalViewStabilizationTimeoutInMs);
int expectedVersion = currentIdealState.getRecord().getVersion();
+ // Cache segment partition id to avoid ZK reads. Similar behavior as cache
used in StrictReplicaGroupAssignment
+ // NOTE:
+ // 1. This cache is used for table rebalance only, but not segment
assignment. During rebalance, rebalanceTable()
+ // can be invoked multiple times when the ideal state changes during
the rebalance process.
+ // 2. The cache won't be refreshed when an existing segment is replaced
with a segment from a different partition.
+ // Replacing a segment with a segment from a different partition should
not be allowed for upsert table because
+ // it will cause the segment being served by the wrong servers. If this
happens during the table rebalance,
+ // another rebalance might be needed to fix the assignment.
+ Object2IntOpenHashMap<String> segmentPartitionIdMap = new
Object2IntOpenHashMap<>();
Review Comment:
I am not against the idea but want some thoughts from @Jackie-Jiang on this.
That cache is private to `StrictRealtimeSegmentAssignment` and not relevant for
other segment assignments, so definitely don't want to expose an interface to
access that. Other option is to make this public, but I kind of think that
leaks the internal behavior of `StrictRealtimeSegmentAssignment` outside.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
