somandal commented on PR #15050:
URL: https://github.com/apache/pinot/pull/15050#issuecomment-2667227134
> points 3,4,5 look great.
> point 2 - sure we can take this up later
> For 1, the tag in each server's section is helpful. What I was suggesting
though, is that we provide the list right on top (Essentially 4 lists called
serversGettingNewSegments, serverAdded, serverRemoved, serverUnchanged) as when
we'll get to clusters with 50-100 servers, parsing through 50-100 individual
json blurbs per server might get tricky.
Got it, updated the PR to add your suggestion for 1.
Sample summary:
Add server:
```
{
"serverInfo" : {
"numServersGettingNewSegments" : 1,
"numServers" : {
"valueBeforeRebalance" : 1,
"expectedValueAfterRebalance" : 2
},
"serversAdded" : [ "Server_localhost_22004" ],
"serversRemoved" : [ ],
"serversUnchanged" : [ "Server_localhost_22001" ],
"serversGettingNewSegments" : [ "Server_localhost_22004" ],
"serverSegmentChangeInfo" : {
"Server_localhost_22004" : {
"serverStatus" : "ADDED",
"totalSegmentsAfterRebalance" : 6,
"totalSegmentsBeforeRebalance" : 0,
"segmentsAdded" : 6,
"segmentsDeleted" : 0,
"segmentsUnchanged" : 0,
"tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
},
"Server_localhost_22001" : {
"serverStatus" : "UNCHANGED",
"totalSegmentsAfterRebalance" : 6,
"totalSegmentsBeforeRebalance" : 12,
"segmentsAdded" : 0,
"segmentsDeleted" : 6,
"segmentsUnchanged" : 6,
"tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
}
}
},
"segmentInfo" : {
"totalSegmentsToBeMoved" : 6,
"maxSegmentsAddedToASingleServer" : 6,
"estimatedAverageSegmentSizeInBytes" : 1690546,
"totalEstimatedDataToBeMovedInBytes" : 10143276,
"replicationFactor" : {
"valueBeforeRebalance" : 1,
"expectedValueAfterRebalance" : 1
},
"numSegmentsInSingleReplica" : {
"valueBeforeRebalance" : 12,
"expectedValueAfterRebalance" : 12
},
"numSegmentsAcrossAllReplicas" : {
"valueBeforeRebalance" : 12,
"expectedValueAfterRebalance" : 12
}
}
}
```
Remove server:
```
{
"serverInfo" : {
"numServersGettingNewSegments" : 1,
"numServers" : {
"valueBeforeRebalance" : 2,
"expectedValueAfterRebalance" : 1
},
"serversAdded" : [ ],
"serversRemoved" : [ "Server_localhost_22004" ],
"serversUnchanged" : [ "Server_localhost_22001" ],
"serversGettingNewSegments" : [ "Server_localhost_22001" ],
"serverSegmentChangeInfo" : {
"Server_localhost_22004" : {
"serverStatus" : "REMOVED",
"totalSegmentsAfterRebalance" : 0,
"totalSegmentsBeforeRebalance" : 6,
"segmentsAdded" : 0,
"segmentsDeleted" : 6,
"segmentsUnchanged" : 0,
"tagList" : [ ]
},
"Server_localhost_22001" : {
"serverStatus" : "UNCHANGED",
"totalSegmentsAfterRebalance" : 12,
"totalSegmentsBeforeRebalance" : 6,
"segmentsAdded" : 6,
"segmentsDeleted" : 0,
"segmentsUnchanged" : 6,
"tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
}
}
},
"segmentInfo" : {
"totalSegmentsToBeMoved" : 6,
"maxSegmentsAddedToASingleServer" : 6,
"estimatedAverageSegmentSizeInBytes" : 1690546,
"totalEstimatedDataToBeMovedInBytes" : 10143276,
"replicationFactor" : {
"valueBeforeRebalance" : 1,
"expectedValueAfterRebalance" : 1
},
"numSegmentsInSingleReplica" : {
"valueBeforeRebalance" : 12,
"expectedValueAfterRebalance" : 12
},
"numSegmentsAcrossAllReplicas" : {
"valueBeforeRebalance" : 12,
"expectedValueAfterRebalance" : 12
}
}
}
```
Also note that I've removed the time estimates for now as recommended by
@klsince as it is hard to come up with a good estimate (different environments
may have different speeds). We can perhaps come up with some heuristic based on
some real rebalances if that makes sense in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
