J-HowHuang opened a new pull request, #19252: URL: https://github.com/apache/pinot/pull/19252
## Description The current implementation does iterative ZK metadata read for each segment. It's bad when segment count is at few hundred thousands, the zk read latency due to the queued read request causes this checker to run more than the default task frequency 5 min. There was even a `TODO: revisit the logic and reduce the ZK access` at the method itself. ## Change Batch read the segment ZK metadata for the entire table, then perform the replica checks and other maths. This approach is acceptable as retention manager, realtime validation manager are all fetching them for the entire table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
