yupeng9 commented on pull request #5631: URL: https://github.com/apache/incubator-pinot/pull/5631#issuecomment-651991905
Thanks for taking a look. We also have hosts with more than 100k segments. In fact, this addition of throttling is to protect the servers with large number of segments. The more segments a host has, the more transition messages will be generated during events like restart, and therefore the higher risk the server/controller run out of zookeeper buffer. The default value of 100k is set corresponding to the default value of `jute.maxbuffer` which is 1MB, but I'm okay with a much higher value like 100MB, if that is a concern. > This wont work for us since we have more than 100k elements per host. We have not enabled batching yet due to other constraints. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
