thetumbled commented on PR #21085: URL: https://github.com/apache/pulsar/pull/21085#issuecomment-2122305937
List some experimetal data to show the improvement. I deploy a cluster with 5 nodes, and there are 1k+ bundles in the cluster. Set topK=20, so that there will be 20*5=100 bundles to be uploaded to zk, which means we filter out 90% bundles. It is ok that we filter so many bundles as most of the them are small bundles. Following graph show the effects:   Write latency has decreased from 200ms to single digits! Not only do write latency and throughput decrease, but read latency and throughput also decrease significantly! Because as long as there is less writing, there is also less reading.   -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
