ckdarby commented on issue #7058: URL: https://github.com/apache/pulsar/issues/7058#issuecomment-636369850
@sijie Firstly, thank you for everything, :bow: . /pulsar/data/bookkeeper/ledgers/current/locations are very small. Don't see any of the bookies with ledger/current/locations any larger than 50 MByte total dir and I have dbStorage_rocksDB_blockCacheSize set to 1 GB at the moment. Happy to report that I changed: dispatcherMaxReadBatchSize=10000 Saw mixed results of decent and extremely good. Extremely good results such as: ``` 03:26:05.111 [main] INFO org.apache.pulsar.testclient.PerformanceReader - Read throughput: 243598.835 msg/s -- 2787.767 Mbit/s 03:26:15.111 [main] INFO org.apache.pulsar.testclient.PerformanceReader - Read throughput: 242916.023 msg/s -- 2779.953 Mbit/s 03:26:25.112 [main] INFO org.apache.pulsar.testclient.PerformanceReader - Read throughput: 241419.952 msg/s -- 2762.832 Mbit/s 03:26:35.112 [main] INFO org.apache.pulsar.testclient.PerformanceReader - Read throughput: 236405.226 msg/s -- 2705.443 Mbit/s ``` The extremely good results are probably when the broker & bookie happen to be on the same node or all three, proxy, broker, and bookie happen to be on the same node. Other results on different partitions (didn't record exactly), looked something like: ``` Read throughput: 100k msg/s -- 1250 Mbit/s ``` This is probably when a proxy, broker and bookie are all on different nodes or at least the broker & bookie are on different nodes. Still much better performance throughput off of EBS than before :+1: Thanks for everything :) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
