thetumbled commented on PR #4258: URL: https://github.com/apache/bookkeeper/pull/4258#issuecomment-2081314971
List some experimental data. A 20 node bookstore cluster has been deployed with `autoRecoveryDaemonEnable=true`, so each cookie is a replicator. 20 replicators are configured with Ensemble Size, WQ, and AQ of `3 3 2`. ## 200 100MB Leggers Experimental conditions: - Create 200 100MB Leggers. Approximately 200 * 3/20=30 Leggers are loaded on each cookie, occupying 3000MB of storage space. Comparison conditions: - The replicationAcquireTaskPerSecond is set to the default value of 0, with no speed limit. - The replicationAcquireTaskPerSecond is set to 1, limiting the task to be read once per second. Compare the read latency and read throughput of zk by decommissing a cookie. Give the result as follows - No speed limit (replicationAcquireTaskPerSecond=0)   We can see that the read delay only reaches 60ms. - Speed limit (replicationAcquireTaskPerSecond=1)   The peak read delay is 3ms, which is indeed a significant decrease compared to the previous 60ms. But because the second level delay has not been reproduced, the persuasiveness is not enough. Let's take a look at the experiment below. ## 5000 2MB Leggers Experimental conditions: - Create 5000 2MB Leggers. Approximately 5000 * 3/20=750 Leggers are loaded on each cookie, occupying 1500MB of storage space. Comparison conditions: - The replicationAcquireTaskPerSecond is set to the default value of 0, with no speed limit. - The replicationAcquireTaskPerSecond is set to 1, limiting the task to be read once per second. - The replicationAcquireTaskPerSecond is set to 0.5, with a limit of reading zk to retrieve tasks every 2 seconds. Observe and compare the read latency and read throughput of zk. - No speed limit (replicationAcquireTaskPerSecond=0)   The read delay reaches 2 seconds, and the traffic for reading zk reaches 146kb/s. - ReplicationAcquireTaskPerSecond=1   The peak read latency is 1.38s, and the peak read traffic significantly decreases to 73KB/s. - ReplicationAcquireTaskPerSecond=0.5   The peak read latency significantly decreased to 40ms, and the peak read traffic was 73KB/s. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
