wang-jiahua commented on PR #10448: URL: https://github.com/apache/rocketmq/pull/10448#issuecomment-4797146614
Thanks for the suggestion! We considered the loop + `RateLimiter` approach but ran benchmarks to compare both, and the batch-persist approach is significantly faster. **Why batch persist instead of loop:** Each call to `deleteTopicConfig` / `deleteSubscriptionGroupConfig` triggers `persist()`, which performs 3 `fsync` operations (file write + 2 directory fsyncs). Looping N times = 3N fsync calls. The batch path removes all entries from memory first, then calls `persist()` once — reducing 3N fsyncs to 3. **Server benchmark (50 topics/groups, with Producer+Consumer load, P99 RT measured):** | Metric | Loop delete (50 topics) | Batch delete (50 topics) | |--------|------------------------|-------------------------| | Delete time | 19,297ms | 107ms (**180x**) | | Producer TPS | 144k (-22%) | 171k (0%) | | Producer P99 RT | 7ms | 5ms | | Consumer TPS | 141k (-21%) | 176k (0%) | | GC delta | +34 | 0 | | Metric | Loop delete (50 groups) | Batch delete (50 groups) | |--------|------------------------|-------------------------| | Delete time | 228,024ms | 43ms (**5,303x**) | | Producer TPS | 165k (-3%) | 171k (0%) | | Consumer Max RT | 511ms (+119%) | 289ms (0%) | | GC delta | +1,154 | +388 (-66%) | **Regarding `RateLimiter.create(10.0)`:** At 10 deletes/sec, 50 topics would take ~5s, still 47x slower than batch (107ms). The batch approach finishes in 100ms — already much smoother than rate-limiting can achieve, with zero TPS/RT impact on running Producer/Consumer. **Regarding regression tests:** Added unit tests in the PR covering valid entries, empty list, blank entries, and non-existent entries. Would appreciate your feedback! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
