Hi Paul, I did some benchmarking as well and couldn't find a marginal difference between KRaft and Zookeeper on end to end latency from producers to consumers. I tested it on Kafka version 3.5.1 and used openmessaging's benchmarking framework https://openmessaging.cloud/docs/benchmarks/ .
What I noticed was if you run the tests long enough(60 mins) the throughput converges to the same value eventually. I also noticed some difference on p99+ latencies between Zookeeper and KRaft clusters but the results were not consistent on repetitive runs. Which version did you make the tests on and what are your findings? On Wed, 31 Jan 2024 at 22:57, Brebner, Paul <paul.breb...@netapp.com.invalid> wrote: > Hi all, > > We’ve previously done some benchmarking of Kafka ZooKeeper vs KRaft and > found no difference in throughput (which we believed is also what theory > predicted, as ZK/Kraft are only involved in Kafka meta-data operations, not > data workloads). > > BUT – latest tests reveal improved producer and consumer latency for Kraft > c.f. ZooKeeper. So just wanted to check if Kraft is actually involved in > any aspect of write/read workloads? For example, some documentation > (possibly old) suggests that consumer offsets are stored in meta-data? In > which case this could explain better Kraft latencies. But if not, then I’m > curious to understand the difference (and if it’s documented anywhere?) > > Also if anyone has noticed the same regarding latency in benchmarks. > > Regards, Paul Brebner >
