Hi all, I've been experimenting a bit with ways of increasing the throughput of some kafka streams applications that have "exactly_once" enabled. (these applications consume from a topic, transform the data, and do 2 different aggregations before finally writing the output to a sink topic) Increasing the producer properties of linger.ms and batch.size helps, but the thing that seems to make the biggest difference by far is increasing commit.interval.ms. The tradeoff to increasing this, as I understand it, is increased overall end-to-end latency. (if I'm wrong on that please let me know)
Here's my question: Is it bad practice to increase commit.interval.ms when using EOS? The reason I ask is because the default with EOS enabled is 100ms, yet without EOS the default is 30000. (30 seconds) That's a huge difference, and I'm curious about the reasoning behind that. Thanks for any guidance, Alex C
