Hello Adam, Thanks for your interests in working on Kafka Streams / KSQL potential performance improvements (I thought the non-key joining will take most of your time :P )
Currently there is no published performance numbers for latest versions of Streams AFAIK. Personally I ran the Streams SimpleBenchmark ( https://github.com/apache/kafka/blob/trunk/tests/ kafkatest/benchmarks/streams/streams_simple_benchmark_test.py) and profile it if necessary trying to figure out the performance bottlenecks. If you are interested you can follow similar approaches, there are also some JIRAs open for potential performance improvements as well: https://issues.apache.org/jira/issues/?jql=project%20% 3D%20KAFKA%20AND%20status%20in%20(Open%2C%20%22In% 20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)% 20AND%20component%20%3D%20%22streams%22%20%20AND% 20labels%20%3D%20performance%20%20 Guozhang On Wed, Aug 22, 2018 at 7:02 AM, Adam Bellemare <adam.bellem...@gmail.com> wrote: > Blog post in question: > https://www.confluent.io/blog/ksql-february-release-streamin > g-sql-for-apache-kafka/ > > On Wed, Aug 22, 2018 at 10:01 AM, Adam Bellemare <adam.bellem...@gmail.com > > > wrote: > > > Hi All > > > > I am looking for performance metrics related to Kafka Streams and KSQL. I > > have been scouring various blogs, including the confluent one, looking > for > > any current performance metrics or benchmarks, official or otherwise, on > > both Kafka Streams and KSQL for Kafka 2.x +. Unfortunately, almost > > everything I am finding is 0.x. > > > > In this particular blog post on KSQL, there is the following quotation: > > > > > For example, our soak testing cluster has racked up over 1,000 hours > > and runs KSQL workloads 24×7. The performance tests we conduct allow us > to > > understand performance characteristics of stateless and stateful KSQL > > queries. We currently run over 42 different tests that collect more than > > 700 metrics. > > > > I assume that there is also some information related to Kafka Streams in > > similar tests. Does anyone know where I can find these results? Or does > > anyone have any blog posts or other materials that look at the > performance > > of either one of these for Kafka 2.x ? > > > > For context, I am asking this question to get a better understanding of > > current Kafka Streams / KSQL performance, such that contributors can > > understand the prioritization of performance-related improvements vs. > > feature-related improvements. > > > > Thanks > > Adam > > > -- -- Guozhang
