Re: Kafka Streams Session store performance degradation from 0.10.2.1 to 0.11.0.3

2019-01-11 Thread Guozhang Wang
Hello Jonathan, I've left a comment on https://issues.apache.org/jira/browse/KAFKA-7652 with a fix trying to resolve the discovered bug in trunk. If it verifies to be the right fix I will push it to older branches as well. Just FYI. Guozhang On Fri, Nov 16, 2018 at 4:26 PM Guozhang Wang

Re: Kafka Streams Session store performance degradation from 0.10.2.1 to 0.11.0.3

2018-11-17 Thread jonathangordon
On 2018/11/17 00:26:56, Guozhang Wang wrote: > Could you create a JIRA with all the current available information uploaded > on the ticket for me to further investigate the issue? This way we will not > lose track of it (email list is not the best venue for potential bug > investigation :).

Re: Kafka Streams Session store performance degradation from 0.10.2.1 to 0.11.0.3

2018-11-16 Thread Guozhang Wang
Hi Jonathan, Could you create a JIRA with all the current available information uploaded on the ticket for me to further investigate the issue? This way we will not lose track of it (email list is not the best venue for potential bug investigation :). At the mean time, I will try to compare the

Re: Kafka Streams Session store performance degradation from 0.10.2.1 to 0.11.0.3

2018-11-15 Thread jonathangordon
On 2018/11/08 00:13:39, "Matthias J. Sax" wrote: > That is what I try to figure out. I went over the 0.10.2.2 to 0.11.0.3 > Jiras but found nothing I could point out. There are couple of > SessionStore related tickets, but none of them should have an effect > like this. > > To narrow it down,

Re: Kafka Streams Session store performance degradation from 0.10.2.1 to 0.11.0.3

2018-11-08 Thread Matthias J. Sax
Thanks for verifying. >> From our perspective, it appears something happened after 0.10.2.1 that made >> the LRU Cache much slower for our use case. That is what I try to figure out. I went over the 0.10.2.2 to 0.11.0.3 Jiras but found nothing I could point out. There are couple of

Re: Kafka Streams Session store performance degradation from 0.10.2.1 to 0.11.0.3

2018-11-08 Thread Jonathan Gordon
Hi Matthias, I upgraded to 2.0.0 and we're experiencing the same problem. I've posted a new screengrab of a threadprofile: https://imgur.com/a/2wncPHw >From our perspective, it appears something happened after 0.10.2.1 that made the LRU Cache much slower for our use case. What would you

Re: Kafka Streams Session store performance degradation from 0.10.2.1 to 0.11.0.3

2018-11-07 Thread jonathangordon
Hi Matthias, I upgraded to 2.0.0 and we're experiencing the same problem. I've posted a new screengrab of a thread profile: https://imgur.com/a/2wncPHw >From our perspective, it appears something happened after 0.10.2.1 that made >the LRU Cache much slower for our use case. What would you

Re: Kafka Streams Session store performance degradation from 0.10.2.1 to 0.11.0.3

2018-11-06 Thread Matthias J. Sax
Not sure atm why you see a performance degradation. Would need to dig into the details. However, did you consider to upgrade to 2.0 instead or 0.11? Also note that we added a new operator `suppress()` in upcoming 2.1 release, that allows you to do rate control without caching:

Kafka Streams Session store performance degradation from 0.10.2.1 to 0.11.0.3

2018-11-06 Thread Jonathan Gordon
I have a Kafka Streams app that I'm trying to upgrade from 0.10.2.1 to 0.11.0.3 but when I do I notice that CPU goes way up and consumption goes down. A thread profile indicates that the most expensive task is during our aggregation, fetching from the cache. Thread profile with caching: