Re: Streams reprocessing whole topic when deployed but not locally

2019-07-08 Thread Alessandro Tagliapietra
I think I'm having again this issue, this time though it only happens on some state stores. Here you can find the code and the logs https://gist.github.com/alex88/1d60f63f0eee9f1568d89d5e1900fffc We first seen that our confluent cloud bill went up 10x, then seen that our streams processor was rest

Re: org.apache.kafka.streams.processor.TimestampExtractor#extract method in version 2.3 always returns -1 as value

2019-07-08 Thread Sophie Blee-Goldman
Thanks for the notice Jonathan! We tracked down the problem and it should be an easy fix: https://github.com/apache/kafka/pull/6719/files On Fri, Jul 5, 2019 at 6:25 AM Jonathan Santilli wrote: > Thanks a lot Bill for creating the issue, I have updated it with a little > bit more of info. > > Ch

Re: Detecting server unavailable

2019-07-08 Thread Liam Clarke
Hi Steve, Have you tried setting "default.api.timeout.ms" to something lower in your consumer configuration? It'll then throw that exception earlier. Kind regards, Liam Clarke On Tue, Jul 9, 2019 at 3:01 AM Gorman, Steve A. wrote: > I have a springboot microservice that listens to a couple of

Re: PR review

2019-07-08 Thread Colin McCabe
Hi M. Manna, I left a review. Take a look. Sorry for the delays. best, Colin On Mon, Jul 8, 2019, at 14:38, M. Manna wrote: > Hello, > > A few requests have been sent already. Could this please be reviewed ? Our > business implementation is holding due to this change. > > > > On Thu, 4 Ju

Re: Kafka 2.3.0 - inMemoryKeyValueStore changes (KAFKA-7918) - java.lang.NullPointerException

2019-07-08 Thread Sophie Blee-Goldman
Hi Nitay, InMemoryKeyValueStore is in the internal package, not part of the public API, so it is not meant to be used directly since it's implementation may change at any time (as has happened here). It's intended that you use the store builders/suppliers to get a new state store, but as you notic

Re: PR review

2019-07-08 Thread M. Manna
Hello, A few requests have been sent already. Could this please be reviewed ? Our business implementation is holding due to this change. On Thu, 4 Jul 2019 at 13:33, M. Manna wrote: > https://github.com/apache/kafka/pull/6771 > > Could this be reviewed please ? > > On Wed, 3 Jul 2019 at 11:35

Broker Startup Time (Prolonged URPs)

2019-07-08 Thread Jamie
Hi All, I'm currently doing some work into investigating why our Kafka deployment has a prolonged period of URPs after a restart 30-45 minutes. At the moment we use a replication factor of 2 due to the cost associated with having another copy of each partition, however, we do plan to move to rep

Re: stream.filter() based on message header

2019-07-08 Thread Sophie Blee-Goldman
Hi Jorg, The transform/transformValues/process operators are not inherently stateful. They *can* be stateful if you choose to connect a statestore, but otherwise they are stateless and in your case, most likely what you want to use. Cheers, Sophie On Thu, Jul 4, 2019 at 2:51 AM Jorg Heymans wro

(KIP-368) SASL mechanism GSSAPI authentication failure

2019-07-08 Thread Tyler Monahan
Hello, I have an issue using using SASL with GSSAPI and kerberos. When I lose a node and a new one is built by our automation all of the kafka nodes continue trying to connect to the replacement node with the old nodes kerberos details which fails. I was expecting KIP-368: "Allow SASL Connections

Detecting server unavailable

2019-07-08 Thread Gorman, Steve A.
I have a springboot microservice that listens to a couple of topics. When the microservice starts up I would like to quickly (5->10 seconds) determine if a kafka broker is unavailable. I am able to see this work with a kafka console consumer but am having a hard time getting Java (spring-kafka)

Kafka 2.3.0 - inMemoryKeyValueStore changes (KAFKA-7918) - java.lang.NullPointerException

2019-07-08 Thread Nitay Kufert
Hey, Following https://issues.apache.org/jira/browse/KAFKA-7918 I had to change the current implementation of our unit tests. Before the change, I created a store using: new InMemoryKeyValueStore[String, BigDecimal]( countersStoreName, Serdes.String, SpecificSerdes.bigDecimalSerde ) It seem

Facing memory issues with kafka streams application

2019-07-08 Thread kalyani yarlagadda
Hi, I need assistance in the below scenario. Please help me with this. I am using the hopping time window in Kafka streams with *suppress*() I am seeing the following memory Errors. *1. Facing the memory issue when the Kafka application is running continuously* for 2 to 3 days of deployment with