Re: INVALID_FETCH_SESSION_EPOCH after upgrade to 1.1.0

2018-08-01 Thread Mark Anderson
Also in this case will it fall back to a full request? Hence no data is lost but it might increase latency? Thanks Mark On Thu, 26 Jul 2018, 12:28 Mark Anderson, wrote: > Ted, > > Below are examples of the DEBUG entries from FetchSession > > [2018-07-26 11:14:43,461] DEBUG Created a new error

Replica not coming up for a partition after restarting broker | Error NOT_LEADER_FOR_PARTITION

2018-08-01 Thread Manjunath N
Hi, Problem: A replica for a partition is not coming up. Steps below; I have a zookeeper cluster with three machines and a kafka cluster with 3 brokers on the same machine. I created a topic test as below. Operation: kafka-topics.sh --zookeeper rh3:2181/kafka --create --topic test --replicati

A question about Kafka Stream API

2018-08-01 Thread Kyle.Hu
Hi, bosses: I have read the word count demo of Kafka Stream API, it is cool that the Kafka Stream keeps the status, and I have a question about it, Weather it would cause memory problems when the keys accumulate a lot ?

Re: A question about Kafka Stream API

2018-08-01 Thread Damian Guy
The count is stored in RocksDB which is persisted to disk. It is not in-memory unless you specifically use an InMemoryStore. On Wed, 1 Aug 2018 at 12:53 Kyle.Hu wrote: > Hi, bosses: > I have read the word count demo of Kafka Stream API, it is cool that > the Kafka Stream keeps the status,

Re: Creating and deleting Kafka Topics in Scala App

2018-08-01 Thread Pulkit Manchanda
Hi Matthias, Thanks for the reply, I had already tried that. But it doesn't work either. Pulkit On Tue, Jul 31, 2018 at 9:22 PM, Matthias J. Sax wrote: > Is `delete.topic.enable` set to `true`? It's a broker configuration. > > > -Matthias > > On 7/31/18 8:57 AM, Pulkit Manchanda wrote: > > HI

Re: Creating and deleting Kafka Topics in Scala App

2018-08-01 Thread Matthias J. Sax
Not sure. Anything interesting in the logs? Maybe you need to enable DEBUG. As an alternative, you might ask a question on the Github page providing the example code. -Matthias On 8/1/18 7:11 AM, Pulkit Manchanda wrote: > Hi Matthias, > > Thanks for the reply, I had already tried that. But it

Total metrics are broken in Kafka streams 2.0.0

2018-08-01 Thread Sam Lendle
Is this a known issue? If not I can create a JIRA. For example, here’s a screenshot of the process-total metric for one processor node and task under stream-processor-node-metrics. Processor-total goes down over time when it shouldn’t. Same thing for total metrics under stream-metrics, stream-*

Re: Total metrics are broken in Kafka streams 2.0.0

2018-08-01 Thread Guozhang Wang
Hello Sam, Your uploaded snapshot was rejected by apache mailing list, could you upload it somewhere else, and share the link with us? Guozhang On Wed, Aug 1, 2018 at 4:37 PM, Sam Lendle wrote: > Is this a known issue? If not I can create a JIRA. > > > > For example, here’s a screenshot of t

Re: Total metrics are broken in Kafka streams 2.0.0

2018-08-01 Thread Sam Lendle
Thought that might happen: https://i.imgur.com/eDV9CIM.png On 8/1/18, 4:50 PM, "Guozhang Wang" wrote: Hello Sam, Your uploaded snapshot was rejected by apache mailing list, could you upload it somewhere else, and share the link with us? Guozhang On We

RE: Replica not coming up for a partition after restarting broker | Error NOT_LEADER_FOR_PARTITION

2018-08-01 Thread 赖剑清
Hi, What's the version of your Kafka brokers? As far as I know, The old version of Kafka (0.9.x?) used the InetAddress.getLocalHost() to bind it by default. That means if you forget to set the leaders' bind address (in the config/server.properties), the followers will be told to connect to the

My client's kafka consumer disappears

2018-08-01 Thread Luan Hongjun
Hi Someone, I've encountered a strange problem. I write a consumer with python to get data from kafka. .. self.consumer = KafkaConsumer(self.kafkatopic, group_id=self.groupid, bootstrap_servers=self.kafkaservers, auto_offset_reset="earliest")

Re: Total metrics are broken in Kafka streams 2.0.0

2018-08-01 Thread Sam Lendle
The problem is that the total metrics in streams are implemented using a Count, which is a SampledStat, so the value it reports is the count over recent time windows, not all of time. The behavior I was seeing was when a window is purged, the reported total decreases. I’ll open a JIRA tomorrow.