Multiple controller in 10.2

2017-05-29 Thread Naanu Bora
Hi, Our cluster in production (10.2) is running into multiple controllers. Rebooting helps temporarily but after few days it again runs into same multiple controller issue. Is there any way to re elect the controller? Thanks

Re: Queries regarding kafka Monitoring tool burrow.

2017-05-29 Thread Todd Palino
The lag numbers are never going to be exactly the same as what the CLI tool returns, as the broker is queried on an interval for the offset at the end of each partition. As far as crashing goes, I’d be interested to hear about specifics as we run it (obviously) and don’t have that problem. It could

Re: Queries regarding kafka Monitoring tool burrow.

2017-05-29 Thread Ian Duffy
Hey Abhimanyu, Not directly answering your questions but in the past we used burrow at my current company and we had a horrible time with it. It would crash daily and its lag metrics were very different to what was returned when you would run the kafka-consumer-group describe command as you noted.

Re: Kafka cluster management lifecycle

2017-05-29 Thread Roman Naumenko
Is there a roadmap to have this feature? It’s pretty basic requirement these days to run stuff in the cloud. — Roman > On May 26, 2017, at 8:57 PM, Waleed Fateem wrote: > > I might have misunderstood what you're asking for, but my understanding is > that you were looking a way to have Kafka a

Re: [E] Re: Kafka Configuration Question

2017-05-29 Thread Hans Jespersen
Not sure why you would need 7 zookeepers, 3 or 5 is more common but since you have setup 7 then zookeeper will not be active unless a “quorum” of the zookeepers are up and running. That means you need to have 4 or more zookeepers up and running first, and then start your kafka brokers. Also, yo

Re: [DISCUSS]: KIP-161: streams record processing exception handlers

2017-05-29 Thread Jay Kreps
Hey Eno, I think this makes sense. I do think people who spend time running production stream processing systems will, over time, end up strongly preferring the current behavior of failing and fixing the root problem rather than skipping, but we don't need to force this on people as long as the de

Re: vpn vs TimeoutException

2017-05-29 Thread Peter Sinoros Szabo
Is this maybe related to https://issues.apache.org/jira/browse/KAFKA-3686 or https://issues.apache.org/jira/browse/KAFKA-4089 ? Thanks, - Sini From: Peter Sinoros Szabo/Hungary/IBM To: users@kafka.apache.org Date: 2017/05/25 17:01 Subject:vpn vs TimeoutException Hi, Please h

Broker does not seem to honor min.compaction.lag.ms

2017-05-29 Thread Frank Lyaruu
Hi Kafka people, I've configured my brokers to use one hour (360 ms) for the min.compaction.lag.ms. In the logs I see that this is picked up in the 'created log' line. I add a bit over a million messages, and I see the new log segments appearing ('Rolling new log segment') but after a few min

Re: Kafka Streams - possible to know partition?

2017-05-29 Thread Srimanth G
Thank you. That worked. Regards, SG On Sun, May 28, 2017 at 2:50 PM, Matthias J. Sax wrote: > I just added this FAQ: > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ- > HowcanIaccessrecordmetadata? > > > -Matthias > > > On 5/28/17 12:36 PM, Srimanth G wrote: > > Hello, > > When u