Configuration control the interval after leader broker switchover

Hi All, We try to upgrade Kafka from 0.8.0 to 0.10.0.0, and there are two brokers in our Kafka cluster. We need to restart each broker one by one during the upgrade procedure, when we shut down the leader broker, seems the follower broker could take the leader role immediately, but the

RE: Kafka upgrade from 0.8.0 to 0.10.0.0

Hi Amit, Thanks for your info! Best Regards Johnny -Original Message- From: Amit Tank [mailto:amittankopensou...@gmail.com] Sent: 2016年11月16日 0:33 To: users@kafka.apache.org Subject: Re: Kafka upgrade from 0.8.0 to 0.10.0.0 Hi Johnny, I haven't tried it yet so I am not 100%

RE: Kafka upgrade from 0.8.0 to 0.10.0.0

Hi, I have another question, for ZK upgrade, we just shut down it, update the new version, then restart it. If it is enough or there is any configuration need change? Thanks! Best Regards Johnny -Original Message- From: Gwen Shapira [mailto:g...@confluent.io] Sent:

RE: Kafka upgrade from 0.8.0 to 0.10.0.0

Hi, The current ZK version is 3.3.4 (Kafka 0.8.0), we want to upgrade to 3.4.6 (Kafka 0.10.0.0), do you mean it's better to upgrade ZK before upgrade Kafka? Thanks! Best Regards Johnny -Original Message- From: Gwen Shapira [mailto:g...@confluent.io] Sent: 2016年11月16日 10:18

Re: Kafka upgrade from 0.8.0 to 0.10.0.0

What version of ZK are you using? If you are using ZK 3.3, I'd recommend upgrading it before you upgrade Kafka. If you are on ZK 3.4.5 or 3.4.6, you are good. On Tue, Nov 15, 2016 at 8:33 AM, Amit Tank wrote: > Hi Johnny, > > I haven't tried it yet so I am not 100%

Kafka windowed table not aggregating correctly

Hi, I have a simple pipeline stream.aggregateByKey(new Initializer() { public List apply() { return new List } }, new Aggregator() { public List apply(key, value, list) { list.add(value) return list } }, keysSerde, valuesSerde, "table") So this

RE: Kafka 0.10 Monitoring tool

But we see the below error: [2016-11-15 17:25:57,240] WARN Bootstrap broker host1:61616 disconnected (org.apache.kafka.clients.NetworkClient) Error while executing consumer group command Request GROUP_COORDINATOR failed on brokers List(host1:61616 (id: -1 rack: null))

Re: Process KTable on Predicate

Nick, if I understand you correctly you can already do this today: Think: KTable.toStream().filter().foreach() (or just KTable.filter().foreach(), depending on what you are aiming to do) Would that work for you? On Sun, Nov 13, 2016 at 12:12 AM, Nick DeCoursin wrote:

Re: Kafka upgrade from 0.8.0 to 0.10.0.0

Hi Johnny, I haven't tried it yet so I am not 100% sure which one should be upgraded in what order but the way I think is - you can upgrade brokers when you are not upgrading ZK too. so if that holds true then the sequence do not matter. Again I can be completely wrong here.May be some one else

Re: Checking the consumer lag when using manual partition assignment with the KafkaConsumer

Hi Matthias, from what I understand it is (was) a bug https://issues.apache.org/jira/browse/KAFKA-3859?focusedCommentId=15376317=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15376317 But I see it would have been fixed late october

RE: Kafka 0.10 Monitoring tool

Yes, we tried with this command but the problem is it does not return the output always. It's really weird. Sometimes it works and sometimes not. Any idea what could be the wrong? Thanks Achintya -Original Message- From: Sam Pegler [mailto:sam.peg...@infectiousmedia.com] Sent:

getting an error about: java.lang.OutOfMemoryError: Direct buffer memory

kafka version - 0.9.0.0 JVM flags: -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true It's not easily reproducible but has caused some issues for us, any insight would be appreciated. Full stack:

Re: Kafka 0.10 Monitoring tool

Have you thought of just wrapping the provided Java classes (kafka-consumer-groups.sh) with the programming language of your choice? For example... /opt/kafka/bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server eu1-kb-1:9092 --describe --group foo GROUP TOPIC

Producer send message fail after upgrade broker from 0.8.0 to 0.10.0.0

Hi All, I have three zookeepers, two Kafka brokers and some clients (consumer and producer) with current version 0.8.0, according to Kafka document, I do a rolling upgrade to 0.10.0.0 following below procedure: 1) Upgrade zookeeper one by one (shut down, update the code with 0.10.0.0,

SASL authenticate error when tring to connect zookeeper

Hi expert, Now I am verifying connection between broker and zookeeper using SASL mechanisms and the zookeeper always claims: GSS initiate failed [Caused by GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed) >From Kerberos log, I see zookeeper sent AS_REQ,

RE: Kafka upgrade from 0.8.0 to 0.10.0.0

Hi All, Anyone knows if zookeeper upgrade should be executed before broker upgrade? Thanks! Best Regards Johnny -Original Message- From: ZHU Hua B Sent: 2016年11月14日 15:07 To: users@kafka.apache.org Subject: RE: Kafka upgrade from 0.8.0 to 0.10.0.0 Hi, Thanks for your