After consuming about 5 gb of messages , it stopped consuming and it got
stuck at a particular offset. After running the following command I got an
error regarding the broker.
$ bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group
air-DummyProductionConsumerGroup-After --zkconnect 172.16.18.65:2181--topic air
[2013-07-23 20:19:42,748] INFO Starting ZkClient event thread.
(org.I0Itec.zkclient.ZkEventThread)
[2013-07-23 20:19:42,757] INFO Client
environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,757] INFO Client environment:host.name=localhost
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,757] INFO Client environment:java.version=1.7.0_21
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,757] INFO Client environment:java.vendor=Oracle
Corporation (org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,757] INFO Client
environment:java.home=/usr/java/jdk1.7.0_21/jre
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,757] INFO Client
environment:java.class.path=:bin/../project/boot/scala-2.8.0/lib/scala-compiler.jar:bin/../project/boot/scala-2.8.0/lib/scala-library.jar:bin/../core/target/scala_2.8.0/kafka-0.7.2.jar:bin/../core/lib/*.jar:bin/../perf/target/scala_2.8.0/kafka-perf-0.7.2.jar:bin/../core/lib_managed/scala_2.8.0/compile/jopt-simple-3.2.jar:bin/../core/lib_managed/scala_2.8.0/compile/log4j-1.2.15.jar:bin/../core/lib_managed/scala_2.8.0/compile/snappy-java-1.0.4.1.jar:bin/../core/lib_managed/scala_2.8.0/compile/zkclient-0.1.jar:bin/../core/lib_managed/scala_2.8.0/compile/zookeeper-3.3.4.jar
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,758] INFO Client
environment:java.library.path=/usr/local/oracle:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,758] INFO Client environment:java.io.tmpdir=/tmp
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,758] INFO Client environment:java.compiler=<NA>
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,758] INFO Client environment:os.name=Linux
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,758] INFO Client environment:os.arch=amd64
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,758] INFO Client
environment:os.version=2.6.32-279.22.1.el6.x86_64
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,758] INFO Client environment:user.name=cleartrip
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,758] INFO Client environment:user.home=/home/cleartrip
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,758] INFO Client
environment:user.dir=/home/cleartrip/kafka/kafka-0.7.2-incubating-src
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,759] INFO Initiating client connection, connectString=
172.16.18.65:2181 sessionTimeout=30000
watcher=org.I0Itec.zkclient.ZkClient@6926e27(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:19:42,783] INFO Opening socket connection to server /
172.16.18.65:2181 (org.apache.zookeeper.ClientCnxn)
[2013-07-23 20:19:42,788] INFO Socket connection established to zoo1/
172.16.18.65:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2013-07-23 20:19:42,800] INFO Session establishment complete on server
zoo1/172.16.18.65:2181, sessionid = 0x13fc7235423015f, negotiated timeout =
30000 (org.apache.zookeeper.ClientCnxn)
[2013-07-23 20:19:42,802] INFO zookeeper state changed (SyncConnected)
(org.I0Itec.zkclient.ZkClient)
air-DummyProductionConsumerGroup-After,air,1-0
(Group,Topic,BrokerId-PartitionId)
Owner =
air-DummyProductionConsumerGroup-After_d1c2-analytics-r13-17.cltp.com-1374589085082-f88ba95e-0
Consumer offset = 1140946916
= 1,140,946,916 (1.06G)
[2013-07-23 20:19:42,948] ERROR Could not parse broker info
d1c2-analytics-r13-4-1374557209186:d1c2-analytics-r13-4:9092
(kafka.tools.ConsumerOffsetChecker$)
air-DummyProductionConsumerGroup-After,air,1-1
(Group,Topic,BrokerId-PartitionId)
Owner =
air-DummyProductionConsumerGroup-After_d1c2-analytics-r13-17.cltp.com-1374589085082-f88ba95e-1
Consumer offset = 1142891729
= 1,142,891,729 (1.06G)
air-DummyProductionConsumerGroup-After,air,2-0
(Group,Topic,BrokerId-PartitionId)
Owner =
air-DummyProductionConsumerGroup-After_d1c2-analytics-r13-17.cltp.com-1374589174762-cd80342e-0
Consumer offset = 1587266466
= 1,587,266,466 (1.48G)
[2013-07-23 20:19:42,956] ERROR Could not parse broker info
d1c2-analytics-r13-5-1374557226228:d1c2-analytics-r13-5:9092
(kafka.tools.ConsumerOffsetChecker$)
air-DummyProductionConsumerGroup-After,air,2-1
(Group,Topic,BrokerId-PartitionId)
Owner =
air-DummyProductionConsumerGroup-After_d1c2-analytics-r13-17.cltp.com-1374589174762-cd80342e-1
Consumer offset = 1566027800
= 1,566,027,800 (1.46G)
BROKER INFO
[2013-07-23 20:19:42,988] INFO Terminate ZkClient event thread.
(org.I0Itec.zkclient.ZkEventThread)
[2013-07-23 20:19:42,992] INFO EventThread shut down
(org.apache.zookeeper.ClientCnxn)
[2013-07-23 20:19:42,992] INFO Session: 0x13fc7235423015f closed
(org.apache.zookeeper.ZooKeeper)
[cleartrip@d1c2-analytics-r13-4 kafka-0.7.2-incubating-src]$ clear
[cleartrip@d1c2-analytics-r13-4 kafka-0.7.2-incubating-src]$
bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group
air-DummyProductionConsumerGroup-After --zkconnect 172.16.18.65:2181--topic air
[2013-07-23 20:20:12,669] INFO Starting ZkClient event thread.
(org.I0Itec.zkclient.ZkEventThread)
[2013-07-23 20:20:12,679] INFO Client
environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,679] INFO Client environment:host.name=localhost
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,679] INFO Client environment:java.version=1.7.0_21
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,679] INFO Client environment:java.vendor=Oracle
Corporation (org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,679] INFO Client
environment:java.home=/usr/java/jdk1.7.0_21/jre
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,679] INFO Client
environment:java.class.path=:bin/../project/boot/scala-2.8.0/lib/scala-compiler.jar:bin/../project/boot/scala-2.8.0/lib/scala-library.jar:bin/../core/target/scala_2.8.0/kafka-0.7.2.jar:bin/../core/lib/*.jar:bin/../perf/target/scala_2.8.0/kafka-perf-0.7.2.jar:bin/../core/lib_managed/scala_2.8.0/compile/jopt-simple-3.2.jar:bin/../core/lib_managed/scala_2.8.0/compile/log4j-1.2.15.jar:bin/../core/lib_managed/scala_2.8.0/compile/snappy-java-1.0.4.1.jar:bin/../core/lib_managed/scala_2.8.0/compile/zkclient-0.1.jar:bin/../core/lib_managed/scala_2.8.0/compile/zookeeper-3.3.4.jar
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,679] INFO Client
environment:java.library.path=/usr/local/oracle:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,680] INFO Client environment:java.io.tmpdir=/tmp
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,680] INFO Client environment:java.compiler=<NA>
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,680] INFO Client environment:os.name=Linux
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,680] INFO Client environment:os.arch=amd64
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,680] INFO Client
environment:os.version=2.6.32-279.22.1.el6.x86_64
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,680] INFO Client environment:user.name=cleartrip
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,680] INFO Client environment:user.home=/home/cleartrip
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,680] INFO Client
environment:user.dir=/home/cleartrip/kafka/kafka-0.7.2-incubating-src
(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,681] INFO Initiating client connection, connectString=
172.16.18.65:2181 sessionTimeout=30000
watcher=org.I0Itec.zkclient.ZkClient@6926e27(org.apache.zookeeper.ZooKeeper)
[2013-07-23 20:20:12,705] INFO Opening socket connection to server /
172.16.18.65:2181 (org.apache.zookeeper.ClientCnxn)
[2013-07-23 20:20:12,710] INFO Socket connection established to zoo1/
172.16.18.65:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2013-07-23 20:20:12,719] INFO Session establishment complete on server
zoo1/172.16.18.65:2181, sessionid = 0x13fc72354230160, negotiated timeout =
30000 (org.apache.zookeeper.ClientCnxn)
[2013-07-23 20:20:12,722] INFO zookeeper state changed (SyncConnected)
(org.I0Itec.zkclient.ZkClient)
air-DummyProductionConsumerGroup-After,air,1-0
(Group,Topic,BrokerId-PartitionId)
Owner =
air-DummyProductionConsumerGroup-After_d1c2-analytics-r13-17.cltp.com-1374589085082-f88ba95e-0
Consumer offset = 1140946916
= 1,140,946,916 (1.06G)
[2013-07-23 20:20:12,828] ERROR Could not parse broker info
d1c2-analytics-r13-4-1374557209186:d1c2-analytics-r13-4:9092
(kafka.tools.ConsumerOffsetChecker$)
air-DummyProductionConsumerGroup-After,air,1-1
(Group,Topic,BrokerId-PartitionId)
Owner =
air-DummyProductionConsumerGroup-After_d1c2-analytics-r13-17.cltp.com-1374589085082-f88ba95e-1
Consumer offset = 1142891729
= 1,142,891,729 (1.06G)
air-DummyProductionConsumerGroup-After,air,2-0
(Group,Topic,BrokerId-PartitionId)
Owner =
air-DummyProductionConsumerGroup-After_d1c2-analytics-r13-17.cltp.com-1374589174762-cd80342e-0
Consumer offset = 1587266466
= 1,587,266,466 (1.48G)
[2013-07-23 20:20:12,835] ERROR Could not parse broker info
d1c2-analytics-r13-5-1374557226228:d1c2-analytics-r13-5:9092
(kafka.tools.ConsumerOffsetChecker$)
air-DummyProductionConsumerGroup-After,air,2-1
(Group,Topic,BrokerId-PartitionId)
Owner =
air-DummyProductionConsumerGroup-After_d1c2-analytics-r13-17.cltp.com-1374589174762-cd80342e-1
Consumer offset = 1566027800
= 1,566,027,800 (1.46G)
BROKER INFO
[2013-07-23 20:20:12,867] INFO Terminate ZkClient event thread.
(org.I0Itec.zkclient.ZkEventThread)
[2013-07-23 20:20:12,871] INFO EventThread shut down
(org.apache.zookeeper.ClientCnxn)
[2013-07-23 20:20:12,871] INFO Session: 0x13fc72354230160 closed
(org.apache.zookeeper.ZooKeeper)
