Hi,
We are able to connect and consume from the remote Kafka topic while
running it locally using eclipse. To make sure storm does not uses the
in-memory zk we have used the overloaded constructor [
LocalCluster("zkServer",port) ], it's working fine and we can see the data
coming. This lead us to conclude that version compatibility might not be
the issue here.
However still no luck when deployed the topology in cluster.
We have verified the connectivity from storm box to zkservers
The znode seems fine also ..
At this point really need some pointers here, what could possibly be wrong
with this and how do we debug that? Never worked with Kafka 0.10x before so
not sure what exactly are we missing.
Really appreciate some help and suggestions
Thanks
On Tue, Jun 13, 2017 at 12:59 AM, Amardeep <[email protected]>
wrote:
> Hi All,
>
> We are trying to connect the new Kafka 0.10.x using Storm 0.10.x using
> Storm Kafka Spout. Getting the below exception as trying to connect to the
> Kafka cluster:
>
> java.lang.RuntimeException: java.lang.RuntimeException:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /brokers/topics/test_topic/partitions
> at
> storm.kafka.DynamicBrokersReader.getBrokerInfo(DynamicBrokersReader.java:81)
> ~[stormjar.jar:?]
> at storm.kafka.trident.ZkBrokerReader.<init>(ZkBrokerReader.java:42)
> ~[stormjar.jar:?]
> at storm.kafka.KafkaUtils.makeBrokerReader(KafkaUtils.java:57)
> ~[stormjar.jar:?]
> at storm.kafka.KafkaSpout.open(KafkaSpout.java:87) ~[stormjar.jar:?]
> at backtype.storm.daemon.executor$fn__5541$fn__5556.invoke(executor.clj:558)
> ~[storm-core-0.10.0.2.4.3.2-1.jar:0.10.0.2.4.3.2-1]
> at backtype.storm.util$async_loop$fn__545.invoke(util.clj:477)
> [storm-core-0.10.0.2.4.3.2-1.jar:0.10.0.2.4.3.2-1]
>
> Also :
>
> 2017-06-12 14:57:40.281 o.a.c.ConnectionState [ERROR] Connection timed out
> for connection string (
> zk_server1
> :2181, zk_server
> 2
> :2181, zk_server
> 3
> :2181) and timeout (15000) / elapsed (15025)
> org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
> ConnectionLoss
> at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:198)
> [stormjar.jar:?]
> at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:88)
> [stormjar.jar:?]
>
> the version we have used is :
>
> For Kafka: 0.10.0.1
> Storm: 0.10.0.2.4.3.2-1 ( this is the version we are using in stage )
> Kafka Spout version: 0.9.3
>
> With this we are successfully able to consume from a lower Kafka version
> 0.9.x
> The Zookeeper connectivity is fine, we are able to telnet to the ZK
> servers and confirm the path exists. The topic is valid and we can use the
> Kafka's console-consumer.sh script to consume from it. However with storm
> it is not working.
>
> Would like to know is it possible to use this version of storm to connect
> to the new Kafka API or are we missing something here.
>
>
> --
> Amardeep Sarkar
>
>
>
> --
> Amardeep Sarkar
>
--
Amardeep Sarkar
