Github user Lewuathe commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-72618892
This ticket was migrated to
[STORM-650](https://issues.apache.org/jira/browse/STORM-650?jql=project%20%3D%20STORM).
---
If your project is set up for it, you can reply t
Github user Lewuathe closed the pull request at:
https://github.com/apache/storm/pull/338
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is ena
Github user Lewuathe commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-67596928
@harshach Exactly! Although it will take time, but I found that the
approach looks better through previous discussion.
---
If your project is set up for it, you can rep
Github user harshach commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-67593515
@Lewuathe Sorry looks like I am not clear . From above discussion it seems
you want to check kafka zookeeper /topics path to check if a topic exists or
not. As per previ
Github user Lewuathe commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-67581406
@harshach Sorry, I cannot figure out what you mean. Why does storing offset
info to zookeeper path break checking topic existence with `/topics`? User can
access zookeepe
Github user harshach commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-67280407
@Lewuathe can you give details whats the agreed upon implementation here.
We could use curator to use kafka zk /topics to check if the given topic
exists or not. The
Github user Lewuathe commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-66860765
@ptgoetz OK. I'll raise new ticket for changing I/F between kafka and
kafka-spout. Thank you!
---
If your project is set up for it, you can reply to this email and have
Github user ptgoetz commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-66830619
@Lewuathe Here is the exception:
https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/common/UnknownTopicOrPartitionException.scala
I'd rather go t
Github user Lewuathe commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-66713313
I posted this question on kafka mailing list and found the project for
improvement of kafka api now. I think it is good way to propose check-existence
api to this project
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-66410673
Sorry I don't know Kafka, but I feel that it's going weird.
Relying on other component's internal is real things to avoid.
Components try its best to keep compat
Github user Lewuathe commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-66392691
@lazyval +1 to your idea. Making KafkaSpout depend on kafka kafka APIs is
heavier task.
---
If your project is set up for it, you can reply to this email and have your
r
Github user lazyval commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-66127049
Guys, how about using curator api for this particular problem and raising
new ticket *Kafka spout should rely on kafka APIs, rather than perform
low-level manipulations wi
Github user Lewuathe commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-65977818
There does not seems to be consumer API that throws
`UnknownTopicOrPartitionException`.
[UnknownTopicOrPartitionException in
Kafka](https://github.com/apache/ka
Github user Lewuathe commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-65887266
@harshach I understand what you mean. I'll modify codes to check the
existence of topic with Kafka api. But I have a question for confirmation.
Anyway we have to access
Github user Parth-Brahmbhatt commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-65885304
I agree with Sriharsha. The current code relies too heavily on kafka's
internal zk metadata storage. This is fragile, we should not be depending
directly on any o
Github user harshach commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-65884958
@Lewuathe its just one kafka request right? . KafkaSpout makes one request
if that request fails than you'll get a exception and if that exception happens
to be UnknownT
Github user Lewuathe commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-65881172
@harshach Catching UnknownTopicOrPartitionException consumes some network
bandwidth, doesn't it? It will check whether kafka server has the topic or not.
Watching ZK does
Github user harshach commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-65817741
@lazyval we shouldn't be using getAllTopics method instead rely on
exceptions thrown by the Kafka api.
---
If your project is set up for it, you can reply to this email
Github user lazyval commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-65817890
@harshach if that's possible to do, I totally agree
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user lazyval commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-65808799
@Lewuathe there is [similar method in kafka's
ZkUtils](https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/utils/ZkUtils.scala#L749-L755)
---
If your proje
Github user harshach commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-65804581
@Lewuathe Can't we catch UnknowTopicOrPartitionException from kafka do
retries based on that instead of zookeeper watch?
---
If your project is set up for it, you can re
Github user Lewuathe commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-65803225
@lazyval I think all nodes in storm clusters should refer its own state
through zookeeper. This is important to keep consistency of cluster. Sorry I
don't know such kafka
Github user lazyval commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-65802076
Why not use kafka api for performing such check?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your p
GitHub user Lewuathe opened a pull request:
https://github.com/apache/storm/pull/338
[STORM-391] KafkaSpout to await for the topic
KafkaSpout is waiting for creation of target topic.
Curator keeps watching existence of topic path for a given time length.
You can merge this pull
24 matches
Mail list logo
