Storm-Kafka, TransactionalSpout, last version dependencies

2014-05-23 Thread Romain Leroux
Hi, I am currently testing storm 0.9.1-incubating and the only working KafkaSpout for Kafka 0.8.x that I found was : groupIdnet.wurstmeister.storm/groupId artifactIdstorm-kafka-0.8-plus/artifactId Sadly, I have never been able to get the TransactionalSpout working as soon as I also used some

Re: Is it possible join another spou/bolt later to the topology?

2014-05-23 Thread Sajith
Hi Gosh, My concern is that I don't want to include all the dependencies required by the bulky bolt to the topology jar. Because that bolt will run in single node(I wrote a scheduler to guarantee that) in the cluster it will be not required to distribute the jar across the cluster. Actually, was

Re: Storm-Kafka, TransactionalSpout, last version dependencies

2014-05-23 Thread Irek Khasyanov
I’m using storm-plus 0.4 with zookeeper 3.3.6. If you using storm 0.9.1-incubating, you should downgrade storm-plus. Or use storem 0.9.2-incubating which compatible with new zookeeper and curator — With best regards, Irek Khasyanov

Re: Sample project using Storm and Kafka (using Scala 2.10)

2014-05-23 Thread János Háber
If only the interfaces used from kafka (looked superficially) the provided scope will be good idea... (both to scala and kafka dependencies) János Háber Fine Solution Ltd On Thu, May 22, 2014 at 6:35 PM, P. Taylor Goetz ptgo...@gmail.com wrote: Come to think about it, this is a non-issue.

Re: Sample project using Storm and Kafka (using Scala 2.10)

2014-05-23 Thread János Háber
I change the issue ( https://issues.apache.org/jira/browse/STORM-325 )... János Háber Fine Solution Ltd On Fri, May 23, 2014 at 10:21 AM, János Háber janos.ha...@finesolution.huwrote: If only the interfaces used from kafka (looked superficially) the provided scope will be good idea... (both

Extend Supervisors

2014-05-23 Thread Matteo Nardelli
I'd like to implement a new application logic in a distributed manner and I was wondering if extending the Supervisor process is the best way of doing it. Basically I want to implement a distributed scheduler and Nimbus peculiarities seems to be too restrictive. Is there a way to extend the

kafka-storm-starter released: code examples that integrate Kafka 0.8 and Storm 0.9

2014-05-23 Thread Michael G. Noll
Hi everyone, to sweeten the upcoming long weekend I have released code examples that show how to integrate Kafka 0.8+ with Storm 0.9+, while using Apache Avro as the data serialization format. https://github.com/miguno/kafka-storm-starter Since the integration of the latest Kafka and Storm

Re: Sample project using Storm and Kafka (using Scala 2.10)

2014-05-23 Thread Michael G. Noll
Marco, Take a look at kafka-storm-starter: https://github.com/miguno/kafka-storm-starter The code is Scala 2.10. Best, Michael On 21.05.2014, at 22:32, Marco zentrop...@yahoo.co.uk wrote: Hi, I'm having some troubles understanding how to boostrap a Kafka + Storm project.

Config maxClientCnxns=100 in LocalCluster

2014-05-23 Thread Max, Shen
Hi all, I'm running some tests on LocalCluster and having issues on connecting to the local zookeeper created by the LocalCluster when the test submitTopology. I've resolved the issue in remote mode by increasing the maxClientCnxns number in the zoo.cfg on my zookeeper box. However, I couldn't

Workers constantly restarted due to session timeout

2014-05-23 Thread Michael Dev
Hi all, We are seeing our workers constantly being killed by Storm with to the following logs: worker: 2014-05-23 20:15:08 INFO ClientCxn:1157 - Client session timed out, have not heard from the server in 28105ms for sessionid 0x14619bf2f4e0109, closing socket and attempting reconnect

Building Storm

2014-05-23 Thread Justin Workman
I have seen this question a couple of times, but for the life of me, I cannot get around it and have not seen an answer that resolves it. Here is the process I am walking through. # git clone https://github.com/apache/incubator-storm # cd incubator-storm # git checkout master Already on 'master'

Re: Workers constantly restarted due to session timeout

2014-05-23 Thread Derek Dagit
2) Is this expected behavior for Storm to be unable to keep up with heartbeat threads under high CPU or is our theory incorrect? Check your JVM max heap size (-Xmx). If you use too much, the JVM will garbage-collect, and that will stop everything--including the thread whose job it is to do

Conceptual question on Streams definition...

2014-05-23 Thread P Ghosh
My definition of stream is continuous feed of data of certain type or with certain purpose (depends on how you want to define your process) I have a situation, where the Domain Object is same across the whole topology, however, each component working on bits and pieces to construct the final

Conceptual question on Streams definition...

2014-05-23 Thread P Ghosh
My definition of stream is continuous feed of data of certain type or with certain purpose (depends on how you want to define your process) I have a situation, where the Domain Object is same across the whole topology, however, each component working on bits and pieces to construct the final