Re: 答复: Best way to Submit a Job to Storm Topology

Hi Sai and All, What I meant by duplicate issue was that while running the jar through StormSubmitter (which was build through maven) I got duplicate jars present for the Storm which was failing at the job submission. Any workaround or standard practice to avoid the same. Thanks! VM On Mar 30,

Re: KafkaBolt output best practices

Hi John, Increasing the Kafka partition should be helpful. Additionally what is the KafkaBolt parallelism you are keeping? Cheers! VM On Jan 10, 2016 11:33 PM, "John Yost" wrote: > Hi Everyone, > > I am attempting to use the KafkaBolt for output from my Storm topology. >

Re: Possibility of using Storm

> > Thank you for quick reply. How about searching past data? Let's say I want > to put a threshold for API calls only up to last month API calls? Does > Storm support this? If so, what kind of database system does Storm use? > > Thanks, > Dimuthu > > On Tue, Sep 1, 2015 a

Re: Possibility of using Storm

Hi Dimuthu, This is the perfect use case you are owning, the power of storm to leverage real time event processing can be achieved prominently. Make sure you keep your spout parallelism and nodes configuration tuned. Cheers! On Sep 1, 2015 10:09 AM, "DImuthu Upeksha"

Re: Storm Message Flow Question

Hi Seung, You can better refer to the section Stream Groupings in the following link attached below https://storm.apache.org/documentation/Concepts.html It will get you better understanding of the tuple distribution in Storm, for clear understanding here is the pictorial representation of the

Re: Storm Message Flow Question

For having the unique tuple access across the Bolts use shuffle group (otherwise for some specific use case refer to my last mail links), it will distribute the data uniformly across all the bolts without heavily loading any of the bolt, it basically works on the hashing principle, assign the

Re: Is there a way to find the details of the target bolt/bolts and the source bolt/bolts in topology.

Hi, Could you elaborate your use case for that? On Mon, Mar 2, 2015 at 12:01 PM, Ravali Kandur kandu...@umn.edu wrote: Hi, I was wondering if there is a way to know the details of the preceding/succeeding bolts information programmatically. For example, in the image shown below, can I

Re: kafka-storm-cassandra example

Hi Kay, It seems like your daemon connectivity is lacking, or perhaps the port is not opened for zookeeper connectivity. Ensure that all the worker and supervisor are up with access on the specific port. you can Telnet from the machines to check the connectivity! On Mon, Mar 2, 2015 at 7:52

Re: Discrepancy in Storm while Job Distribution on available nodes

-of-a-Storm-topology.html On Wed, Feb 25, 2015 at 9:39 PM, Vineet Mishra clearmido...@gmail.com wrote: Hi Nathan, I guess you missed the first mail of this thread. As mentioned before I m having 3 node cluster out of which node 1 is for nimbus and ui while other 2 nodes are for worker

Re: Discrepancy in Storm while Job Distribution on available nodes

the worker processes in the topology configuration to use the other hardware. On Feb 25, 2015 10:38 AM, Vineet Mishra clearmido...@gmail.com wrote: If I am taking you right, by adding more workers you mean adding more nodes to the existing cluster or/and enhancing the existing configuration

Re: Storm Field Grouping with specific fields

, Vineet Mishra clearmido...@gmail.com wrote: Hi Nathan, Thanks for your revert but eventually what I am following is the same approach you have mentioned but still couldn't get the benefit of parallelism, so just to brief I have 3 node cluster setup with 1 supervisor and 2 workers. After

Re: Storm Field Grouping with specific fields

to curb this gap! Thanks! On Mon, Feb 23, 2015 at 6:30 PM, Nathan Leung ncle...@gmail.com wrote: You can put user and host in separate tuple fields and do fields grouping on those fields. On Feb 23, 2015 6:18 AM, Vineet Mishra clearmido...@gmail.com wrote: I tried looking for a solution

Re: Storm - Client is being closed, and does not take requests any more

, supervisors in background? looks like you are sshing into machines and running ./bin/storm nimbus in foreground which will get killed when you exit the ssh session. Make sure you use supervisord http://supervisord.org/ to run nimbus, supervisors. On Sat, Feb 7, 2015, at 11:04 AM, Vineet Mishra

[no subject]

Hi All, I am running a Kafka Storm topology in distributed mode, its running good for the initial run when I start the cluster(3 node cluster) deploy the Storm Topology and leave it to run. There are often times the whole cluster goes down(nimbus, supervisor, workers) and this is most of the time

Failed to Load Native Library Snappy - org.xerial.snappy.SnappyError

Hi Al, I have recently got stuck into a issue, I was trying to run a Storm Topology with Kafka-Storm as Spout, recently I moved the Storm Cluster on different machines(earlier 3 nodes were sharing both the Kafka and Storm). While I am running the topology again I am ending up in error saying,

Re: Failed to Load Native Library Snappy - org.xerial.snappy.SnappyError

, Vineet Mishra clearmido...@gmail.com wrote: Hi Al, I have recently got stuck into a issue, I was trying to run a Storm Topology with Kafka-Storm as Spout, recently I moved the Storm Cluster on different machines(earlier 3 nodes were sharing both the Kafka and Storm). While I am running

Re: Storm Kafka Processing

and incase of null it chooses round-robin to distribute among partitions. Its better to use a random UUID to distribute among all of your partitions. -Harsha On Tue, Feb 3, 2015, at 12:44 AM, Vineet Mishra wrote: Do you mean to say that the event published to Kafka is not partition

Re: Storm Kafka Processing

AM, Vineet Mishra wrote: Hi, I am running Kafka Storm Engine to process real time data generated on a 3 node distributed cluster. Currently I have set 10 Executors for Storm Spout, which I don't think is running in parallel. Moreover earlier I was running the Kafka Topology with Replication

Re: Storm Kafka Processing

, Vineet Mishra wrote: Hi Harsha, I am using storm.kafka.KafkaSpout.KafkaSpout implementation from https://github.com/wurstmeister/storm-kafka-0.8-plus Thanks! On Mon, Feb 2, 2015 at 8:14 PM, Harsha st...@harsha.io wrote: Vineet, Which kafka spout are you using? -Harsha On Mon

Re: Storm Job picking cached/old jar file

should we schedule the job in distributed mode Looking out for quick response. Thanks in advance! On Wed, Jan 28, 2015 at 11:49 AM, Vineet Mishra clearmido...@gmail.com wrote: Hi Jens, No its not referring to the old jars in the log(bcoz that has been already deleted) rather its picking

Re: Storm Job picking cached/old jar file

, 2015 at 7:24 AM, Vineet Mishra clearmido...@gmail.com wrote: Well thanks all, I got it working, it seems that the topology jar itself was having the topology in the build path, the reason of which it was referring the old code. I got it working but only in local mode. Moreover I was looking out

Re: Storm Job picking cached/old jar file

! On Tue, Jan 27, 2015 at 11:07 PM, Jens-Uwe Mozdzen jmozd...@nde.ag wrote: Hi Vineet, Zitat von Vineet Mishra clearmido...@gmail.com: Hi Naresh and Jens, Well first I tried running a job in local mode that was running good, but I wanted to run it in distributed environment, later I killed