Re: Question About Emitted, Transferred and Acked Bolts

Hi Nathan, I have more acked than executed for some bolts and reverse in others. In what scenario will this happen? Regards, Shivendra From: Nathan Leung mailto:ncle...@gmail.com>> Reply-To: "user@storm.apache.org" mailto:user@storm.apache.org>> Date: Monday, Febr

Check pending spouts in real-time

Hi, Is there a way to check the number of spouts pending in storm at any given time or can we derive it from the numbers present in the Storm UI/REST apis? Thanks, Shivendra

RE: Tuples never reach bolts

Verify that you have a shuffleGrouping call on your last Bolt. I experienced the exact same problem earlier this week. builder.setSpout(KAFKA_SPOUT_NAME, new KafkaSpout(kafkaConfig)); builder.setBolt("echo-bolt", new EchoBolt()).shuffleGrouping(KAFKA_SPOUT_NAM

Tuples never reach bolts

Hi, Is someone already faced with tuples that never reach a bolt ? On a project, we have deployed a topology composed of 1 kafka spout and 3 bolts to a cluster. Messages are read from a kafka queue. Then, tuples are handled by a json parser bolt and re-emitted to a third bolt to be transformed.

RE: Why is the toplogy.workers is hardcoded to 1

With 0.9.0.1 we didn’t have to use setNumWorkers or topology.workers. Storm was automatically distributing…I am trying to understand why the upgrade to 0.9.3 is not doing the same? If this is the only way, we can make that change. But just want to be sure. Thank you very much for the help. Fro

Re: Why is the toplogy.workers is hardcoded to 1

I am not sure I follow. You are not setting numWorkers in your topology so by default this will 1 worker. If you deploy a topology it will be assigned to one worker. If you want to distribute the topology among multiple workers add conf.setNumWorkers(desired_workers) On Thu, Feb 26, 2015, at 02

RE: Why is the toplogy.workers is hardcoded to 1

I guess it’s a problem…. I am looking at the following lines in Nimbus.clj in defn compute-new-task->node+port function total-slots-to-use (min (storm-conf TOPOLOGY-WORKERS) (+ (count available-slots) (count alive-assigned))) If the storm-conf does not have the T

RE: Why is the toplogy.workers is hardcoded to 1

No, we are not. I searched once more. One more thing I noticed was, before submitting a topology I changed the log level to debug in the cluster.xml and started the supervisor. I see this in the log file topology.workers" 1. b.s.d.supervisor - Starting Supervisor with conf {"dev.zookeeper.path"

Re: Why is the toplogy.workers is hardcoded to 1

Are you settting numWorkers in you topology config like here https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/WordCountTopology.java#L92 On Thu, Feb 26, 2015, at 12:40 PM, Srividhya Shanmugam wrote: > Thanks for the reply Harsha. We have distributed superv

0.9.4 timeframe

Hi All, any idea what's the targeted timeframe for Storm 0.9.4? Thanks

RE: Why is the toplogy.workers is hardcoded to 1

Thanks for the reply Harsha. We have distributed supervisor nodes (2) and a nimbus node. The storm.yaml file has topology.workers property commented out. When a topology gets submitted that has one spout and a bolt with parallelism hint of 10 for each, before 0.9.3 upgrade storm distributes thi

Re: Why is the toplogy.workers is hardcoded to 1

Srividhya, Storm topologies requires at least one worker to be available to run. Hence the config will set as 1 for the topology.workers as default value. Can you explain in more detail what you are trying to achieve. Thanks, Harsha On Thu, Feb 26, 2015, at 12:12 PM, Srividhya Shanmugam wrote: >

Why is the toplogy.workers is hardcoded to 1

I have commented this property in the storm.yaml. But still it always defaults to 1 after we upgraded storm to 0.9.3. Any idea why its hardcoded? This email and any files transmitted with it are confidential, proprietary and intended solely for the individual or entity to whom they are addressed

Re: Cluster Storm

I'm sure that the port is open because I can open a telnet connection telnet IPServer1 6627 Trying 10.0.8.79... Connected to 10.0.8.79. Escape character is '^]'. Thanks 2015-02-26 13:28 GMT-06:00 Parth Brahmbhatt : > You need to ensure your nimbus port is open for other hosts to connect. > The

Re: Cluster Storm

You need to ensure your nimbus port is open for other hosts to connect. The default value is 6627. Thanks Parth From: Armando Martinez Briones mailto:arma...@tralix.com>> Reply-To: "user@storm.apache.org" mailto:user@storm.apache.org>> Date: Thursday, February 26,

Cluster Storm

Hi. I have a cluster of storm with 3 servers: Server 1: zookeeper, nimbus and one supervisor Server 2: one supervisor Server 3: one supervisor When I start the supervisors on the server 2 and server 3, the Server 1 identify OK the supervisors and I can see on the UI 3 supervisors, but when I depl

Re: Urgent - Some workers stop processing after a few seconds

By the way, I'm using an *unreliable* spout. Here is the source of my spout: https://github.com/millecker/storm-apps/blob/master/commons/src/at/illecker /storm/commons/spout/DatasetSpout.java Maybe this might be the problem? 2015-02-26 18:30 GMT+01:00 Martin Illecker : > Hi, > > I believe this i

Re: Urgent - Some workers stop processing after a few seconds

Hi, I believe this issue belongs to Storm or EC2 because on a single node (one worker) my topology is operating fine. I have tried different combinations of the following parameters: - *shuffleGrouping* and *allGrouping* between the spout and the first bolt - spout parallelism from 1 to numberO

Re: Urgent - Some workers stop processing after a few seconds

Martin, Can't find anything wrong in the logs or in your topologyBuilder code. In your bolts code how are you doing the acking of the tuples. You've maxSpout pending set to 2k tuples do you see any where in your bolt code can be hanging before acking the tuple?. -Harsha On Wed, Feb 25, 2015, at

Profiling Storm Spout and Bolt through YourKit

We are using storm in the product and we currently have different variations of spouts and bolts. I am using yourkit profiler to profile each bolt. While doing so, I am seeing the bolt threads sleep most of the time. The way I am testing is using more tasks for the spout, so it can generate more

Re: Do I need Kafka to have a reliable Storm spout?

Thank you, Forian, for your very clear and extremely clarifying response! |>oug > From: Florian Hussonnois > To: user@storm.apache.org > Cc: > Date: Wed, 25 Feb 2015 14:41:17 +0100 > Subject: Re: Do I need Kafka to have a reliable Storm spout? > Hi, > Actually, the tuples aren't persisted into

Re: Discrepancy in Storm while Job Distribution on available nodes

Hi Andrew, Thanks for your response, well I am already keeping the number of workers equal to the number of worker nodes as I have already set that property, Config config=new Config(); config.put(Config.TOPOLOGY_WORKERS, 2); config.setNumAckers(2); config.setNumWo

Trident with IRichSpout

Hi, I try to run the WordCount topology with a simple IRichSpout implementation (instead of IBatchSpout) but doesn't work. I show the aggregated tuples with a Debug filter, but isKeep never called. It's everything ok with a IBatchSpout. Any ideas ? Thanks in advance!