Re: Configure scheduler for host affinity

Hi! I have made some mistakes. You can see on the comments, that you have to specify in the $STORM_HOME/conf/storm.yaml of the Worker node (not in the Nimbus node): supervisor.scheduler.meta: name: "special-supervisor" Regarding your question, I don't know that there is such a mechanism. Reg

Re: Question on Parallelsim

In that case, I would increase tho numbers of bolts and/or spouts. If your use case permits*, I'd say you can safely increase those numbers. The machine you describe should be able to support about 15 times as much. Study your current performance to see where do you need more power - is your spout

Re: Question on Parallelsim

What would be best to do if you have more than the number of cores? For example, we have 4 spout and 13 bolts and our machine has 32 CPUs with 8 cores each.. *Seungtack Baek | Precocity, LLC* Tel/Direct: (972) 378-1030 | Mobile: (214) 477-5715 *seungtackb...@precocityllc.com * | www.precocityl

Re: Question on Parallelsim

I would say, configure so that your total parallelism matches the number of cores available (i.e. if you have a topology with X spouts, Y boltAs and Z boltBs, make it so that X+Y+Z = cores available). And one worker per machine, inter-JVM communications are expensive. When you have more bolts and

Question on Parallelsim

I was reading on "How many Workers should I use?" (link ) and it suggested us to use parallelism hint that is same as the total number of cores in the cluster. I just want to clarify that this parallelism is solely for

Mixing java/python with trident

hello We have a Trident topology, and some bolts need to execute python code. currently, we call them by WebService but it's harder to scale... Need to modify the parallellism of this bolt Need to add VM of those webservices, Need to modify haproxy to include those new servers... I know it'

State of Inter-worker communication during the course of a running topology after Connection exceptions

Hello All, I had a question regarding the state of the inter-worker communication while a Topology running, after a worker restart. We have observed in our Topology that a worker gets restarted due to a zookeeper heartbeat timing out. The supervisor daemon on that node promptly restarts the worker

Re: Does Storm keep sending tick tuples to bolts when a topology is deactivated?

Hi! AFAIK, "setup-tick!" is called from start of executor (which schedules tick timer for each executor), and tick tuples will be emitted unless worker is going to be shutdown. In short, your fellow is correct. Thanks! Jungtaek Lim (HeartSaVioR) 2015-06-08 21:52 GMT+09:00 Dan Blanchard : > >

Storm Trident for kafka not showing up proper emitted/transfered values in UI

Hello, I am trying to use trident topology for Kafka i.e. *TransactionalTridentKafkaSpout*. All works fine except the Storm UI. Even though I've not produced any data to my topic, the Storm UI keeps showing invalid emitted/transferred values. Meaning the count keeps on increasing even when there i

Re: Does Storm keep sending tick tuples to bolts when a topology is deactivated?

Hi, I have checked the source of Storm executor.clj, ticks is an independent timer thread and it does not care whether the topology is alive or dead. It just send information to `receive-queue` with stream id `_tick`. But I think there is question that only the `spout` executor can receive t

Re: unsubscribe

Ditto.. From: Cynthia Huynh mailto:huynh_cynt...@hotmail.com>> Reply-To: "user@storm.apache.org" mailto:user@storm.apache.org>> Date: Monday, June 8, 2015 at 8:22 AM To: "user@storm.apache.org" mailto:user@storm.apache.org>> Subject: R

RE: unsubscribe

Same here please . From: bhn...@hotmail.com To: user@storm.apache.org Subject: RE: unsubscribe Date: Mon, 8 Jun 2015 09:00:23 + Hi Mark This doesn't work. I had already sent a tons of unsubscribe requests to user-unsubscr...@storm.apache.org, and didn't got any mail back to confirm I'

Does Storm keep sending tick tuples to bolts when a topology is deactivated?

As part of the development of streamparse, we have a BatchingBolt that processes tuples in batches. It's intended for use with things like databases that are more performant when you send things in batches. I've recently proposed switching our BatchingBolt implementation over from using a time

Re: Best Spout implementation for Reading input Data From File

Hello! You can also have a look at this post: http://stackoverflow.com/questions/24413088/storm-max-spout-pending. It might be helpful. Regards, Florin On Sun, Jun 7, 2015 at 4:17 PM, Nathan Leung wrote: > You should emit with a message id, which will prevent too many messages > from being in

RE: unsubscribe

Hi Mark This doesn't work. I had already sent a tons of unsubscribe requests to user-unsubscr...@storm.apache.org, and didn't got any mail back to confirm I'm unsubscribing.I also had contacted the admin, but got no response. It's really frustrating. Can you please help? Thanks Date: Sat, 6 Jun