Re: Gathering opinion: changing multilang heartbeat mechanism

2015-07-14 Thread Dan Blanchard
If the GIL is a problem with both approaches, I think the best course of action would be you just stick with what is already in the Multi-Lang protocol, rather than adding another thing that Storn libraries will need to support. Also, as long as the amount of time that a ShellBolt will wait to

Re: Shell bolt issue

2015-07-09 Thread Dan Blanchard
Hi hermanth, The Python multi-lang implementation provided with Storm is not really production ready for exactly the reasons you outline. At Parse.ly, we use streamparse for our Python Bolts/Spouts. It has built in logging support and also takes over stdin/stdout to prevent you from inadvertent

Re: Gathering opinion: changing multilang heartbeat mechanism

2015-07-09 Thread Dan Blanchard
Hi Jungtaek, Sorry I didn’t notice this earlier, as I was the person who filed STORM–513 in the first place. Having just implemented the new heartbeat protocol in Python (for streamparse) and Perl (for IO::Storm), I’m not crazy about needing to add another heartbeat approach to multiple librar

Does Storm keep sending tick tuples to bolts when a topology is deactivated?

2015-06-08 Thread Dan Blanchard
As part of the development of streamparse, we have a BatchingBolt that processes tuples in batches. It's intended for use with things like databases that are more performant when you send things in batches. I've recently proposed switching our BatchingBolt implementation over from using a time