Re: Storm with Python

[Andrew Montalenti]

We are building a new Storm and Python interop option that is called
streamparse:

https://github.com/Parsely/streamparse

It includes a heavily rewritten Storm interop library and a command line
tool, sparse, for managing local and remote Storm clusters. The idea is to
make Storm projects as easy to build and manage in Python as RQ or Celery
projects.

It currently has support for running local clusters in a single command,
managing virtualenvs on remote worker machines, submitting topologies,
listing/killing topologies, and tailing remote log files. The multilang
layer also has better support for logging and exception/error handling.
Multiple topologies can be built from a single codebase and multiple remote
Storm clusters can be supported via a simple JSON configuration file.

We are already using it for production topologies atop Storm 0.9.1 and
Storm 0.8. We welcome contributions and if you join our mailing list, feel
free to make requests. We continue to develop it actively and in an open
manner.

-Andrew Montalenti
CTO, Parse.ly
On May 29, 2014 6:35 PM, "Ashu Goel"  wrote:

> (the reason being is that we are still running Python 2.6 but Petrel is
> only compatible with 2.7)
> On May 29, 2014, at 2:48 PM, Ashu Goel  wrote:
>
> Awesome! I'm looking more into using the storm.thrift to define a non-JVM
> DSL... does anyone have any working examples of this? Python preferred but
> any example will do. the wiki is a bit confusing...
> On May 28, 2014, at 1:54 PM, FRANCISCO JESUS GOMEZ RODRIGUEZ <
> franciscojesus.gomezrodrig...@telefonica.com> wrote:
>
> Ashu, take a look this project: http://github.com/AirSage/Petrel
>
> Write, submit, debug and monitor in python.
>
> @ffranz
> El 28/05/2014 22:49, Ashu Goel  escribió:
>  Any examples where the entire infra is written in Python (including
> topology)? or is that not possible
>  On May 28, 2014, at 1:33 PM, Dilpreet Singh 
> wrote:
>
>
> https://github.com/apache/incubator-storm/tree/master/examples/storm-starter
>
>  The WordCountTopology contains an example python bolt.
>
>  Regards,
> Dilpreet
>
>
> On Thu, May 29, 2014 at 1:59 AM, Ashu Goel  wrote:
>
>> Does anyone have a good example program/instructions of using Python with
>> storm? I can't seem to find anything concrete online.
>>
>> Thanks,
>> Ashu Goel
>
>
>
>
> --
>
> Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario,
> puede contener información privilegiada o confidencial y es para uso
> exclusivo de la persona o entidad de destino. Si no es usted. el
> destinatario indicado, queda notificado de que la lectura, utilización,
> divulgación y/o copia sin autorización puede estar prohibida en virtud de
> la legislación vigente. Si ha recibido este mensaje por error, le rogamos
> que nos lo comunique inmediatamente por esta misma vía y proceda a su
> destrucción.
>
> The information contained in this transmission is privileged and
> confidential information intended only for the use of the individual or
> entity named above. If the reader of this message is not the intended
> recipient, you are hereby notified that any dissemination, distribution or
> copying of this communication is strictly prohibited. If you have received
> this transmission in error, do not read it. Please immediately reply to the
> sender that you have received this communication in error and then delete
> it.
>
> Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário,
> pode conter informação privilegiada ou confidencial e é para uso exclusivo
> da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário
> indicado, fica notificado de que a leitura, utilização, divulgação e/ou
> cópia sem autorização pode estar proibida em virtude da legislação vigente.
> Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique
> imediatamente por esta mesma via e proceda a sua destruição
>
>
>
>

Re: Storm with Python

[Andrew Montalenti]

For one thing, a recently accepted Storm pull request has made this
serialization pluggable and someone has already implemented a protobuf
variety. We plan to investigate alternative serialization options for
multilang once we get the other tooling out of the way.

For another, it is true the overhead for serialization is non trivial, but
the overhead also tends to be a constant factor applied to data size, and
machines are cheap while programming time is expensive. Storm and Python's
data analysis and data integration libraries are a pretty powerful combo
worth the performance penalty.
On May 30, 2014 1:42 PM, "Larry Palmer"  wrote:

> We had experimented with Storm/Python 6 months ago or so, but found the
> JSON serialization/deserialization overhead was quite high, on the order of
> several hundred usec per tuple every time it transitioned from java to
> python or vice versa, limiting total throughput on a 12 core server to
> around 25k tuples/second. Considered trying to switch to a different
> serializer but ended up just doing everything in Java instead.
>
> Is that still the case, or perhaps has the speed been improved?
>
>
> On Thu, May 29, 2014 at 10:06 PM, Andrew Montalenti 
> wrote:
>
>> We are building a new Storm and Python interop option that is called
>> streamparse:
>>
>> https://github.com/Parsely/streamparse
>>
>> It includes a heavily rewritten Storm interop library and a command line
>> tool, sparse, for managing local and remote Storm clusters. The idea is to
>> make Storm projects as easy to build and manage in Python as RQ or Celery
>> projects.
>>
>> It currently has support for running local clusters in a single command,
>> managing virtualenvs on remote worker machines, submitting topologies,
>> listing/killing topologies, and tailing remote log files. The multilang
>> layer also has better support for logging and exception/error handling.
>> Multiple topologies can be built from a single codebase and multiple remote
>> Storm clusters can be supported via a simple JSON configuration file.
>>
>> We are already using it for production topologies atop Storm 0.9.1 and
>> Storm 0.8. We welcome contributions and if you join our mailing list, feel
>> free to make requests. We continue to develop it actively and in an open
>> manner.
>>
>> -Andrew Montalenti
>> CTO, Parse.ly
>> On May 29, 2014 6:35 PM, "Ashu Goel"  wrote:
>>
>>> (the reason being is that we are still running Python 2.6 but Petrel is
>>> only compatible with 2.7)
>>> On May 29, 2014, at 2:48 PM, Ashu Goel  wrote:
>>>
>>> Awesome! I'm looking more into using the storm.thrift to define a
>>> non-JVM DSL... does anyone have any working examples of this? Python
>>> preferred but any example will do. the wiki is a bit confusing...
>>> On May 28, 2014, at 1:54 PM, FRANCISCO JESUS GOMEZ RODRIGUEZ <
>>> franciscojesus.gomezrodrig...@telefonica.com> wrote:
>>>
>>>  Ashu, take a look this project: http://github.com/AirSage/Petrel
>>>
>>> Write, submit, debug and monitor in python.
>>>
>>> @ffranz
>>> El 28/05/2014 22:49, Ashu Goel  escribió:
>>>  Any examples where the entire infra is written in Python (including
>>> topology)? or is that not possible
>>>  On May 28, 2014, at 1:33 PM, Dilpreet Singh 
>>> wrote:
>>>
>>>
>>> https://github.com/apache/incubator-storm/tree/master/examples/storm-starter
>>>
>>>  The WordCountTopology contains an example python bolt.
>>>
>>>  Regards,
>>> Dilpreet
>>>
>>>
>>> On Thu, May 29, 2014 at 1:59 AM, Ashu Goel  wrote:
>>>
>>>> Does anyone have a good example program/instructions of using Python
>>>> with storm? I can't seem to find anything concrete online.
>>>>
>>>> Thanks,
>>>> Ashu Goel
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario,
>>> puede contener información privilegiada o confidencial y es para uso
>>> exclusivo de la persona o entidad de destino. Si no es usted. el
>>> destinatario indicado, queda notificado de que la lectura, utilización,
>>> divulgación y/o copia sin autorización puede estar prohibida en virtud de
>>> la legislación vigente. Si ha recibido este mensaje por error, le rogamos
>>> que nos lo comunique inmediatamente por esta misma vía y proceda a su
>>> destrucción.
>>>
>>>

Re: Storm with Python

[Andrew Montalenti]

We decided to start with topology definitions in Clojure because a) that
ensures that the topologies can support 100% of Storm's Clojure DSL
out-of-the-box and b) that allows easy mixing of Python, Java, Clojure, and
even other multi-lang bolts. For example, we plan on producing example
topologies that use Python bolts for processing, but use the built-in JVM
Kafka spout as a performant data integration option.

Above and beyond a nice bundling of the Clojure DSL provided with Storm,
streamparse provides:

i) *sparse*, a command-line tool that can quickstart a Storm project; run local
Storm clusters (using LocalCluster under the hood) in a single command;
manage virtualenvs on remote worker machines; submit topologies to remote
Nimbus nodes via SSH tunneling; list and kill topologies using the same;
tail remote log files across a cluster of Storm worker machines. So, in
short -- it provides a *whole lot* and it's still in very early releases.
It provides for a rapid development cycle that Pythonistas are used to when
using technologies like Storm. It makes Storm feel as lightweight as
something like RQ or Celery.

ii) An actively-developed *roadmap*. Planned future features include remote
debugging of crashed Storm bolts using pdb-over-a-socket; visualization
tools for Storm topologies; and higher-level Bolt patterns, such as a
BatchingBolt we have been using at Parse.ly that allows for simple time-
and data-based batching patterns with proper ack/fail semantics abstracted.

iii) The Python *streamparse *package, which is an improved multi-lang
layer that has better support for logging and exception/error handling. It
was rewritten from scratch but modeled on Storm's bundled storm.py to
implement the basics of IPC. In the future, it might also support
serializations beyond plain JSON.

I'm personally of the opinion that writing DSLs for Storm topologies in
languages other than Clojure is a bit of a rabbit hole. Doable -- but may
not be worth it, and the pro's of topologies defined in Clojure outweigh
the con's. But I can understand a widespread distaste for Clojure and even
"multi-language" projects as a concept.

We'd gladly accept contributions for a (simple) Python DSL that maps down
to Clojure's DSL. I think trying to build a "native" DSL using the Thrift
structures ends up with a lot of complexity for little gain. If you're
interested in that sort of thing, some other projects have been floating
around that attempt it -- but if you want my opinion, you're in for a world
of pain :)


On Fri, May 30, 2014 at 7:59 PM, Ashu Goel  wrote:

> Andrew,
>
> From what I understand streamparse still requires that the topologies be
> in Clojure... not entirely sure how this is different from what storm already
> provides. I was looking more for a DSL that we could use w/ Python 2.6 and
> be 100% Python, but it looks like that is not available.
>
> -Ashu
>
> On May 30, 2014, at 2:14 PM, Andrew Montalenti  wrote:
>
> For one thing, a recently accepted Storm pull request has made this
> serialization pluggable and someone has already implemented a protobuf
> variety. We plan to investigate alternative serialization options for
> multilang once we get the other tooling out of the way.
>
> For another, it is true the overhead for serialization is non trivial, but
> the overhead also tends to be a constant factor applied to data size, and
> machines are cheap while programming time is expensive. Storm and Python's
> data analysis and data integration libraries are a pretty powerful combo
> worth the performance penalty.
> On May 30, 2014 1:42 PM, "Larry Palmer"  wrote:
>
>> We had experimented with Storm/Python 6 months ago or so, but found the
>> JSON serialization/deserialization overhead was quite high, on the order of
>> several hundred usec per tuple every time it transitioned from java to
>> python or vice versa, limiting total throughput on a 12 core server to
>> around 25k tuples/second. Considered trying to switch to a different
>> serializer but ended up just doing everything in Java instead.
>>
>> Is that still the case, or perhaps has the speed been improved?
>>
>>
>> On Thu, May 29, 2014 at 10:06 PM, Andrew Montalenti 
>> wrote:
>>
>>> We are building a new Storm and Python interop option that is called
>>> streamparse:
>>>
>>> https://github.com/Parsely/streamparse
>>>
>>> It includes a heavily rewritten Storm interop library and a command line
>>> tool, sparse, for managing local and remote Storm clusters. The idea is to
>>> make Storm projects as easy to build and manage in Python as RQ or Celery
>>> projects.
>>>
>>> It currently has support for running local clusters in a s

Re: Writing Bolts in Python

[Andrew Montalenti]

The ShellBolt looks for "scancount.py" in the resources/ directory in your
JAR, which will be extracted to each worker machine. It then simply invokes
"python scancount.py" in that directory. So you need to make sure the
scancount.py file will be on the classpath under resources/, as well the
storm.py interop library it depends upon.

Based on the official word count example
<https://github.com/apache/incubator-storm/blob/master/examples/storm-starter/src/jvm/storm/starter/WordCountTopology.java#L40-L56>,
your Java ShellBolt definition looks OK.

The storm.py interop library that you're probably using then communicates
with the rest of Storm via the Multi-lang Protocol
<https://storm.incubator.apache.org/documentation/Multilang-protocol.html>.
This means your Python process is really sending JSON messages over stdout
and receiving JSON messages over stdin. That's the relationship between
Python & Java (& Storm) in this case.

A library I'm working on with my team, streamparse
<https://github.com/Parsely/streamparse>, makes this workflow easier by
bundling upon a command-line tool for building/submitting/running Python
topologies. For example, getting a Storm + Python "wordcount" example to
run locally is just a matter of:

sparse quickstart wordcount
cd wordcount
sparse run

It also eliminates the need to write the Java glue code you're putting
together here. It's still in early development but we're already using it
for real Storm 0.8 and 0.9 production clusters & local development.

---
Andrew Montalenti
Co-Founder & CTO
http://parse.ly



On Mon, Jun 2, 2014 at 12:37 PM, Ashu Goel  wrote:

> Hi all,
>
> I am experimenting with writing bolts in Python and was wondering how the
> relationship between the Java and Python code works. For example, I have a
> Python bolt that looks like this:
>
> class ScanCountBolt(storm.BasicBolt):
>
>   def __init__(self):
> #super(ScanCountBolt, self).__init__(script='scancount.py')
> self._count = defaultdict(int)
>
>   def process(self, tup):
> product = tup.values[0]
> self._count[product] += 1
> storm.emit([product, self._count[product]])
>
> ScanCountBolt().run()
>
>
> And my corresponding Java code looks like this:
>
>  public static class ScanCount extends ShellBolt implements IRichBolt {
>
> public ScanCount() {
>   super("python", "scancount.py");
> }
>
> @Override
> public void declareOutputFields(OutputFieldsDeclarer declarer) {
>   declarer.declare(new Fields("product", "scans"));
> }
>
> @Override
>  public Map getComponentConfiguration() {
>return null;
>  }
>   }
>
> Is that all I need to make it work or do I need to declare the data
> structures in the Java code as well. I am a bit confused...
>
> -Ashu

Re: MultiLag (Python) bolt gives error

[Andrew Montalenti]

On the worker machine, do you have Python installed? You can check by
running "python -V". You need to ensure you're using the same $PATH as
whatever environment is running your Storm supervisor/worker. From the
exception stack trace, it looks like your ShellBolt does not see a python
interpreter on the $PATH, therefore it can't run your Python bolt.

At least, that's how I read "Caused by: java.io.IOException: Cannot run
program "python" (in directory
"/tmp/5f3c8318-14da-4999-a974-584a9d200fdb/supervisor/stormdist/mongo_20140528_02
-1-1401786854/resources"): error=2, No such file or directory".


On Wed, Jun 4, 2014 at 7:33 PM, Hamza Asad  wrote:

> Help required plz.. I'm facing this issue while using pytjon bolt..
> Haven't resolved it yet.. Anyone having solution
>
> *** This message has been sent using QMobile A500 ***
>
>
> Hamza Asad  wrote:
>
> I have checked that resources folder is NOT placed in location
>
> */tmp/6a090639-b975-42b8-8bc1-8de6093ad3e1/supervisor/stormdist/mongo_20140528_02-1-1401881161/resources*
> There are only two files i.e stormcode.ser  stormconf.ser but NO resources
> folder. Why? How can i resolve this issue. Im using *storm*
> *0.9.1-incubating* and compiling code using netbeans.
>
>
> On Tue, Jun 3, 2014 at 2:31 PM, Hamza Asad  wrote:
>
>> Hi,
>> I'm using python bolt which is in the resource directory but storm giving
>> me error
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *7226 [Thread-19-exclaim1] INFO  backtype.storm.daemon.executor -
>> Preparing bolt exclaim1:(3) 7231 [Thread-10] INFO
>> backtype.storm.daemon.executor - Loading executor exclaim1:[4 4]7232
>> [Thread-19-exclaim1] ERROR backtype.storm.util - Async loop
>> died!java.lang.RuntimeException: Error when launching multilang subprocess
>> at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:105)
>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]at
>> backtype.storm.daemon.executor$eval5170$fn__5171$fn__5183.invoke(executor.clj:689)
>> ~[na:na] at backtype.storm.util$async_loop$fn__390.invoke(util.clj:431)
>> ~[na:na]at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
>> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55]Caused by:
>> java.io.IOException: Cannot run program "python" (in directory
>> "/tmp/5f3c8318-14da-4999-a974-584a9d200fdb/supervisor/stormdist/mongo_20140528_02-1-1401786854/resources"):
>> error=2, No such file or directory at
>> java.lang.ProcessBuilder.start(ProcessBuilder.java:1041) ~[na:1.7.0_55]
>> at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:50)
>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]at
>> backtype.storm.task.ShellBolt.prepare(ShellBolt.java:102)
>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] ... 4 common frames
>> omittedCaused by: java.io.IOException: error=2, No such file or
>> directoryat java.lang.UNIXProcess.forkAndExec(Native Method)
>> ~[na:1.7.0_55]at java.lang.UNIXProcess.(UNIXProcess.java:135)
>> ~[na:1.7.0_55] at java.lang.ProcessImpl.start(ProcessImpl.java:130)
>> ~[na:1.7.0_55]at
>> java.lang.ProcessBuilder.start(ProcessBuilder.java:1022) ~[na:1.7.0_55]
>> ... 6 common frames omitted7233 [Thread-10] INFO
>> backtype.storm.daemon.task - Emitting: exclaim1 __system ["startup"] 7233
>> [Thread-10] INFO  backtype.storm.daemon.executor - Loaded executor tasks
>> exclaim1:[4 4]7233 [Thread-19-exclaim1] ERROR
>> backtype.storm.daemon.executor - java.lang.RuntimeException: Error when
>> launching multilang subprocess at
>> backtype.storm.task.ShellBolt.prepare(ShellBolt.java:105)
>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]at
>> backtype.storm.daemon.executor$eval5170$fn__5171$fn__5183.invoke(executor.clj:689)
>> ~[na:na] at backtype.storm.util$async_loop$fn__390.invoke(util.clj:431)
>> ~[na:na]at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
>> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55]Caused by:
>> java.io.IOException: Cannot run program "python" (in directory
>> "/tmp/5f3c8318-14da-4999-a974-584a9d200fdb/supervisor/stormdist/mongo_20140528_02-1-1401786854/resources"):
>> error=2, No such file or directory at
>> java.lang.ProcessBuilder.start(ProcessBuilder.java:1041) ~[na:1.7.0_55]
>> at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:50)
>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]at
>> backtype.storm.task.ShellBolt.prepare(ShellBolt.java:102)
>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] ... 4 common frames
>> omittedCaused by: java.io.IOException: error=2, No such file or
>> directoryat java.lang.UNIXProcess.forkAndExec(Native Method)
>> ~[na:1.7.0_55]at java.lang.UNIXProcess.(UNIXProcess.java:135)
>> ~[na:1.7.0_55] at java.lang.ProcessImpl.start(ProcessImpl.java:130)
>> ~[na:1.7.0_55]at
>> java.lang.ProcessBuilder.start(ProcessBuilder.java:1022) ~[na:1.7.0_55]   *
>>
>

Re: Multilang shell bolts exceptions

[Andrew Montalenti]

What multi-lang driver for PHP are you using? There is a way for multi-lang
drivers to report errors up to Storm (see ShellBolt's handling of the
"error" command here
).
Perhaps your driver is not making use of this facility and that's why your
exceptions are being buried?


On Thu, Jun 5, 2014 at 11:46 AM, adiya n  wrote:

> If there is a failure in my php code in one of the shell bolts exec
> process, the error isn't obvious  in all the cases when the topology comes
> up /handles the request.
>
> Only in certain types of failures I see my exceptions getting printed but
> for some, it just doesnt do anything.
>
> here is a simple example:
> this is part of my code in my shell_bolt.php  under multilang/resources:
>
> try {
>  .
>  .
>   $fh = fopen("/tmp/pytrace.txt", 'a') or die("Can't open file");
>   fwrite($fh, "hello world");
>   fwrite($fh, print_r($tuple->values));
>   fclose($fh);
>   $this->emit(array($tuple->values[0], $word));
>   } catch (Exception $e) {
> throw new Exception("exception " . $e);
>   }
>
> the problem area is print_r() function. If I remove print_r() and just
> print $tuple->values, everything works fine and the shell bolt emits the
> word.  With print_r, it just hangs and my drpc calls timeout .
>
> What is the best way to figure out what is going on with the shell bolt
> exec process??
>
>
> thanks!!
>
>
>   On , adiya n  wrote:
>
>
> Should have looked at the wiki/links better. Troubleshooting guide
> helped.  thanks!
>
>
>   On Wednesday, June 4, 2014 11:29 PM, adiya n 
> wrote:
>
>
> Hello all,
>
> What do negative id values mean in the logs?  Does that indicate any error?
> I got these while running a local topology.  Also, is there any good
> documentation about understanding the logs. Most of it is self-explanatory
> from the log messages but if there is any existing documentation somewhere,
> that will really help.
>
> 5433 [Thread-19] INFO  backtype.storm.daemon.executor  - Processing
> received message source: prepare-request:7, stream: default, id: {-
> 4212877050522192524=6354306792350206104}, [3323165896464350833, 33]
> Tuple received by bolt is source: prepare-request:7, stream: default, id:
> {-4212877050522192524=6354306792350206104}, [3323165896464350833, 33]
> 5433 [Thread-10] INFO  backtype.storm.daemon.worker  - Worker
> 445b9038-ac6f-4629-a86f-82cd5eecf0e3 for storm drpc-demo-1-1401943429 on
> 9b5a127d-1a02-4362-883e-f5193c6f7a92:4 has finished loading
> 5434 [Thread-36] INFO  backtype.storm.daemon.executor  - Processing
> received message source: spout:8, stream: __ack_init, id: {},
> [-4212877050522192524 -4291234877071288404 8]
> 5434 [Thread-19] INFO  backtype.storm.daemon.task  - Emitting: bolt0
> default [3323165896464350833,  MyBolt is emitting: ]
> 5434 [Thread-36] INFO  backtype.storm.daemon.executor  - Processing
> received message source: prepare-request:7, stream: __ack_ack, id: {}, [-
> 4212877050522192524 340043085440436800]
> 5435 [Thread-19] INFO  backtype.storm.daemon.task  - Emitting: bolt0
> __ack_ack [-4212877050522192524 5906040704921991109]
>
> thanks
> Aiya
>
>
>
>
>
>

Re: python via java+scriptengine+jython

[Andrew Montalenti]

Don't see why this wouldn't be possible. I assume a ScriptEngine
implementation would be using jython, so the usual caveats about jython
apply (e.g. you don't have access to all 3rd-party libraries and you tend
to be behind in Python versions). What kind of comparison are you looking
for, a performance-oriented one? It's bound to be faster, I suppose,
because it can avoid multi-lang serialization/deserialization, although
jython isn't exactly widely-renowned for its interpreter speed.


On Wed, Jun 4, 2014 at 11:36 AM, Tyson Norris  wrote:

> Hi -
> I've seen several people attempting to run python bolts via multilang, and
> wondered if anyone has tried and can compare with running some python
> functions via java ScriptEngine? That is uses a normal java bolt that calls
> python functions via ScriptEngine.
>
> We are experimenting with this approach (script engine), but don't have
> anything useful built yet.
>
> Thanks
> Tyson

Re: Cluster Failure and Data Recovery

[Andrew Montalenti]

Sounds like you might benefit from considering something like Kafka instead
of a standard MQ. We have some notes about this

publicly online from our PyData talk on Kafka/Storm. You can configure
Kafka to have an SLA on data that is in terms of data size or time; if your
entire topology crashes or goes down, then you can resume messages at the
spout from the moment the failure happened, and pay no penalty.

(Of course, then you need to figure out how to guarantee your Kafka plant
is always online, but this is do-able given its distributed architecture.)

This doesn't sound like a problem that Storm should think about solving --
after all, if your entire Storm cluster fails, all of the high availability
guarantees of each component are, by definition, out the window.


On Thu, Jun 5, 2014 at 2:08 PM, Nima Movafaghrad <
nima.movafagh...@oracle.com> wrote:

> Thanks Srinath. We are already using the reliable message processing for
> bolts failure etc. My problem is with a catastrophic cases. For example,
>  what happens if the entire cluster goes down or what if the Topology fully
> fails. At the moment we are reading from MQ and although keeping the
> transactions open would resolve our data loss prevention issue it isn't
> quiet feasible. Some of our bolts listen and batch for up to 30 seconds so
> they have big enough batches that can be committed to RDBMS. Keeping the
> transactions open for that long slows things down considerably.
>
>
>
> So I guess to  frame to question better I should ask, if there a way to
> persist the intermediate data?
>
>
>
> Thanks,
>
> Nima
>
>
>
> *From:* Srinath C [mailto:srinat...@gmail.com]
> *Sent:* Wednesday, June 04, 2014 5:49 PM
> *To:* user
> *Subject:* Re: Cluster Failure and Data Recovery
>
>
>
> Hi Nima,
>
> Use the reliable message processing
>  
> mechanism
> to ensure that there is no data loss. You would need support for
> transactional semantics from the tuple source where spout can commit/abort
> a read (kestrel, kafka, rabbitmq, etc can do this). Yes you would need to
> keep the queue transactions open until the spout receives an "ack" or
> "fail" for every tuple.
>
> IMO, this ensures that each tuple is processed "atleast once" and not
> "exactly once" so you need to be prepared to end up with duplicate entries
> in your DB or have a way to figure out that a write to DB is duplicate or
> earlier write. This is case where there are crashes with intermediate data
> in memory.
>
>
>
> Regards,
>
> Srinath.
>
>
>
>
>
> On Thu, Jun 5, 2014 at 5:37 AM, Nima Movafaghrad <
> nima.movafagh...@oracle.com> wrote:
>
> Hi everyone,
>
>
>
> We are in the process of designing a high available system with zero data
> loss tolerance. Plan is for the spouts to read from a queue and process
> them down in several different specialized bolts and then flush to DB. How
> can we guarantee no data loss here? Should we keep the queue transactions
> open until data is committed to DB? Should we persist the state of all the
> bolts? What happens to the intermediate data if the whole cluster fails?
>
>
>
> Any suggestions are much appreciated.
>
>
>
> Nima
>
>
>

Re: MultiLag (Python) bolt gives error

[Andrew Montalenti]

I suppose the other possibility is that you don't have a "resources/"
directory on your CLASSPATH at compile-time, when you end up building your
JAR for submission to the Storm cluster. You could verify this by
inspecting the JAR you are about to submit to the nimbus. Do something like
this:

mkdir foo
mv foo.jar foo/
cd foo
unzip foo.jar
ls resources/

If you have a resources/ directory, then your JAR is being packaged up
correctly. If you don't, that explains it.

This is described in the "Packaging your stuff" section in the multi-lang
protocol
<https://storm.incubator.apache.org/documentation/Multilang-protocol.html>
docs.


On Thu, Jun 5, 2014 at 6:41 PM, Hamza Asad  wrote:

> Thnx for reply.. Yes i have installed python on worker node. and i have
> checked at that specific directory too.. I guess the issue is that storm
> looking for resources folder in desired path but its not there. Why? How n
> where should i give its path ? Assigning which variable?
>
> *** This message has been sent using QMobile A500 ***
>
> Andrew Montalenti  wrote:
>
> On the worker machine, do you have Python installed? You can check by
> running "python -V". You need to ensure you're using the same $PATH as
> whatever environment is running your Storm supervisor/worker. From the
> exception stack trace, it looks like your ShellBolt does not see a python
> interpreter on the $PATH, therefore it can't run your Python bolt.
>
> At least, that's how I read "Caused by: java.io.IOException: Cannot run
> program "python" (in directory
> "/tmp/5f3c8318-14da-4999-a974-584a9d200fdb/supervisor/stormdist/mongo_20140528_02
> -1-1401786854/resources"): error=2, No such file or directory".
>
>
> On Wed, Jun 4, 2014 at 7:33 PM, Hamza Asad  wrote:
>
>> Help required plz.. I'm facing this issue while using pytjon bolt..
>> Haven't resolved it yet.. Anyone having solution
>>
>> *** This message has been sent using QMobile A500 ***
>>
>>
>> Hamza Asad  wrote:
>>
>> I have checked that resources folder is NOT placed in location
>>
>> */tmp/6a090639-b975-42b8-8bc1-8de6093ad3e1/supervisor/stormdist/mongo_20140528_02-1-1401881161/resources*
>> There are only two files i.e stormcode.ser  stormconf.ser but NO
>> resources folder. Why? How can i resolve this issue. Im using *storm*
>> *0.9.1-incubating* and compiling code using netbeans.
>>
>>
>> On Tue, Jun 3, 2014 at 2:31 PM, Hamza Asad 
>> wrote:
>>
>>> Hi,
>>> I'm using python bolt which is in the resource directory but storm
>>> giving me error
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *7226 [Thread-19-exclaim1] INFO  backtype.storm.daemon.executor -
>>> Preparing bolt exclaim1:(3) 7231 [Thread-10] INFO
>>> backtype.storm.daemon.executor - Loading executor exclaim1:[4 4]7232
>>> [Thread-19-exclaim1] ERROR backtype.storm.util - Async loop
>>> died!java.lang.RuntimeException: Error when launching multilang subprocess
>>> at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:105)
>>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]at
>>> backtype.storm.daemon.executor$eval5170$fn__5171$fn__5183.invoke(executor.clj:689)
>>> ~[na:na] at backtype.storm.util$async_loop$fn__390.invoke(util.clj:431)
>>> ~[na:na]at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
>>> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55]Caused by:
>>> java.io.IOException: Cannot run program "python" (in directory
>>> "/tmp/5f3c8318-14da-4999-a974-584a9d200fdb/supervisor/stormdist/mongo_20140528_02-1-1401786854/resources"):
>>> error=2, No such file or directory at
>>> java.lang.ProcessBuilder.start(ProcessBuilder.java:1041) ~[na:1.7.0_55]
>>> at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:50)
>>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]at
>>> backtype.storm.task.ShellBolt.prepare(ShellBolt.java:102)
>>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] ... 4 common frames
>>> omittedCaused by: java.io.IOException: error=2, No such file or
>>> directoryat java.lang.UNIXProcess.forkAndExec(Native Method)
>>> ~[na:1.7.0_55]at java.lang.UNIXProcess.(UNIXProcess.java:135)
>>> ~[na:1.7.0_55] at java.lang.ProcessImpl.start(ProcessImpl.java:130)
>>> ~[na:1.7.0_55]at
>>> java.lang.ProcessBuilder.start(ProcessBuilder.java:1022) ~[na:1.7.0_55]
>>> ... 6 common frames omitted7233 [Thread-10] INFO
>>> backtype.storm.daemon.task - Emitting: exclaim1 __system ["startup"] 7233
>>> [Thread-10] INFO  backtype.storm.daemon.executor - Loaded executor tasks
>>> exclaim1:[4 4]7233 [Thread-19-exclaim1] ERROR
>>> backtype.storm.daemon.executor - java.lang.RuntimeException: Error when
>>> launching multilang subprocess at
>>> backtype.storm.task.ShellBolt.prepare(ShellBolt.java:105)
>>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]at
>>> backtype.storm.daemon.executor$eval5170$fn__5171$fn__5183.invoke(executor.clj:689)*
>>>
>>

Re: [VOTE] Storm Logo Contest - Final Round

[Andrew Montalenti]

#9 - 5 pts.


On Mon, Jun 9, 2014 at 2:38 PM, P. Taylor Goetz  wrote:

> This is a call to vote on selecting the winning Storm logo from the 3
> finalists.
>
> The three candidates are:
>
>  * [No. 6 - Alec Bartos](
> http://storm.incubator.apache.org/2014/04/23/logo-abartos.html)
>  * [No. 9 - Jennifer Lee](
> http://storm.incubator.apache.org/2014/04/29/logo-jlee1.html)
>  * [No. 10 - Jennifer Lee](
> http://storm.incubator.apache.org/2014/04/29/logo-jlee2.html)
>
> VOTING
>
> Each person can cast a single vote. A vote consists of 5 points that can
> be divided among multiple entries. To vote, list the entry number, followed
> by the number of points assigned. For example:
>
> #1 - 2 pts.
> #2 - 1 pt.
> #3 - 2 pts.
>
> Votes cast by PPMC members are considered binding, but voting is open to
> anyone. In the event of a tie vote from the PPMC, votes from the community
> will be used to break the tie.
>
> This vote will be open until Monday, June 16 11:59 PM UTC.
>
> - Taylor
>

Supervisor kills *all* workers for topology due to heartbeat :timed-out state

[Andrew Montalenti]

I am trying to understand why for a topology I am trying to run on
0.9.1-incubating, the supervisor on the machine is killing *all* of the
topology's Storm workers periodically.

Whether I use topology.workers=1,2,4, or 8, I always get logs like this:

https://gist.github.com/amontalenti/cd7f380f716f1fd17e1b

Which basically indicates that the supervisor thinks all the workers timed
out at exactly the same time, and then it kills them all.

I've tried tweaking the worker timeout seconds, bumping it up to e.g. 120
secs, but this hasn't helped at all. No matter what, periodically, the
workers just get whacked by the supervisor and the whole topology has to
restart.

I notice that this does happen less frequently if the machine is under less
load, e.g. if I drop topology.max.spout.pending *way* down, to e.g. 100 or
200, then it runs for awhile without crashing. But I've even seen it crash
in this state.

I saw on some other threads that people indicated that the supervisor will
kill all workers if "the nimbus fails to see a heartbeat from zookeeper".
Could someone walk me through how I could figure out if this is the case?
Nothing in the logs seems to point me in this direction.

Thanks!

Andrew

Re: Extracting Performance Metrics

[Andrew Montalenti]

I haven't used it yet, but a lot of people get pointed to metrics_storm:

https://github.com/ooyala/metrics_storm

With this blog post that discusses it:

http://engineering.ooyala.com/blog/open-sourcing-metrics-storm

Michael Noll also has a nice blog post about streaming Storm 0.9 metrics to
Graphite:

http://www.michael-noll.com/blog/2013/11/06/sending-metrics-from-storm-to-graphite/

Currently, when we use Storm, we do a lot of custom metrics in Graphite
using statsd, as described in this post (not about Storm, but about
Graphite/statsd):

http://codeascraft.com/2011/02/15/measure-anything-measure-everything/




On Mon, Jun 16, 2014 at 4:37 PM, Anis Nasir  wrote:

> Dear all,
>
> I am running a cluster with 1 kafka + 1 nimbus + 10 supervisor + 1
> zookeeper nodes. I am executing multiple topologies on the cluster and I
> want to extract different metrics that I am mentioning below. Can someone
> help me by recommending tools that I can use to extract this information.
>
>
> Per Topology
>  - Throughput
>  - Latency
>
> Per Spout or Bolt
>  - Throughput
>  - Latency
>  - Execution Time
>  - Queuing Time
>  - Number of Messages Processed
>
> Regards
> Anis
>

Re: Extracting Performance Metrics

[Andrew Montalenti]

Also, I came across this presentation by Visible Measures which actually
walks through a lot of great options covering most of what you want to know
about:

http://files.meetup.com/5809742/storm%20monitoring.pdf

One other thing to be aware of is that in Storm 0.9.2 (forthcoming
release), there is a new REST API used by the Storm UI for gathering some
of these metrics:

https://github.com/apache/incubator-storm/pull/101
https://issues.apache.org/jira/browse/STORM-205


On Mon, Jun 16, 2014 at 6:13 PM, Andrew Montalenti 
wrote:

> I haven't used it yet, but a lot of people get pointed to metrics_storm:
>
> https://github.com/ooyala/metrics_storm
>
> With this blog post that discusses it:
>
> http://engineering.ooyala.com/blog/open-sourcing-metrics-storm
>
> Michael Noll also has a nice blog post about streaming Storm 0.9 metrics
> to Graphite:
>
>
> http://www.michael-noll.com/blog/2013/11/06/sending-metrics-from-storm-to-graphite/
>
> Currently, when we use Storm, we do a lot of custom metrics in Graphite
> using statsd, as described in this post (not about Storm, but about
> Graphite/statsd):
>
> http://codeascraft.com/2011/02/15/measure-anything-measure-everything/
>
>
>
>
> On Mon, Jun 16, 2014 at 4:37 PM, Anis Nasir  wrote:
>
>> Dear all,
>>
>> I am running a cluster with 1 kafka + 1 nimbus + 10 supervisor + 1
>> zookeeper nodes. I am executing multiple topologies on the cluster and I
>> want to extract different metrics that I am mentioning below. Can someone
>> help me by recommending tools that I can use to extract this information.
>>
>>
>> Per Topology
>>  - Throughput
>>  - Latency
>>
>> Per Spout or Bolt
>>  - Throughput
>>  - Latency
>>  - Execution Time
>>  - Queuing Time
>>  - Number of Messages Processed
>>
>> Regards
>> Anis
>>
>
>

Re: Python - When spout dies, bolts keep receiving input on STDIN

[Andrew Montalenti]

I believe this is a problem in the storm.py multilang prototype adapter
bundled with Storm. We fixed this issue in a more full fledged multilang
adapter that is available here:

https://github.com/Parsely/streamparse

You could try pip install streamparse and change your bolt to subclass
streamparse.bolt.Bolt instead and see if the problem goes away. Full API
docs here:

http://streamparse.readthedocs.org/en/latest/api.html
On Jun 11, 2014 3:27 PM, "Scot Kronenfeld"  wrote:

> I am using Storm w/ my bolts and spouts written in Python.
>
> When I am running in a test environment locally I have a problem where if
> the spout dies, the bolts consume 100% CPU and gradually increase their
> memory.  Here are the details:
>
> My spout is reading from mongo.  Sometimes it loses its cursor (due to a
> network hiccup or something) and it raises an exception and bails out.  I
> can also reliably reproduce this problem by using "kill -9 ".
>
> Using strace and then a debugger, I figured out that the bolts are stuck
> in this tight loop in the readMsg function inside storm.py (which ships
> with storm)
>
> while True:
> line = sys.stdin.readline()[:-1]
> if line == "end":
> break
> msg = msg + line + "\n"
>
> The readline() call is a blocking call, but the bolt keeps getting blank
> lines as input.
>
> Note: the memory problem is because newlines keep getting appended.  Since
> the input is used as JSON, it would probably be safe to just remove the
> addition of the newline (I'm not 100% positive because that might not work
> if there are newlines in an embedded string within the JSON).  But that
> still doesn't fix the core issue.
>
> I think the problem is that if the spout does not come down cleanly,
> something in the Java keeps sending input to the bolts.  I'm about to dig
> into the Java code but I don't know any Java so I figured it was worth a
> quick message to the Storm list to see if this is a known problem.  Or even
> if anyone has a pointer for where to look in the Java.  I haven't
> previously looked at the Storm source - it's just been a black box to this
> point for me.
>
> Thanks,
> scot
>

Re: Python - When spout dies, bolts keep receiving input on STDIN

[Andrew Montalenti]

Also looks like a recent pull request fixes this issue in storm.py:

https://github.com/apache/incubator-storm/pull/140/files


On Tue, Jun 17, 2014 at 7:33 AM, Andrew Montalenti 
wrote:

> I believe this is a problem in the storm.py multilang prototype adapter
> bundled with Storm. We fixed this issue in a more full fledged multilang
> adapter that is available here:
>
> https://github.com/Parsely/streamparse
>
> You could try pip install streamparse and change your bolt to subclass
> streamparse.bolt.Bolt instead and see if the problem goes away. Full API
> docs here:
>
> http://streamparse.readthedocs.org/en/latest/api.html
> On Jun 11, 2014 3:27 PM, "Scot Kronenfeld"  wrote:
>
>> I am using Storm w/ my bolts and spouts written in Python.
>>
>> When I am running in a test environment locally I have a problem where if
>> the spout dies, the bolts consume 100% CPU and gradually increase their
>> memory.  Here are the details:
>>
>> My spout is reading from mongo.  Sometimes it loses its cursor (due to a
>> network hiccup or something) and it raises an exception and bails out.  I
>> can also reliably reproduce this problem by using "kill -9 ".
>>
>> Using strace and then a debugger, I figured out that the bolts are stuck
>> in this tight loop in the readMsg function inside storm.py (which ships
>> with storm)
>>
>> while True:
>> line = sys.stdin.readline()[:-1]
>> if line == "end":
>> break
>> msg = msg + line + "\n"
>>
>> The readline() call is a blocking call, but the bolt keeps getting blank
>> lines as input.
>>
>> Note: the memory problem is because newlines keep getting appended.
>> Since the input is used as JSON, it would probably be safe to just remove
>> the addition of the newline (I'm not 100% positive because that might not
>> work if there are newlines in an embedded string within the JSON).  But
>> that still doesn't fix the core issue.
>>
>> I think the problem is that if the spout does not come down cleanly,
>> something in the Java keeps sending input to the bolts.  I'm about to dig
>> into the Java code but I don't know any Java so I figured it was worth a
>> quick message to the Storm list to see if this is a known problem.  Or even
>> if anyone has a pointer for where to look in the Java.  I haven't
>> previously looked at the Storm source - it's just been a black box to this
>> point for me.
>>
>> Thanks,
>> scot
>>
>

v0.9.2-incubating and .ser files

[Andrew Montalenti]

I built the v0.9.2-incubating rc-3 locally and once verifying that it
worked for our topology, pushed it into our cluster. So far, so good.

One thing for the community to be aware of. If you try to upgrade an
existing v0.9.1-incubating or 0.8 cluster to v0.9.2-incubating, you may hit
exceptions upon nimbus/supervisor startup about stormcode.ser/stormconf.ser.

The issue is that the new cluster will try to re-submit the topologies that
were already running before the upgrade. These will fail because Storm's
Clojure version has been upgraded from 1.4 -> 1.5, thus the serialization
formats & IDs have changed. This would be true basically if any class
serial IDs change that happen to be in these .ser files (stormconf.ser &
stormcode.ser, as defined in Storm's internal config

).

The solution is to clear out the storm data directories on your worker
nodes/nimbus nodes and restart the cluster.

I have some open source tooling that submits topologies to the nimbus using
StormSubmitter. This upgrade also made me realize that due to the use
of serialized
Java files
,
it is very important the StormSubmitter class used for submitting and the
running Storm cluster be precisely the same version / classpath. I describe
this more in the GH issue here:

https://github.com/Parsely/streamparse/issues/27

I wonder if maybe it's worth it to consider using a less finicky
serialization format within Storm itself. Would that change be welcome as a
pull request?

It would make it easier to script Storm clusters without consideration for
client/server Storm version mismatches, which I presume was the original
reasoning behind putting Storm functionality behind a Thrift API anyway.
And it would prevent crashed topologies during minor Storm version upgrades.

Re: v0.9.2-incubating and .ser files

[Andrew Montalenti]

Another interesting 0.9.2 issue I came across: the IConnection interface
has changed, meaning any pluggable transports no longer work without a code
change.

I implemented changes to storm-0mq to get it to be compatible with this
interface change in my fork here.

https://github.com/Parsely/storm-0mq/compare/ptgoetz:master...master

I tested that and it nominally works in distributed mode with two
independent workers in my cluster. Don't know what the performance impact
is of the interface change.

I get that zmq is no longer part of storm core, but maintaining a stable
interface for pluggable components like this transport is probably
something that should be in the release test suite. Otherwise bitrot will
take its toll. I am glad to volunteer help with this.

My team is now debugging an issue where Storm stops asking our spout for
next tuples after awhile of running the topology, causing the tool go to
basically freeze with no errors in the logs. At first blush, seems like a
regression from 0.9.1. But we'll have more detailed info once we isolate
some variables soon.
On Jun 18, 2014 4:32 PM, "Andrew Montalenti"  wrote:

> I built the v0.9.2-incubating rc-3 locally and once verifying that it
> worked for our topology, pushed it into our cluster. So far, so good.
>
> One thing for the community to be aware of. If you try to upgrade an
> existing v0.9.1-incubating or 0.8 cluster to v0.9.2-incubating, you may hit
> exceptions upon nimbus/supervisor startup about stormcode.ser/stormconf.ser.
>
> The issue is that the new cluster will try to re-submit the topologies
> that were already running before the upgrade. These will fail because
> Storm's Clojure version has been upgraded from 1.4 -> 1.5, thus the
> serialization formats & IDs have changed. This would be true basically if
> any class serial IDs change that happen to be in these .ser files
> (stormconf.ser & stormcode.ser, as defined in Storm's internal config
> <https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/config.clj#L143-L153>
> ).
>
> The solution is to clear out the storm data directories on your worker
> nodes/nimbus nodes and restart the cluster.
>
> I have some open source tooling that submits topologies to the nimbus
> using StormSubmitter. This upgrade also made me realize that due to the use
> of serialized Java files
> <https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtype/storm/utils/Utils.java#L73-L97>,
> it is very important the StormSubmitter class used for submitting and the
> running Storm cluster be precisely the same version / classpath. I describe
> this more in the GH issue here:
>
> https://github.com/Parsely/streamparse/issues/27
>
> I wonder if maybe it's worth it to consider using a less finicky
> serialization format within Storm itself. Would that change be welcome as a
> pull request?
>
> It would make it easier to script Storm clusters without consideration for
> client/server Storm version mismatches, which I presume was the original
> reasoning behind putting Storm functionality behind a Thrift API anyway.
> And it would prevent crashed topologies during minor Storm version upgrades.
>

Re: v0.9.2-incubating and .ser files

[Andrew Montalenti]

FYI, the issue happened with both zmq and netty transports. We will
investigate more tomorrow. We think the issue only happens with more than
one supervisor and multiple workers.
On Jun 19, 2014 7:32 PM, "P. Taylor Goetz"  wrote:

> Hi Andrew,
>
> Thanks for pointing this out. I agree with your point about bit rot.
>
> However, we had to remove the the 0mq transport due to license
> incompatibilities with Apache, so any kind of release test suite would have
> to be maintained outside of Apache since it would likely pull in
> LGPL-licensed dependencies. So if something like you're suggesting could be
> accomplished in the storm-0mq project, that would be the best option.
>
> I'm open to pull requests, help, contributions, etc. to storm-0mq. It just
> can't be part of Apache.
>
> I'll test out your changes to storm-0mq to see if I can reproduce the
> issue you're seeing. As Nathan mentioned, any additional information
> (thread dumps, etc.) you could provide would help.
>
> Thanks (and sorry for the inconvenience),
>
> Taylor
>
>
> On Jun 19, 2014, at 6:09 PM, Andrew Montalenti  wrote:
>
> Another interesting 0.9.2 issue I came across: the IConnection interface
> has changed, meaning any pluggable transports no longer work without a code
> change.
>
> I implemented changes to storm-0mq to get it to be compatible with this
> interface change in my fork here.
>
> https://github.com/Parsely/storm-0mq/compare/ptgoetz:master...master
>
> I tested that and it nominally works in distributed mode with two
> independent workers in my cluster. Don't know what the performance impact
> is of the interface change.
>
> I get that zmq is no longer part of storm core, but maintaining a stable
> interface for pluggable components like this transport is probably
> something that should be in the release test suite. Otherwise bitrot will
> take its toll. I am glad to volunteer help with this.
>
> My team is now debugging an issue where Storm stops asking our spout for
> next tuples after awhile of running the topology, causing the tool go to
> basically freeze with no errors in the logs. At first blush, seems like a
> regression from 0.9.1. But we'll have more detailed info once we isolate
> some variables soon.
> On Jun 18, 2014 4:32 PM, "Andrew Montalenti"  wrote:
>
>> I built the v0.9.2-incubating rc-3 locally and once verifying that it
>> worked for our topology, pushed it into our cluster. So far, so good.
>>
>> One thing for the community to be aware of. If you try to upgrade an
>> existing v0.9.1-incubating or 0.8 cluster to v0.9.2-incubating, you may hit
>> exceptions upon nimbus/supervisor startup about stormcode.ser/stormconf.ser.
>>
>> The issue is that the new cluster will try to re-submit the topologies
>> that were already running before the upgrade. These will fail because
>> Storm's Clojure version has been upgraded from 1.4 -> 1.5, thus the
>> serialization formats & IDs have changed. This would be true basically if
>> any class serial IDs change that happen to be in these .ser files
>> (stormconf.ser & stormcode.ser, as defined in Storm's internal config
>> <https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/config.clj#L143-L153>
>> ).
>>
>> The solution is to clear out the storm data directories on your worker
>> nodes/nimbus nodes and restart the cluster.
>>
>> I have some open source tooling that submits topologies to the nimbus
>> using StormSubmitter. This upgrade also made me realize that due to the use
>> of serialized Java files
>> <https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtype/storm/utils/Utils.java#L73-L97>,
>> it is very important the StormSubmitter class used for submitting and the
>> running Storm cluster be precisely the same version / classpath. I describe
>> this more in the GH issue here:
>>
>> https://github.com/Parsely/streamparse/issues/27
>>
>> I wonder if maybe it's worth it to consider using a less finicky
>> serialization format within Storm itself. Would that change be welcome as a
>> pull request?
>>
>> It would make it easier to script Storm clusters without consideration
>> for client/server Storm version mismatches, which I presume was the
>> original reasoning behind putting Storm functionality behind a Thrift API
>> anyway. And it would prevent crashed topologies during minor Storm version
>> upgrades.
>>
>
>

Re: v0.9.2-incubating and .ser files

[Andrew Montalenti]

Really good news here. I dug into this issue more -- including doing a
detailed analysis with stack traces that I put up on Github:

https://gist.github.com/amontalenti/8ff0c31a7b95a6dea3d2
(now updated with the below e-mail's text since the exploration was
obsoleted by the fix)

The issue had nothing to do with Storm and everything to do with Ubuntu
14.04 and its interaction with Xen network kernel drivers in EC2.

I was staring at the results of this research and thinking, "What could
possibly cause the network subsystem of Storm to just hang?"

My first impulse: firewalls. Maybe as the network was ramping up, I was
hitting up against a firewall rule?

I checked our munin monitoring graphs and noticed a bunch of eth0 errors
correlated with our topologies running. I checked our production Storm
0.8.2 cluster -- no errors. Ah hah! It must be firewall rules or something!

That led me to run dmesg on the supervisor nodes. I found a bunch of
entries like this:

xen_netfront: xennet: skb rides the rocket: 20 slots
xen_netfront: xennet: skb rides the rocket: 19 slots

That's odd. I also saw some entries related to ufw (Ubuntu's firewall
service). So, I try running `ufw disable`. No change.

I then dig in more to these error messages and I come across this open bug
on Launchpad:

https://bugs.launchpad.net/ubuntu/+source/linux-lts-raring/+bug/1195474

I dig in there and come across the current workaround, running:

sudo ethtool -K eth0 sg off

On the server. I issue that command, restart my topology, and VOILA, the
Storm topology is *now running at full performance*.

Back in my earliest days as a professional programmer, I had a friend named
Jimmy. I once spent 3 days debugging a JVM garbage collection issue with
him. We ran profilers, did detailed code traces, extensive logging, etc.
And in the end, the fix to the problem was a single line of code change --
a mistaken allocation of an expensive object that was happening in a tight
loop. At that moment, I coined "Jimmy's Law", which is:

"The amount of time it takes to discover a bug's fix is inversely
proportional to the lines of code changed by the fix, with infinite time
converging to one line."

After hours of investigating and debugging this issue, that's certainly how
I feel. Shame on me for upgrading my Storm cluster and Ubuntu version
simultaneously!

Now for the really good news: I'm 14 million tuples into my Storm
0.9.2-incubating cluster (running with Netty) and everything is humming
along, running fast. 92 tasks, 8 workers, 2 supervisors. My simplest Python
bolt has 1.10ms process latencies -- some of the fastest I've seen.

Thanks for the help investigating, and here's to an awesome 0.9.2 release!

p.s. I'm glad that *something* positive came out of this, at least -- my
contribution to sync up the storm-0mq driver for those who prefer it. Glad
to continue to help with that, if for no other reason than to have a
reliable second transport so Storm community members can debug *actual* Netty
issues they may come across.


On Thu, Jun 19, 2014 at 8:53 PM, P. Taylor Goetz  wrote:

> Okay. Keep me posted. I still plan on looking at and testing your patch to
> storm-0mq, but probably won't get to that until early next week.
>
> -Taylor
>
> On Jun 19, 2014, at 7:43 PM, Andrew Montalenti  wrote:
>
> FYI, the issue happened with both zmq and netty transports. We will
> investigate more tomorrow. We think the issue only happens with more than
> one supervisor and multiple workers.
> On Jun 19, 2014 7:32 PM, "P. Taylor Goetz"  wrote:
>
>> Hi Andrew,
>>
>> Thanks for pointing this out. I agree with your point about bit rot.
>>
>> However, we had to remove the the 0mq transport due to license
>> incompatibilities with Apache, so any kind of release test suite would have
>> to be maintained outside of Apache since it would likely pull in
>> LGPL-licensed dependencies. So if something like you're suggesting could be
>> accomplished in the storm-0mq project, that would be the best option.
>>
>> I'm open to pull requests, help, contributions, etc. to storm-0mq. It
>> just can't be part of Apache.
>>
>> I'll test out your changes to storm-0mq to see if I can reproduce the
>> issue you're seeing. As Nathan mentioned, any additional information
>> (thread dumps, etc.) you could provide would help.
>>
>> Thanks (and sorry for the inconvenience),
>>
>> Taylor
>>
>>
>> On Jun 19, 2014, at 6:09 PM, Andrew Montalenti 
>> wrote:
>>
>> Another interesting 0.9.2 issue I came across: the IConnection interface
>> has changed, meaning any pluggable transports no longer work without a code
>> change.
>>
>> I implemented changes to storm-0mq to get it

Re: Unit testing topologies

[Andrew Montalenti]

It's somewhat old but there is a great series of blog posts about testing
Storm topologies using Clojure:

http://www.pixelmachine.org/2011/12/17/Testing-Storm-Topologies.html

and if you take a look at Storm's testing.clj file, there are some great
Clojure utilities used for testing within Storm itself:

https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/testing.clj

I learned a bunch by reading the multilang test cases here as a starting
point:

https://github.com/apache/incubator-storm/blob/master/storm-core/test/clj/backtype/storm/multilang_test.clj

and then learning more about the Clojure DSL:

http://storm.incubator.apache.org/documentation/Clojure-DSL.html

Of course, if you're a Java programmer, you may be looking at all this
Clojure-based testing infrastructure and saying, "Great, now I have two
problems -- my unit tests, and learning Clojure." Indeed :) But, Clojure is
a pretty nice language and worth learning for use cases like this.

---
Andrew Montalenti
Co-Founder & CTO
http://parse.ly


On Thu, Jul 3, 2014 at 3:41 AM, Richards Peter 
wrote:

> Hi,
>
> Could anyone suggest me some more good materials for unit testing storm
> topologies?
>
> I found the following code:
>
> https://github.com/xumingming/storm-lib/blob/master/src/jvm/storm/TestingApiDemo.java
>
> I would be nice if anyone can present a talk about unit testing topologies
> in any meetup/event. Will it be possible for anyone to conduct such
> sessions?
>
> Thanks,
> Richards Peter.
>

Re: Choosing where your tasks run in Storm

[Andrew Montalenti]

I don't think this is possible right now, though I have thought about the
same thing before. It *might* be true that Storm's support for YARN could
eventually lead to this kind of thing, but I don't know much about it. For
now, you're best off having separate Storm clusters for different classes
of machines. You could consider putting Kafka queues between them to ensure
cross-topology message reliability guarantees. (e.g. have your I/O bound
topology read from kafka and write to kafka, and have your CPU-bound
topology read from the Kafka topic produced by the first queue).

---
Andrew Montalenti
Co-Founder & CTO
http://parse.ly

On Fri, Jul 4, 2014 at 7:59 AM, jeff saremi  wrote:

> I'm wondering if this concept applies to Storm and if there's a way to do
> this.
>
> I'd like to limit the machines that certain spouts or bolts run on. There
> are many reasons for this. But for one let's assume that I have a bolt
> that is just a proxy for some legacy service. I want to monitor that
> service by way of the bolt and use it in my topology.
> Another way of looking at it is that I want to have a topology that spans
> different "classes" of machines.
> Let's say I have 3 classes of machines: small, medium, and large. Some
> topologies are limited to only one class of machines however some other
> topologies need to span two or more classes of machines.
> How can I do this in storm?
> Thanks
> Jeff
>

Re: b.s.m.n.Client [INFO] Reconnect

[Andrew Montalenti]

fyi, you won't be able to change to zeromq in 0.9.2 from Netty unless you
use this updated storm-0mq module that I forked:

https://github.com/Parsely/storm-0mq/blob/master/README.md

I also added a detailed README with instructions. We'll get these changes
merged into ptgoetz's repo eventually so it is all in one place.

The truth is, though, there shouldn't be a good reason to use zeromq in
0.9.2 over Netty.

I built that module because I thought Netty was being slow, but it was just
a red herring for another issue.
On Jul 10, 2014 1:15 PM, "Harsha"  wrote:

>  Storm 0mq package is here https://github.com/ptgoetz/storm-0mq . You
> need to add that package in STORM_HOME/lib
> and add this config to storm.yaml
>
> storm.messaging.transport: "backtype.storm.messaging.zmq"
>
>
> On Thu, Jul 10, 2014, at 10:02 AM, Suparno Datta wrote:
>
> anyone here knows how to switch to zmq from netty? Just wanted to check
> that once before going down to 0.8.1.
>
>
> On 10 July 2014 18:46, Suparno Datta  wrote:
>
> @Stephan, Worked like a charm. How stupid of me not to change the local
> directory.
>
> @Harsha. Didnt solve the original problem :( .
>
> Now getting this ones
>
> #.s.m.n.Client [INFO] Reconnect started for
> Netty-Client-cluster1-fos-ThinkPad-T520/10.42.0.21:6700... [11]
>
> and after 30 retries finally the worker crashes
>
> 14-07-10 18:44:43 b.s.m.n.Client [INFO] Closing Netty Client
> Netty-Client-cluster1-fos-ThinkPad-T520/10.42.0.21:6700
> 2014-07-10 18:44:43 b.s.m.n.Client [INFO] Waiting for pending batchs to be
> sent with Netty-Client-cluster1-fos-ThinkPad-T520/10.42.0.21:6700...,
> timeout: 60ms, pendings: 0
> 2014-07-10 18:44:43 b.s.util [ERROR] Async loop died!
> java.lang.RuntimeException: java.lang.RuntimeException: Client is being
> closed, and does not take requests any more
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.disruptor$consume_loop_STAR_$fn__758.invoke(disruptor.clj:94)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55]
>  Caused by: java.lang.RuntimeException: Client is being closed, and does
> not take requests any more
> at backtype.storm.messaging.netty.Client.send(Client.java:194)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at backtype.storm.utils.TransferDrainer.send(TransferDrainer.java:54)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__5927$fn__5928.invoke(worker.clj:322)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__5927.invoke(worker.clj:320)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> ... 6 common frames omitted
> 2014-07-10 18:44:43 b.s.util [INFO] Halting process: ("Async loop died!")
>
> Seems 0.8.1 it is.
>
>
> On 10 July 2014 18:21, Kemper, Stephan  wrote:
>
> We ran into this same problem this week.  The problem isn't with
> ZooKeeper, but the local state files in your ${storm.local.dir}.  If you
> delete the ./localstate directory there and restart the node, you should be
> OK again.
>
> More info on the problem was in last month's "v0.9.2-incubating and .ser
> files" thread from this mailing list.
>
>
> Stephan Kemper
> ViaSat
>
> *From: *Harsha 
> *Reply-To: *"user@storm.incubator.apache.org" <
> user@storm.incubator.apache.org>
> *Date: *Thursday, July 10, 2014 at 9:15 AM
> *To: *"user@storm.incubator.apache.org" 
> *Subject: *Re: b.s.m.n.Client [INFO] Reconnect
>
> Suparno,
>   Old storm data in zookeeper might conflict with newer versions
> of storm. I would suggest you to bring down the topologies and clean
> zookeeper /storm dir.
> -Harsha
>
>
>
> On Thu, Jul 10, 2014, at 09:06 AM, Suparno Datta wrote:
>
> okay that got worse. I just downloaded the 0.9.2. and failed to launch the
> supervisors (nimbus is running though). You don't have to don any clean up
> before you launch the new version right ?
>
> Anyways the stack trace of the error
>
> 014-07-10 18:01:27 b.s.event [ERROR] Error when processing event
> java.lang.RuntimeException: java.io.InvalidClassException

Re: Storm 0.9.2 Performance on EC2 Larges

[Andrew Montalenti]

What's limiting your throughput? Your e-mail doesn't have enough
information to make a diagnosis.

Whether 4k or 6k processed messages per second is "fast" depends on a lot
of factors -- average message size, parallelism, hardware, batching
approach, etc.

P. Taylor Goetz has a nice slide presentation discussing various factors to
think about when scaling Storm topologies for throughput:

http://www.slideshare.net/ptgoetz/scaling-storm-hadoop-summit-2014

One trick I tend to use to identify throughput bottlenecks is to lay out a
topology with mock bolts that do nothing but "pass tuples through",
configured identically from a partitioning / paralellism standpoint to my
actual topology. Then see how much throughput I get simply piping tuples
from the spout through that mock topology. This can often help you find
issues with things like performance bugs originating at the spout,
acking/emitting bugs, or other similar problems. It can also let you remove
some components from your topology to performance test them in isolation.

You can also review this recent JIRA ticket about improvements to the Netty
transport. Not only is this a lot of engineering effort going into Storm's
performance at scale, but benchmarks listed in there show throughput levels
of several hundred thousand messages per second, saturating cores and
network on topology machines.

https://issues.apache.org/jira/browse/STORM-297

Please don't roll your own stream processor -- the world doesn't need
another. :-D Something is likely wrong with the topology's layout and I'm
sure it's fixable.

HTH,

---
Andrew Montalenti
Co-Founder & CTO
http://parse.ly



On Fri, Jul 11, 2014 at 6:38 PM, Gary Malouf  wrote:

> Hi everyone,
>
> We've been banging our heads against the wall trying to get reasonable
> performance out of a small storm cluster.
>
> Setup after stripping down trying to debug:
>
> - All servers on EC2 m3.larges
> - 2 Kestrel 2.4.1 queue servers
> - 3 Storm Servers (1 running ui + nimbus, all running supervisors and thus
> workers)
> - 2 workers per instance, workers get 2GB of RAM max
> - 1 topology with 2 KestrelSpouts
>
> We measure performance by doing the following:
>
> - loading up the queues with a couple million items in each
> - deploying the topology
> - pulling up the storm ui and tracking the changes in ack counts over time
> on the spouts to compute average throughputs
>
>
> With acking enabled on our spouts we were getting around 4k messages/second
> With acking disabled on our spouts, we were seeing around 6k
> messages/second
>
>
> Adding a few bolts with acking quickly bring performance down below 800
> messages/second - pretty dreadful.  Based on the reports many other people
> have posted about their Storm clusters, I find these numbers really
> disappointing.  We've tried tuning the worker jvm options, number of
> workers/executors with this simple setup but could not squeeze anything
> more out.
>
> Does anyone have any further suggestions about where we should be looking?
>  We are about set to pull storm out of production and roll our own
> processor.
>
> Thanks,
>
>  Gary
>

Re: Netty transport errors.

[Andrew Montalenti]

Tomas,

You don't happen to be running Ubuntu 14.04 on Xen kernel, do you? Eg on
Amazon EC2.

I discovered an issue where running Storm across many workers on that OS
led to me hitting an annoying network driver bug that would cause timeouts
and topology freezes like you are seeing. Check dmesg for odd messages from
your network stack. Just a guess.

(copied from my reply to another similar thread)

-AM
On Jul 23, 2014 10:07 AM, "Tomas Mazukna"  wrote:

> I am really puzzled why processing stopped in the topology.
> Looks like the acking threads all stopped communicating. Only hint I saw
> was this netty exception:
> Any hints how to prevent this from happening again?
>
> 2014-07-23 08:56:03 b.s.m.n.Client [INFO] Closing Netty Client
> Netty-Client-ndhhadappp3.tsh.mis.mckesson.com/10.48.132.224:9703
>
> 2014-07-23 08:56:03 b.s.m.n.Client [INFO] Waiting for pending batchs to be
> sent with
> Netty-Client-ndhhadappp3.tsh.mis.mckesson.com/10.48.132.224:9703...,
> timeout: 60ms, pendings: 0
>
> 2014-07-23 08:56:03 b.s.m.n.Client [INFO] Closing Netty Client
> Netty-Client-ndhhadappp3.tsh.mis.mckesson.com/10.48.132.224:9700
>
> 2014-07-23 08:56:03 b.s.m.n.Client [INFO] Waiting for pending batchs to be
> sent with
> Netty-Client-ndhhadappp3.tsh.mis.mckesson.com/10.48.132.224:9700...,
> timeout: 60ms, pendings: 0
>
> 2014-07-23 08:56:03 b.s.util [ERROR] Async loop died!
>
> java.lang.RuntimeException: java.lang.RuntimeException: Client is being
> closed, and does not take requests any more
>
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>
> at
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>
> at
> backtype.storm.disruptor$consume_loop_STAR_$fn__758.invoke(disruptor.clj:94)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>
> at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
>
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_60]
>
> Caused by: java.lang.RuntimeException: Client is being closed, and does
> not take requests any more
>
> at backtype.storm.messaging.netty.Client.send(Client.java:194)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>
> at
> backtype.storm.utils.TransferDrainer.send(TransferDrainer.java:54)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>
> at
> backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__5927$fn__5928.invoke(worker.clj:322)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>
> at
> backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__5927.invoke(worker.clj:320)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>
> at
> backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>
> ... 6 common frames omitted
>
> 2014-07-23 08:56:03 b.s.util [INFO] Halting process: ("Async loop died!")
>
>
> Configuration:
>
> worker.childopts: "-Xmx2048m -Xss256k -XX:MaxPermSize=256m
> -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
> -XX:NewSize=128m -XX:CMSInitiatingOccupancyFraction=70
> -XX:-CMSConcurrentMTEnabled -Djava.net.preferIPv4Stack=true"
>
> supervisor.childopts: "-Xmx256m -Djava.net.preferIPv4Stack=true"
>
> nimbus.childopts: "-Xmx1024m -Djava.net.preferIPv4Stack=true"
>
> ui.childopts: "-Xmx768m -Djava.net.preferIPv4Stack=true"
>
> nimbus.thrift.threads: 256
>
>
> storm.messaging.transport: "backtype.storm.messaging.netty.Context"
>
> storm.messaging.netty.server_worker_threads: 1
>
> storm.messaging.netty.client_worker_threads: 1
>
> storm.messaging.netty.buffer_size: 5242880
>
> storm.messaging.netty.max_retries: 100
>
> storm.messaging.netty.max_wait_ms: 1000
>
> storm.messaging.netty.min_wait_ms: 100
>
>
> Thanks,
> --
> Tomas Mazukna
> 678-557-3834
>

Re: Decouple topology configuration from code

[Andrew Montalenti]

Storm bundles a comprehensive Clojure DSL for topology definition. In the
streamparse project, we extended this DSL to make Python bolts and spouts
easier to define, and the bundled streamparse CLI tool can submit Clojure
DSL topologies to remote or local clusters. Some docs on this are here.

http://streamparse.readthedocs.org/en/latest/quickstart.html#defining-topologies

We are using the Clojure DSL to wire up polyglot topologies, eg the
official JVM storm-kafka Spout and CPython / JVM mix of bolts in a single
topology. Very powerful. We plan to develop this idea further.
On Sep 16, 2014 4:29 AM,  wrote:

>  Hi folks,
>
>
>
> Apache Camel has a number of DSL which allow its topologies (routes wrt.
> Camel terminology) to be set up and configured easily.
>
> I am interested in such approach for Storm.
>
> I found java beans usage in:
>
> https://github.com/granthenke/storm-spring/
>
> but sounds fairly limited to me.
>
>
>
> Is there any other DSL like initiative for Storm ?
>
>
>
> My second concern is storm cluster management: we'd like to have a
> registry of topologies and be able to
> register/destroy/launch/suspend/kill/update registered topologies
> using a REST API.
>
>
>
> Is there any tool/initiative to support that ?
>
>
>
> Thx,
>
>
>
> /DV
>
>
>
> *Dominique Villard*
>
> *Architecte logiciel / Lead Developer*
> Orange/OF/DTSI/DSI/DFY/SDFY
>
> *tél. 04 97 46 30 03*
> dominique.vill...@orange.com 
>
>
>
>
>
> _
>
> Ce message et ses pieces jointes peuvent contenir des informations 
> confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu 
> ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages 
> electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou 
> falsifie. Merci.
>
> This message and its attachments may contain confidential or privileged 
> information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete 
> this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been 
> modified, changed or falsified.
> Thank you.
>
>