Re: Sliding window on numerical data?

2014-11-11 Thread Klausen Schaefersinho
Hi, IBM InfoSphere Streams has some kind of visual query editor. However I doubt that you can model a heart beat in it. If you need to start quick just write your own bolt and use some kind of ring-buffer to store the last n events. Then writer a function that converts it to a vector. Please

Re: Who is the contact point for the Storm Topology Visualization ?

2014-11-11 Thread Adrian Portabales
For now we don’t implemented trident topologies but in the next iteration is a strong possibility. Regards Adrian P. El 10/11/2014, a las 17:02, Jean-Sebastien Vachon jean-sebastien.vac...@wantedanalytics.com escribió: ​Looks promising... will this support Trident topologies as well?

Ensure tuples are processed in order while still avoiding bottlenecks

2014-11-11 Thread Bryan Hernandez
Hi, I'd like to know if there is a way to do the following in Storm: The topology: Spout1 - Bolt1 - Bolt2 *Spout1*: emits *about* 1 tuple per second. *Bolt1*: execute() method takes, *on average*, 5 seconds to process each tuple. *Bolt2*: must receive tuples in the same order that they were

Re: Ensure tuples are processed in order while still avoiding bottlenecks

2014-11-11 Thread Bryan Hernandez
I should clarify that I am aware I could make Bolt2 also subscribe to Spout1, so that it knows the correct order. However, I am wondering if there is a built-in Storm way of handling this requirement in general. Thanks! Best, Bryan On Tue, Nov 11, 2014 at 5:03 PM, Bryan Hernandez

acking message for GMP while reading from kafka queue

2014-11-11 Thread Jyotirmoy Sundi
Hi Folks, I am trying to handle Kafka messages in a storm topology and make sure messages are processed at least once while maintaining a queue in spouts and evict them when an ack is recd. Any ideas on which of the above would be better ?? Approach 1: 1. Use the low-level API and manage

Evolving models that scale without the use of a dynamic topology

2014-11-11 Thread Bryan Hernandez
Hi, I would like to implement the following data processing scheme using Storm, but it seems to me that the only natural way to do this is with a dynamic topology, which I know Storm doesn't support. Is there a way to do this with a static topology, or is this use case truly outside of Storm's

Re: Sliding window on numerical data?

2014-11-11 Thread Manoj Jaiswal
Johannes, Esper has several concepts like rolling and sliding windows. Whatever you are trying to achieve through storm by using transactions etc, can be achieved easily in Esper. In our bolt, we initialize the queries on receiving one type of message, then the real time data is used in sliding

Re: Evolving models that scale without the use of a dynamic topology

2014-11-11 Thread Manoj Jaiswal
Bryan, It looks like you are trying to build a self evolving model structure where you have n+m models and over time based on performance you want to select only n models and discard remaining m. This looks difficult , but I see a possibility if you use Esper. Based on same logic the model can

Re: Which version of Storm to use 0.9.1 or 0.9.2

2014-11-11 Thread Manoj Jaiswal
yes there is a bug and you should use version 0.9.3 https://issues.apache.org/jira/browse/STORM-406 On Tue, Nov 11, 2014 at 2:25 PM, Jovaughn Lockridge jova...@gmail.com wrote: Hey guys, My team has been testing Apache Storm 0.9.1 for quite some time. They are claiming there is a known bug