Is Kafka Spout still in use?

Hi All, Is Kafka Spout still in use?. I mean given that it doesn't support the latest version of Kafka (0.8), is it still under development?. Will it support Kafka 0.8 in the near future?. If Kafka Spout is still in use, which is the latest stable version (I'm not able to find it

Re: Is Kafka Spout still in use?

https://github.com/wurstmeister/storm-kafka-0.8-plus On Mon, Mar 17, 2014 at 2:04 PM, Manthosh Kumar T manth...@gmail.comwrote: Hi All, Is Kafka Spout still in use?. I mean given that it doesn't support the latest version of Kafka (0.8), is it still under development?. Will

RE: Is Kafka Spout still in use?

+1 for this question. Looking for the same. -- Thanks Regards, Gufran Pathan | +91-9566811502 | www.mu-sigma.comhttp://www.mu-sigma.com/ | From: Manthosh Kumar T [mailto:manth...@gmail.com] Sent: Monday, March 17, 2014 11:35 AM To: user@storm.incubator.apache.org Subject: Is Kafka Spout still

Re: Is Kafka Spout still in use?

Yes, it is popular. There is discussion about introducing this into storm source tree as a contrib module. http://qnalist.com/questions/4738568/discuss-pulling-contrib-modules-into-apache On Mon, Mar 17, 2014 at 2:41 PM, Manthosh Kumar T manth...@gmail.comwrote: Hi Zhong, I

Storm - field grouping question for Class instance tuples

Hi , I have a Bolt1 that emits a tuple of type a.b.c.MyClass1. MyClass1 has a String field - class1Key1. I need to segregate all tuples having same value for a.b.c.MyClass1:class1Key1 into a specific task in another Bolt2. How do I go about defining such a topology ? I have seen egs where tuples

Re: Storm - field grouping question for Class instance tuples

Code sample : Topology : === TopologyBuilder builder = new TopologyBuilder(); builder.setSpout(epd-spout,new MyJmsSpout()); builder.setBolt(epd-CPY001-bolt1, new Bolt1(CPY001), parallelismHintMap.get(CPY001)) .shuffleGrouping(epd-spout);

Profiling

Hi, I am testing a typology that is frequently running out of memory. Has any body already made some experiences how to best profile storm? Kind Regards, Klaus

Re: Profiling

have you tried yourkit profiler? On Mon, Mar 17, 2014 at 1:17 AM, Klausen Schaefersinho klaus.schaef...@gmail.com wrote: Hi, I am testing a typology that is frequently running out of memory. Has any body already made some experiences how to best profile storm? Kind Regards, Klaus

JDBC Connection in prepare()

Hi All, I'm trying add data to DB using JDBC in bolt. When I create Connection in the execute() method and close it at the end of execute() method, it works fine. Since it was slower, I created the Connection in prepare() method. But this time I'm getting SQLException saying No Suitable

Re: JDBC Connection in prepare()

Hi Tran, I had deployed the JDBC Drivers properly. As I had said before, it works fine if the connection is created in the execute() method. So doesn't that mean Drivers are properly deployed?. Please correct me if I'm wrong. On 17 March 2014 15:25, An Tran tra...@gmail.com wrote:

Re: JDBC Connection in prepare()

I see sometimes something like classloaders 'Race Condition' - when I can't get in one moment DB due to No suitable driver, and in next Connection create I got it just fine. Maybe you should call Connection.create (which can fail), and add some init-delay for a few secs, and then second one could

Re: JDBC Connection in prepare()

That sounds weird. Are you sure that the expected database operation is being performed when the connection is established in execute()? Please add logs and see whether the expected results are generated. Program should not fail in both cases ideally. Only point is that the driver should be

Improve upsert performance - JDBC

Hi All, Currently when I upsert values via JDBC, each upsertion takes 14-20 ms on an average. Auto commit is set to true. Is there a way to improve this without batch insert?. -- Cheers, Manthosh Kumar. T

Re: Advice

Hi, Thanks for the reply. I played with the topology and I seem to have 'fixed' it. Still, any clarification you can make on this topic would be very useful to me. It would be good to have a definitive way to calculate how many workers/Executors/Tasks are most efficient given a set of

Wirbelsturm released, 1-click deployments of Storm clusters

Hi everyone, I have released a tool called Wirbelsturm (https://github.com/miguno/wirbelsturm) that allows you to perform local and remote deployments of Storm. It's also a small way of saying a big thank you to the Storm community. Wirbelsturm uses Vagrant for creating and managing machines,

setting message timeout on a per spout basis

any way to do that? i've been looking and can't find. i assume that's because you can't. just wanted to verify. thanks, sean

Best way to get JMX access to storm metrics for spout tuples emitted / acked | bolt tuples emitted, latency etc.

Hi, I'm interested in getting metrics via JMX on not onlycontainer level factors (such as # of garbage collects, heap usage,etc.), but also metrics that describe how spouts and bolts are performing (e.g., # of tuples emitted, # transferred -- the same kind of stuff that the storm UI

Re: Best way to get JMX access to storm metrics for spout tuples emitted / acked | bolt tuples emitted, latency etc.

Depending on what you need, you could use the Thrift interface to Nimbus to grab the statistics. The UI project does do some calculations on the raw metrics, so it's not quite the same. The algorithms it uses aren't too difficult to replicate. We ended up building something very similar to what

Re: Best way to get JMX access to storm metrics for spout tuples emitted / acked | bolt tuples emitted, latency etc.

Thanks, Noel and Michael ! On Mon, Mar 17, 2014 at 5:55 PM, Noel Milton Vega nmv...@computingarchitects.com wrote: I used Nimbus Thrift on my end. The one thing to note about that approach is that, if you're planning to plot those metrics (say, on Graphite), the results aren't very

when running a negative test that forces exception in my spout i find that the JVM abruptly exits. Any way to intercept this ?

Hi there.. I tried writing a negative testng test which starts my topology in local mode, then submits it. I provide parameters that deliberately cause my code called by my spout to choke and throw a RuntimeException. This causes my test runner (maven's surefire) to complain that The forked

Storm optimization and single -point problem

Hello!Everybody I have a question to ask , how do storm cluster optimization and how to solve the problem nimbus single point? I very much hope to get your reply!Thanks!