FailedException

Re: Kafka high level consumer in storm

Is there any downside to using Kafka high level consumer as spout? The main downside of the high level consumer is that you won't be able to control exactly when it will request a broker for more data and that it will always commit the latest offset you read from the stream it provides. In a

Streams don't get fair distribution of work

Hello, Problem: I have a couple of busy streams (a few of billion tuples per day) that are getting an unfair distribution of work. I'm using the Trident api. storm 0.8.0. One of the streams (spouts) doesn't get to run. Details: I've been using storm for almost 2 years and this topology has

Re: Can you please suggest how FailedException is working in Storm(Trident)?

It is not a good idea to store anything on zookeeper other than the metadata. You can use a centralized data-store for storing your logs. Now for the spout thing , why do you want to remove the files using a spout you can do that using another bolt say removebolt which is meant for removing the

One solution to the stdio redirect issue

I put together a more complete solution to the insidious STDOUT/STDERR buffer filling issue. Basically, if STDOUT/STDERR is not redirected/consumed in cluster mode it will fill the buffer and eventually take down your topology. The original thread on this issue was not migrated to JIRA but

RESEND: Latency metrics questions

Hi all - anyone have any insight into my question below? It would be so hugely helpful to have a doc on what metrics are available in storm, what data structures each contains and what each element means. I've amassed some idea of what thing mean, but I suspect I'm like others and still

Re: serialization exception when using multiple worker/supervisors

Hello, I tried to configure storm to indicate that it should use the kryo FieldsSerializer with the following configuration in the strom.yml file: topology.kryo.register: - com.rabbitmq.client.impl.LongStringHelper$ByteArrayLongString This is as described here:

Understanding Message timeout

Hi With respect to the TOPOLOGY_MESSAGE_TIMEOUT_SECShttp://nathanmarz.github.io/storm/doc/backtype/storm/Config.html#TOPOLOGY_MESSAGE_TIMEOUT_SECS*, *is this the time that the spout waits for the ack or the fail callback before timing out? Or is this the time that the acker does not get any ack

Re: Kafka high level consumer in storm

Thanks Mattijs. I understand from your mail that there is no issue using HL consumer except for the loss of flexibility. Regards, Kashyap On Mon, Mar 24, 2014 at 3:20 AM, Mattijs Ugen matt...@holmes.nl wrote: Is there any downside to using Kafka high level consumer as spout? The main

Error running code in storm-starter using Maven

Hi, I'm a new user to storm and I've gotten some errors running the storm-starter repository ( https://github.com/apache/incubator-storm/tree/master/examples/storm-starter ) using Maven. I'm running Mac OS X 10.9.2, Maven 3.2.1 and JDK 1.6.0_65 . Executing the following command (which is found