Re: messages traveling slowly through storm

I did some more digging around and found that a message gets transmitted from the kafka spout, reaches the bolt but never goes in to the bolt's execute method. Eventually after a minute kafka spout fails the message. I am using a custom FailureHandler which does not allow storm to replay the messag

Re: messages traveling slowly through storm

Does it matter that I am ack-ing the tick tuples in the execute method of the bolt? Will that give rise to double ack-ing of the tuples and create some inconsistent state? Thanks. On Wed, Oct 7, 2015 at 10:53 PM, Rohit Kelkar wrote: > Thanks for responding. > Kafka Spout parallelism = 1, number

Re: messages traveling slowly through storm

Thanks for responding. Kafka Spout parallelism = 1, number of partitions of kafka topic=1, Tried various values for topology.max.spout.pending=64,100, 1024 Thanks. On Wed, Oct 7, 2015 at 10:06 PM, Harsha wrote: > whats your Kafka Spout parallelism and how many partitions you've in your > kafka

Re: messages traveling slowly through storm

whats your Kafka Spout parallelism and how many partitions you've in your kafka topic. Also did you try to tune topology.max.spout.pending -Harsha On Wed, Oct 7, 2015, at 10:54 AM, Rohit Kelkar wrote: > I have a kafka spout and single bolt topology running on a cluster in > debug mode. I see that

Re: Storm and enviroment variables

I think you can specify a config file that is other than the default. Can you try —config command line option while running storm command ? From: "Garcia-Contractor, Joseph (CORP)" Reply-To: "user@storm.apache.org" Date: Wednesday, October 7, 2015 at 2:15 PM To: "u

RE: Storm and enviroment variables

Ok so an additional question here… Can I tell storm on startup to use a storm.yaml file that I specify with a full path, or will it only look in /etc/storm/conf/storm.yaml? Joe From: Priyank Shah [mailto:ps...@hortonworks.com] Sent: Wednesday, September 30, 2015 2:31 PM To: user@storm.apache.or

messages traveling slowly through storm

I have a kafka spout and single bolt topology running on a cluster in debug mode. I see that the message is sent from kafka spout at 17:02:48 and received at the bolt at 17:02:56 Here are the relevant logs - http://pastebin.com/WBxmPjLk I checked the network traffic. No signs of saturation. Am I fi