Backpressure with fieldsgrouping

Hi, We have a topology which consumes data generated by multiple applications via Kafka. The data for one application is aggregated in a single bolt task using fieldsgrouping. All applications push data at different rates so some executors of the bolt are busier/overloaded than others and capacity

Storm logger file

Hi, On Storm UI, we have an option to enable logger for specific classes using the following section - [image: Screen Shot 2019-04-05 at 5.30.21 PM.png] This is great but every time we redeploy our topology, we have to redefine all our loggers in the UI again which is quite a bit of work if ther

Re: Storm-Hive : lock acquiring problem

there should be a way for users to explicitly specify which hive release they want to use storm-hive with. The documentation for storm-hive should also be updated to reflect this requirement. Happy to provide prs if that sounds like a good idea. Thanks. On Fri, Jun 8, 2018 at 3:21 PM, Abhishek Raj

Re: Storm-Hive : lock acquiring problem

Hi. We faced a similar problem earlier when trying HiveBolt in storm with hive on emr. We were seeing java.lang.IllegalStateException: Unexpected DataOperationType: UNSET > agentInfo=Unknown txnid:130551 in hive logs. Any help here would be appreciated. On Fri, Jun 8, 2018 at 10:26 AM, Milind

Acking unanchored tuples

Hi. I had a pretty simple question. What happens if we call OutputCollector's ack method on an unanchored tuple? Is it a no op? Can anybody please point me to the source? Thanks, -- Abhishek

Re: Re: open storm ui very slow(Internet mail)

We upgraded from storm 0.9.4 to 1.0.2 and started seeing higher ui load time as well. The only thing we increased during the upgrade was the zookeeper cluster size so it could be a possible cause. Thanks, On Jun 23, 2017 12:05 AM, "Erik Weathers" wrote: > [+storm user list again, please remembe

Re: Stateful topology hangs

is > *literally* stuck? If you have monitoring of the topic's consumption on > kafka then it should be obvious. If you do not then you could use > Wireshark's cmdline tool called tshark to sniff kafka requests and see if > the same offset is being requested. > > - E

Re: Stateful topology hangs

storm-kafka-client)? > > -Taylor > > On Feb 22, 2017, at 4:32 PM, Abhishek Raj wrote: > > Hello, I am using storm's state management feature in a topology. > The topology has a kafkaspout and a StatefulBolt which uses > RedisKeyValueState. What I observe is that after s

Stateful topology hangs

Hello, I am using storm's state management feature in a topology. The topology has a kafkaspout and a StatefulBolt which uses RedisKeyValueState. What I observe is that after some time of running smoothly, the spout just stops consuming from the kafka topic and the $checkpointspout stops emitting a

Stateful bolts

Hi. I was looking into state management which looks like a really cool feature but I am wondering if it'd be right for my use case. One of the bolts does bulk updates to database and stores the update information temporarily in hashm

Re: Storm 1.0.2 - Nimbus reporting Gzip failure in spite of resetting Storm and Zookeeper state

Try changing "storm.zookeeper.root" in your storm.yaml and restarting storm daemons. Thanks, Abhishek On Oct 28, 2016 1:06 PM, "Øystein Larsen" wrote: > Hi! I'm having trouble running Nimbus following a systems upgrade of my > Ubuntu 14.04 server yesterday. I've tried using Java 8 from Oracle w

Re: Error when running pacemaker.

uilt jar > - run pacemaker and nimbus with JDK 7 > - see it reappears > > If it works perfectly, I guess that's the cause. If then please file an > issue so that we can track it. > > Please let me know you'd like me to test this. > > Thanks, > Jungtaek Lim (

Error when running pacemaker.

Hello, I am using storm 1.0.2. When I start the pacemaker daemon and then start nimbus, I see the following error in pacemaker.log repetitively and can't submit topologies. WARNING: An exception was thrown by a user handler while handling an >> exception event ([id: 0xa4126833, /127.0.0.1:43693 =>

Re: Some bolts stop processing after a while.

Hi Abhishek, > Did you check whether spout is really emitting messages? > > On Thu, Aug 4, 2016 at 5:42 PM, Abhishek Raj > wrote: > >> Thanks for the quick response. According to storm documentation, if a >> worker/node dies it's automatically restarted. Also, the bolts

Re: Some bolts stop processing after a while.

nity.hortonworks.com/articles/36151/debugging-an-apache-storm-topology.html > > On a side note, having 8 bolts seems like a rather complicated situation. > This is if it is Spout ---> Bolt1 ---> Bolt2 ---> Bolt3 >and so on ---> > Bolt8. Takes too long for an ack. Design ch

Some bolts stop processing after a while.

Hi. We are using storm 0.9.4. Our topology consists of a linear chain of 1 spout and 8 bolts. In the 4th bolt we call an external bolt written in php which emits to 5th bolt after some processing. We are seeing that after some time, the 6th, 7th and 8th bolt completely stop processing. The execute

Use case of localOrShuffleGrouping?

I understand the difference between "shuffle grouping" and "local or shuffle grouping". But I haven't yet come across any instances where I would prefer using localOrShuffleGrouping instead of shuffleGrouping. The way I see it, shuffleGrouping helps increasing the concurrency if the tasks of a g

RE: No logs directory in apache storm on Mavericks

y login as root user and do a find of nimbus and supervisor logs files. > find / -name nimbus*log* > find / -name supervisor*log* > > On 5/30/15, Abhishek Raj wrote: > > @skgadalay by "./logs" I meant logs directory in apache storm. > > @Jeffery Maass, I tried t

RE: No logs directory in apache storm on Mavericks

ckoverflow.com/users/373418/maassql > > + > > > > > > On Fri, May 29, 2015 at 12:42 AM, Abhishek Raj > > wrote: > > > >> With nimbus the values are > >> > >> > >> -Dstorm.log.dir=/usr/local/apache-storm-0.9.4/lo

RE: Trident and Kafka - Examples

Hi Ashish. You might find this helpful. https://github.com/apache/storm/tree/master/external/storm-kafka#trident-spout From: ashish.us...@gmail.com Date: Thu, 28 May 2015 17:25:46 -0400 Subject: Trident and Kafka - Examples To: user@storm.apache.org Hi , Are there any examples of Trident and Kaf

RE: No logs directory in apache storm on Mavericks

w.com/users/373418/maassql + On Thu, May 28, 2015 at 10:17 AM, Abhishek Raj wrote: Hi Jeff. Thanks for the response. I've checked ${storm.log.dir} and it seems to be set correctly. I verified the logs directory in the startup script (bin/storm) as well. It is set t

RE: No logs directory in apache storm on Mavericks

ctory ( wherever you uncompressed Storm ), look at ./logback/cluster.xml You will see that logs write to a variable, ${storm.log.dir} Thank you for your time! + Jeff Maass linkedin.com/in/jeffmaass stackoverflow.com/users/373418/maassql + On Thu, May 28, 2015

No logs directory in apache storm on Mavericks

I am using apache-storm 0.9.4 on OS X Mavericks. Here's my storm.yamlstorm.zookeeper.servers: - "localhost" storm.zookeeper.port: 2181 storm.local.dir: "/var/lib/storm" storm.local.hostname: "localhost" nimbus.host: "localhost" nimbus.childopts: "-Xmx512m" supervisor.childopts: "-Xmx256m" s