Workers maxing out heap, excessive GC, mk_acker_bolt->pending map is huge (0.9.5)

2015-08-20 Thread Jason Chen
Hi all. Here’s what I’m seeing. I’ve got a fairly simple topology, consisting of 3 bolts. Kafka spout, simple processing bolt (JSON parse to POJO, a bit of processing, and back to JSON), and Kafka Bolt (output). 12 workers, Xmx1G. It runs happily for a little over a day, then basically slow

Re: Expiring session 0x14f4657e8dd001b, timeout of 20000ms exceeded

2015-08-20 Thread Nathan Leung
Prepare shouldn't cause a timeout. When I said gc I meant something like stop the world garbage collection. On Thu, Aug 20, 2015 at 6:52 AM, swapnil joshi wrote: > Hi Nathan, > Yes, One of my bolt takes lot of time to prepare because it contains some > data preparation Operation. So what are so

Storm UI doubts

2015-08-20 Thread Nithesh Kb
Hi Storm experts, please help me with understanding below mentioned doubt. I have created a sample twitter processing program in Storm and it is been deployed in a two node cluster. But the question is i don't understand why the blot executed is more than the spout emitted number. [image: Inline

Re: Expiring session 0x14f4657e8dd001b, timeout of 20000ms exceeded

2015-08-20 Thread swapnil joshi
Hi Nathan, Yes, One of my bolt takes lot of time to prepare because it contains some data preparation Operation. So what are solutions for such problem. On Thu, Aug 20, 2015 at 4:19 PM, Nathan Leung wrote: > Do you have long running gc? I've seen this cause zk connection loss. > On Aug 20, 2015

Re: Expiring session 0x14f4657e8dd001b, timeout of 20000ms exceeded

2015-08-20 Thread Nathan Leung
Do you have long running gc? I've seen this cause zk connection loss. On Aug 20, 2015 2:30 AM, "swapnil joshi" wrote: > Thanks!!! For giving me response. I had change configuration in storm.yml > file. But still I got following error > > *15/08/20 11:44:17 ERROR imps.CuratorFrameworkImpl: Backgro