Re: GC overhead limit exceeded

2016-04-07 Thread sam mohel
sorry it's supposed to use hashmap.remove() to not to make it reach to heapsize right ! On Fri, Apr 8, 2016 at 1:43 AM, sam mohel wrote: > Really thanks for your patience , what i got about hashmap you mentioned > that it's supposed not to use hashmap.remove(); > Right ? >

Re: GC overhead limit exceeded

2016-04-07 Thread sam mohel
Really thanks for your patience , what i got about hashmap you mentioned that it's supposed not to use hashmap.remove(); Right ? On Thu, Apr 7, 2016 at 10:45 AM, Spico Florin wrote: > Hi! > By release the hashmap, I mean that you need to remove the keys at some >

Re: storm ui, execute latency of multi-lang

2016-04-07 Thread Xiang Wang
Dear Julien, Thanks for your reply. Your suggestion really makes sense. Thanks! Best, Xiang --- Xiang Wang PhD Candidate Database Research Group School of Computer Science and Engineering The University of New South Wales Sydney, Australia On Thu, Apr 7, 2016 at

Re: storm ui, execute latency of multi-lang

2016-04-07 Thread Xiang Wang
Dear Matthias, Thank you for your reply. I find a much easier way to do this: http://storm.apache.org/releases/0.10.0/STORM-UI-REST-API.html. Best, Xiang --- Xiang Wang PhD Candidate Database Research Group School of Computer Science and Engineering The University

Re: storm ui, execute latency of multi-lang

2016-04-07 Thread Julien Nioche
Hi Xiang For your question #2, assuming ShellBolt is used, the execute method just creates the message to send and stores it before exiting, hence the low latency shown on the UI - see https://github.com/apache/storm/blob/master/storm-core/src/jvm/org/apache/storm/task/ShellBolt.java#L148 You

Re: storm ui, execute latency of multi-lang

2016-04-07 Thread Matthias J. Sax
For (1), you can use > Client c = NimbusClient.getConfiguredClient(..) > TopologyInfo info = c.getTopologyInfo(...) > List stats = info.get_executors(); > ExecutorStats exStats = stats.get(...).get_stats(); -Matthias On 04/07/2016 10:03 AM, Xiang Wang wrote: > Hi All, > > I got several

Re: Need for capacity planning suggestions for setting up KAFKA - STORM cluster in AWS

2016-04-07 Thread Spico Florin
Hi! Well is not only about memory. Is about also about availability, failover, if your processing is CPU intensive and also the velocity and the volume of data that you ingest. Florin On Mon, Apr 4, 2016 at 7:26 AM, researcher cs wrote: > I have same question . what

storm ui, execute latency of multi-lang

2016-04-07 Thread Xiang Wang
Hi All, I got several questions regarding storm ui and multi-lang processing. 1. Is there any APIs to get the storm statistics, e.g., execute latency, capacity, as shown in the storm UI? I would like to get those statistics automatically using a program and save them to a file such that I don't

RE: Storm Topology Hanging

2016-04-07 Thread Alexander T
Hi Nitin, You might get problems with max spout pending if you delay acking of one message until another arrives. In this case having a low max spout pending can cause the buffers to fill before the awaited message arrives. But this type of deadlock should cause a message timeout. If you have

Re: Storm Topology Hanging

2016-04-07 Thread jinhong lu
Try to check the log of kafka at the moment topology began to hung, and the load of kafka cluster at that moment. I met a situation is that the kafka is too busy to response, it cause storm topology hung, especially when you use transactional topology. lujinhong > 在 2016年4月7日,13:58,Nitin