Re: Storm Memory Consumption Issue

2015-09-12 Thread Hong Wind
Thanks for reply Nathan. But our case is that the rule evaluation depends on field grouping and each rule might have different grouping condition. /wind 2015年9月12日 下午11:01,"Nathan Leung" 写道: > I would try making lots of bolt 2 in parallel and have each bolt 2 process > the full list of rules. >

Re: Storm Memory Consumption Issue

2015-09-12 Thread Nathan Leung
I would try making lots of bolt 2 in parallel and have each bolt 2 process the full list of rules. On Sep 12, 2015 10:37 AM, "Hong Wind" wrote: > Hi all again, > > Since no reply I guess the question might be not described clearly enough. > So let me try to elaborate it ... > > Performance requir

Re: Storm Memory Consumption Issue

2015-09-12 Thread Hong Wind
Hi all again, Since no reply I guess the question might be not described clearly enough. So let me try to elaborate it ... Performance requirements for our system is 100K TPS with latency must be < 500 ms. And the system needs to evaluate some rules for each incoming message but we don't know how

Storm Memory Consumption Issue

2015-09-09 Thread Hong Wind
Hi all, I am doing a performance benchmark for our topology which is to evaluate a set of rules. The topology is as below: Spout -> Bolt#1(Dispatch) -> Bolt#2(Evaluation) The bolt#1 load the rules from DB and dispatch them to bolt#2 for evaluation. One bolt#2 task evaluates one rule. So how many