Hi. We want to make a Storm topology than handles DRPC requests, and we want the requests to be handled concurrently. I.e., if a DRPC request comes into the topology at time T0 that takes a long time to complete (let's say 100 seconds), we don't want this request to block another request that comes in at T0 + 1 sec and will only take 1 second to complete. I.e, we'd like to get the result for second request at time T0 + 2 sec or T0 + 3 sec or so, and not at time T0 + 101 sec.
Does this behavior happen automatically? Or would we have to do something special to make Storm behave this way? Or is it just not the way that Storm works? I haven't found anything about this in the online Storm documentation or in any books. I looked through the user@storm.apache.org archives looking for an answer to this question before posting. I found this question posted a few months ago: >From "JianJian, Bu" <jian...@amazon.com> Subject Issues about concurrent request to Trident DRPC Date Mon, 29 Dec 2014 07:08:10 GMT Dear All, We’re encountering some issue when use trident DRPC -- It seems that trident DRPC request is executed sequentially with later request blocked by previous one no matter how the parallelism parameter would be set. Our usage is : we setup a client faced web services which is hosted by a group of hosts. The web service receive requests from our client and call storm DRPC to query the data and send back the result to the Client. Now the issue is some of the request will take much time to be finished by DRPC. So these requests will block other clients from getting the result. I can see many introductions in storm wiki about how drpc works but there is little threads about how drpc handles multiple requests. There is only one sentence about the concurrent reuqests in this wiki https://storm.apache.org/documentation/Distributed-RPC.html “KeyedFairBolt for weaving the processing of multiple requests at the same time” . But it has little help. There may be some ideas like start multiple similar topology in the same cluster but I think that would have other issue like balance the load. Does anyone ever have the similar issue and how did that get solved? Unfortunately, there are no responses in the archive to this previous posting. Is Storm DRPC not widely used? |>oug