Thanks for your response Itay. I understood you. How would you modify the above code to make sure I am synchronizing/make calls from the same Thread ?
2014-12-16 16:43 GMT+02:00 Itai Frenkel <[email protected]>: > > Idan, > > > Consider you have 1000 concurrent tuples ... and the spout does not > throttle traffic. It means that the last bolt would be > handling 1000 concurrent requests. Now consider you have 100,000 concurrent > tuples.... Eventually the operating system or the NIO buffer would exhaust > its resources. You would have been better off with throtteling. > > > The output collector is the object that you perform "ack" or "fail" the > tuple. You probably call them from a future callback. Make sure that all of > these callbacks are called from the same thread, or are synchronized. > > > Itai > > > ------------------------------ > *From:* Idan Fridman <[email protected]> > *Sent:* Tuesday, December 16, 2014 3:58 PM > *To:* [email protected] > *Subject:* Re: Using AsyncHttpReuqest inside a Bolt > > > Hi, > > Any non-blocking bolt does not push back on the previous bolt if it is out > of resources. So you should consider using max-spout-pending for spout > level throttling. > > > @Itai, > My async bolt is the last bolt in the chain. so i guess I dont have this > problem?? > > Keep in mind you'll need to synchronize the OutputCollector when your > NIO response workers handle the returned requests as OutputCollector is not > thread safe. > > @Michael, > I am not sure how the OutputCollector is concerned to my issue? > > This is my execution code.. that code could caouse me any side-effects > in my topology? > > > @Override > public void execute(Tuple tuple, BasicOutputCollector basicOutputCollector) { > > PushMessage pushMessage = (PushMessage) > tuple.getValueByField("pushMessage"); > final String messageId = pushMessage.getMessageId(); > asyncHttpClient.preparePost("some_url").execute(new > AsyncCompletionHandler<Response>() { > @Override > public Response onCompleted(Response response) throws Exception { > String innerMessageId = messageId; > System.out.printf("\n messageId=" + innerMessageId + > "responseBody=" + response.getResponseBody()); > return response; > } > > @Override > public void onThrowable(Throwable t) { > t.printStackTrace(); > } > }); > } > > thanks. > > > 2014-12-15 19:30 GMT+02:00 Michael Rose <[email protected]>: >> >> Keep in mind you'll need to synchronize the OutputCollector when your NIO >> response workers handle the returned requests as OutputCollector is not >> thread safe. > > > Michael Rose (@Xorlev <https://twitter.com/xorlev>) > Senior Platform Engineer, FullContact <http://www.fullcontact.com/> > [email protected] > > On Mon, Dec 15, 2014 at 9:20 AM, Itai Frenkel <[email protected]> wrote: >> >> Hi, >> >> >> Any non-blocking bolt does not push back on the previous bolt if it is >> out of resources. So you should consider using max-spout-pending for spout >> level throttling. >> >> >> Regards, >> >> Itai >> ------------------------------ >> *From:* Idan Fridman <[email protected]> >> *Sent:* Monday, December 15, 2014 10:19 AM >> *To:* [email protected] >> *Subject:* Using AsyncHttpReuqest inside a Bolt >> >> Hi All, >> My bolt need to dispatch async request to remote service. >> >> I am using AsyncHttpReuest library( >> https://github.com/AsyncHttpClient/async-http-client) which based on NIO >> channels to get the response asynchronously while not allocating Thread for >> each request. >> >> I was wondering if any side-effects could cause this implementation >> within Storm Bolt ? >> >> thank you. >> >
