Oh I know what is all synchronized about. Just I wasn't sure if the synchronization in my storm bolt is concerned only to the execute method.
2014-12-16 18:08 GMT+02:00 Nathan Leung <[email protected]>: > > you should look up the synchronized keyword in java: > > http://docs.oracle.com/javase/tutorial/essential/concurrency/syncmeth.html > http://docs.oracle.com/javase/tutorial/essential/concurrency/locksync.html > > On Tue, Dec 16, 2014 at 9:50 AM, Idan Fridman <[email protected]> wrote: > >> Thanks for your response Itay. I understood you. >> How would you modify the above code to make sure I am synchronizing/make >> calls from the same Thread ? >> >> 2014-12-16 16:43 GMT+02:00 Itai Frenkel <[email protected]>: >>> >>> Idan, >>> >>> >>> Consider you have 1000 concurrent tuples ... and the spout does not >>> throttle traffic. It means that the last bolt would be >>> handling 1000 concurrent requests. Now consider you have 100,000 concurrent >>> tuples.... Eventually the operating system or the NIO buffer would exhaust >>> its resources. You would have been better off with throtteling. >>> >>> >>> The output collector is the object that you perform "ack" or "fail" >>> the tuple. You probably call them from a future callback. Make sure that >>> all of these callbacks are called from the same thread, or are synchronized. >>> >>> >>> Itai >>> >>> >>> ------------------------------ >>> *From:* Idan Fridman <[email protected]> >>> *Sent:* Tuesday, December 16, 2014 3:58 PM >>> *To:* [email protected] >>> *Subject:* Re: Using AsyncHttpReuqest inside a Bolt >>> >>> >>> Hi, >>> >>> Any non-blocking bolt does not push back on the previous bolt if it is >>> out of resources. So you should consider using max-spout-pending for spout >>> level throttling. >>> >>> >>> @Itai, >>> My async bolt is the last bolt in the chain. so i guess I dont have this >>> problem?? >>> >>> Keep in mind you'll need to synchronize the OutputCollector when your >>> NIO response workers handle the returned requests as OutputCollector is not >>> thread safe. >>> >>> @Michael, >>> I am not sure how the OutputCollector is concerned to my issue? >>> >>> This is my execution code.. that code could caouse me any side-effects >>> in my topology? >>> >>> >>> @Override >>> public void execute(Tuple tuple, BasicOutputCollector basicOutputCollector) >>> { >>> >>> PushMessage pushMessage = (PushMessage) >>> tuple.getValueByField("pushMessage"); >>> final String messageId = pushMessage.getMessageId(); >>> asyncHttpClient.preparePost("some_url").execute(new >>> AsyncCompletionHandler<Response>() { >>> @Override >>> public Response onCompleted(Response response) throws Exception { >>> String innerMessageId = messageId; >>> System.out.printf("\n messageId=" + innerMessageId + >>> "responseBody=" + response.getResponseBody()); >>> return response; >>> } >>> >>> @Override >>> public void onThrowable(Throwable t) { >>> t.printStackTrace(); >>> } >>> }); >>> } >>> >>> thanks. >>> >>> >>> 2014-12-15 19:30 GMT+02:00 Michael Rose <[email protected]>: >>>> >>>> Keep in mind you'll need to synchronize the OutputCollector when your >>>> NIO response workers handle the returned requests as OutputCollector is not >>>> thread safe. >>> >>> >>> Michael Rose (@Xorlev <https://twitter.com/xorlev>) >>> Senior Platform Engineer, FullContact <http://www.fullcontact.com/> >>> [email protected] >>> >>> On Mon, Dec 15, 2014 at 9:20 AM, Itai Frenkel <[email protected]> wrote: >>>> >>>> Hi, >>>> >>>> >>>> Any non-blocking bolt does not push back on the previous bolt if it >>>> is out of resources. So you should consider using max-spout-pending for >>>> spout level throttling. >>>> >>>> >>>> Regards, >>>> >>>> Itai >>>> ------------------------------ >>>> *From:* Idan Fridman <[email protected]> >>>> *Sent:* Monday, December 15, 2014 10:19 AM >>>> *To:* [email protected] >>>> *Subject:* Using AsyncHttpReuqest inside a Bolt >>>> >>>> Hi All, >>>> My bolt need to dispatch async request to remote service. >>>> >>>> I am using AsyncHttpReuest library( >>>> https://github.com/AsyncHttpClient/async-http-client) which based on >>>> NIO channels to get the response asynchronously while not allocating Thread >>>> for each request. >>>> >>>> I was wondering if any side-effects could cause this implementation >>>> within Storm Bolt ? >>>> >>>> thank you. >>>> >>> >
