[ https://issues.apache.org/jira/browse/HADOOP-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204595#comment-15204595 ]
Sanjay Radia commented on HADOOP-12909: --------------------------------------- I haven't had a chance to look at the patch or review all the comments, but wanted to bring attention to one issue wrt async rpc that is well known by implementors and practitioners of message passing & rpc systems (excuse me if this has already been covered): * One needs to watch out for buffer management. ie. aync rpc/message passing has the potential to use up memory for buffering the messages. This is prevented in Sync rpc systems: ** the sender (client) blocks and cannot flood the receiver unless it uses threads ** the receiver (server) is guaranteed that the sender (ie client) is waiting to receive and if it has died then the reply can be discarded. With asyn rpc , my suggestion is to consider something along the following lines: * the client needs to allocate some buffer (or space for it) where replies are stored. On each async rpc call, it passes a ref to this buffer for storing replies. If the client does not pick up the replies fast enough then his next async call using that buffer space will block. * Note this makes the clients code tricky in what to do if it is blocked since one must ensure that a deadlock or starvation does not happen (but async messaging has always been tricky which is why cs community went with sync rpc). Note this problem does not arise on server side async-rpc since the client is blocked waiting for reply (unless the client also did async call but in that case its buffer, as per my suggestion, must be there to store the reply). > Change ipc.Client to support asynchronous calls > ----------------------------------------------- > > Key: HADOOP-12909 > URL: https://issues.apache.org/jira/browse/HADOOP-12909 > Project: Hadoop Common > Issue Type: New Feature > Components: ipc > Reporter: Tsz Wo Nicholas Sze > Assignee: Xiaobing Zhou > Attachments: HADOOP-12909-HDFS-9924.000.patch, > HADOOP-12909-HDFS-9924.001.patch, HADOOP-12909-HDFS-9924.002.patch, > HADOOP-12909-HDFS-9924.003.patch > > > In ipc.Client, the underlying mechanism is already supporting asynchronous > calls -- the calls shares a connection, the call requests are sent using a > thread pool and the responses can be out of order. Indeed, synchronous call > is implemented by invoking wait() in the caller thread in order to wait for > the server response. > In this JIRA, we change ipc.Client to support asynchronous mode. In > asynchronous mode, it return once the request has been sent out but not wait > for the response from the server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)