[
https://issues.apache.org/jira/browse/HADOOP-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204595#comment-15204595
]
Sanjay Radia commented on HADOOP-12909:
---------------------------------------
I haven't had a chance to look at the patch or review all the comments, but
wanted to bring attention to one issue wrt async rpc that is well known by
implementors and practitioners of message passing & rpc systems (excuse me if
this has already been covered):
* One needs to watch out for buffer management. ie. aync rpc/message passing
has the potential to use up memory for buffering the messages. This is
prevented in Sync rpc systems:
** the sender (client) blocks and cannot flood the receiver unless it uses
threads
** the receiver (server) is guaranteed that the sender (ie client) is waiting
to receive and if it has died then the reply can be discarded.
With asyn rpc , my suggestion is to consider something along the following
lines:
* the client needs to allocate some buffer (or space for it) where replies
are stored. On each async rpc call, it passes a ref to this buffer for storing
replies. If the client does not pick up the replies fast enough then his next
async call using that buffer space will block.
* Note this makes the clients code tricky in what to do if it is blocked since
one must ensure that a deadlock or starvation does not happen (but async
messaging has always been tricky which is why cs community went with sync rpc).
Note this problem does not arise on server side async-rpc since the client is
blocked waiting for reply (unless the client also did async call but in that
case its buffer, as per my suggestion, must be there to store the reply).
> Change ipc.Client to support asynchronous calls
> -----------------------------------------------
>
> Key: HADOOP-12909
> URL: https://issues.apache.org/jira/browse/HADOOP-12909
> Project: Hadoop Common
> Issue Type: New Feature
> Components: ipc
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Xiaobing Zhou
> Attachments: HADOOP-12909-HDFS-9924.000.patch,
> HADOOP-12909-HDFS-9924.001.patch, HADOOP-12909-HDFS-9924.002.patch,
> HADOOP-12909-HDFS-9924.003.patch
>
>
> In ipc.Client, the underlying mechanism is already supporting asynchronous
> calls -- the calls shares a connection, the call requests are sent using a
> thread pool and the responses can be out of order. Indeed, synchronous call
> is implemented by invoking wait() in the caller thread in order to wait for
> the server response.
> In this JIRA, we change ipc.Client to support asynchronous mode. In
> asynchronous mode, it return once the request has been sent out but not wait
> for the response from the server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
