[ https://issues.apache.org/jira/browse/HADOOP-11552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688162#comment-15688162 ]
Jian He commented on HADOOP-11552: ---------------------------------- We'll get this committed tomorrow, if no more comments from others > Allow handoff on the server side for RPC requests > ------------------------------------------------- > > Key: HADOOP-11552 > URL: https://issues.apache.org/jira/browse/HADOOP-11552 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc > Reporter: Siddharth Seth > Assignee: Siddharth Seth > Labels: BB2015-05-TBR > Attachments: HADOOP-11552.05.patch, HADOOP-11552.06.patch, > HADOOP-11552.07.patch, HADOOP-11552.08.patch, HADOOP-11552.1.wip.txt, > HADOOP-11552.2.txt, HADOOP-11552.3.txt, HADOOP-11552.3.txt, HADOOP-11552.4.txt > > > An RPC server handler thread is tied up for each incoming RPC request. This > isn't ideal, since this essentially implies that RPC operations should be > short lived, and most operations which could take time end up falling back to > a polling mechanism. > Some use cases where this is useful. > - YARN submitApplication - which currently submits, followed by a poll to > check if the application is accepted while the submit operation is written > out to storage. This can be collapsed into a single call. > - YARN allocate - requests and allocations use the same protocol. New > allocations are received via polling. > The allocate protocol could be split into a request/heartbeat along with a > 'awaitResponse'. The request/heartbeat is sent only when there's a request or > on a much longer heartbeat interval. awaitResponse is always left active with > the RM - and returns the moment something is available. > MapReduce/Tez task to AM communication is another example of this pattern. > The same pattern of splitting calls can be used for other protocols as well. > This should serve to improve latency, as well as reduce network traffic since > the keep-alive heartbeat can be sent less frequently. > I believe there's some cases in HDFS as well, where the DN gets told to > perform some operations when they heartbeat into the NN. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org