[ https://issues.apache.org/jira/browse/HADOOP-11552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated HADOOP-11552: ------------------------------------ Attachment: HADOOP-11552.3.txt Updated patch to fix the rat check and javac warning. I don't think the unit test failure is related. bq. I would like to see some code actually using this before we add it, to make sure we are getting the APIs right. [~cmccabe] - the APIs are being added as unstable for now. There's at least one follow up jira to make use a common thread pool to handle requests and send responses - or at least have the RPC layer provide a pool for responses. At the mement apps would have to setup threads to send responses in parallel. The test does show how this would be used in an application - where the specific method on a protocol would need to indicate intent to send a delayed response, and relinquish control. I think it'll be useful to get this in - so that it's possible to start trying out this mechanism. > Allow handoff on the server side for RPC requests > ------------------------------------------------- > > Key: HADOOP-11552 > URL: https://issues.apache.org/jira/browse/HADOOP-11552 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc > Reporter: Siddharth Seth > Assignee: Siddharth Seth > Attachments: HADOOP-11552.1.wip.txt, HADOOP-11552.2.txt, > HADOOP-11552.3.txt, HADOOP-11552.3.txt > > > An RPC server handler thread is tied up for each incoming RPC request. This > isn't ideal, since this essentially implies that RPC operations should be > short lived, and most operations which could take time end up falling back to > a polling mechanism. > Some use cases where this is useful. > - YARN submitApplication - which currently submits, followed by a poll to > check if the application is accepted while the submit operation is written > out to storage. This can be collapsed into a single call. > - YARN allocate - requests and allocations use the same protocol. New > allocations are received via polling. > The allocate protocol could be split into a request/heartbeat along with a > 'awaitResponse'. The request/heartbeat is sent only when there's a request or > on a much longer heartbeat interval. awaitResponse is always left active with > the RM - and returns the moment something is available. > MapReduce/Tez task to AM communication is another example of this pattern. > The same pattern of splitting calls can be used for other protocols as well. > This should serve to improve latency, as well as reduce network traffic since > the keep-alive heartbeat can be sent less frequently. > I believe there's some cases in HDFS as well, where the DN gets told to > perform some operations when they heartbeat into the NN. -- This message was sent by Atlassian JIRA (v6.3.4#6332)