[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637010#comment-17637010 ]
xinqiu.hu commented on HADOOP-18536: ------------------------------------ HADOOP-18536 is a better approach than HADOOP-18533, with the goal of reducing rpc request copies. HADOOP-18534 is for early release of direct memory > RPC Client Improvement > ---------------------- > > Key: HADOOP-18536 > URL: https://issues.apache.org/jira/browse/HADOOP-18536 > Project: Hadoop Common > Issue Type: Improvement > Components: rpc-server > Reporter: xinqiu.hu > Priority: Minor > > In the RPC Client, before a request (including RpcRequestHeaderProto, > RequestHeaderProto, Message Payload) is sent, they will be copied to the three > CodedOutputStream internal byte arrays, and then aggregated to the > ResponseBuffer > internal byte array. Then the ResponseBuffer byte array is written to a > BufferedOutputStream and finally to a SocketOutputStream. > To simplify the writing process, Maybe we can copy them directly to a big > CodedOutputStream and send them directly to the IpcStreams#out. To achieve > this, I > propose HADOOP-18533. But it brings the following two side effects. > # The generic declaration of rpcRequestQueue inside Client has been changed > to Object > # The serialization of protobuf has been moved to rpcRequestThread, because > rpcRequestThread is a single thread for each connection, which may have a > performance impact. > For the above reasons, I propose this. This pr brings the following benefits > # For each rpc request, avoid creating a ResponseBuffer of 1024 bytes > # For each rpc request, Reduce one copy > # For each rpc request, combine the three fragmented CodedOutputStreams into > one > # No side effects like HADOOP-18533 -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org