[
https://issues.apache.org/jira/browse/HADOOP-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13544625#comment-13544625
]
Binglin Chang commented on HADOOP-9151:
---------------------------------------
IMO, current RPC protocol has lot of issues, some can be solved without
breaking compatibility, some can not, this JIRA(maybe not worth to change) is
just one of them, and many others in HADOOP-8990 are not complete either. How
about first propose a complete change solution, with performance test results,
then evaluate whether it is worth or not?
In addition, list some of the unresolved or potential issues of RPC protocol:
# We can't assume msg.writeDelimitedTo avoids buffer copy, I read some protobuf
code, in fact it create a huge buffer to do serialization(basically the same as
"message.toByteArray()"), unfortunately "writeDelimitedTo" is used everywhere,
and compare to buffer allocation, zeroing, some extra buffer copy is hardly
performance bottleneck. So some statement in HADOOP-9163 and HADOOP-8084 is not
valid. If we truly want a zero copy implementation, perhaps we should reuse
CodedOutputStream throughout RPC code rather than each time create a new one,
this probably give more performance gains than avoid buffer copy. Whether or
not, first some test results is required before change.
# Length prefix have no convention at all, we have 4 byte, varint prefixes, and
some has -1 to denote special meaning(null string or ping id), some not, all
need additional code to deal with.
# Protobuf style varint is not convenient for writing aio, non-blocking
client/server(still possible), so is -1 prefix, if aio rpc is on the road map,
consider unified to 4 byte int prefix with no special meaning.
# Merge request/response into single proto has one extra benefits, the whole
protocol is now truly compatible, new "sub packets" can be freely added or
removed, whether it is a new header or error message, or some new packet type
to implement RPC_CONTINUATION_PACKET?
> Include RPC error info in RpcResponseHeader instead of sending it separately
> ----------------------------------------------------------------------------
>
> Key: HADOOP-9151
> URL: https://issues.apache.org/jira/browse/HADOOP-9151
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Sanjay Radia
> Assignee: Sanjay Radia
> Attachments: HADOOP-9151.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
