[
https://issues.apache.org/jira/browse/HADOOP-8990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu updated HADOOP-8990:
----------------------------
Description:
There are a few inconsistencies in the current RPC protocol that makes new
client implementation (especially non-blocking) unnecessarily hard. For example:
# proto file naming
RpcPayloadHeader.proto include not only RpcPayLoadHeaderProto, but also
RpcResponseHeaderProto, which is irrelevant to the file name.
hadoop_rpc.proto only include HadoopRpcRequestProto, and the filename
"hadoop_rpc" is strange comparing to other .proto file names.
How about merge those two file into HadoopRpc.proto?
# proto class naming
In rpc request RpcPayloadHeaderProto includes callId, but in rpc response
callId is included in RpcResponseHeaderProto, and there is also
HadoopRpcRequestProto, this is just too confusing.
# The rpc system is not fully protobuf based, there are still some Writables:
RpcRequestWritable, RpcResponseWritable, rpc response exception name, stack
trace string and sasl request/response.
RpcRequestWritable uses protobuf style varint32 prefix, but RpcResponseWritable
uses int32 prefix, why this inconsistency?
Currently rpc request is splitted into length, PayLoadHeader and PayLoad, and
response into RpcResponseHeader, response and error message. Wrapping request
and response into single RequestProto and ResponseProto would be better, since
this gives a formal complete wire format definition.
These issues above make it very confusing and hard for developers to use these
rpc interfaces.
was:
1. proto file naming
RpcPayloadHeader.proto include not only RpcPayLoadHeaderProto, but also
RpcResponseHeaderProto, which is irrelevant to the file name.
hadoop_rpc.proto only include HadoopRpcRequestProto, and the filename
"hadoop_rpc" is strange comparing to other .proto file names.
How about merge those two file into HadoopRpc.proto?
2. proto class naming
In rpc request RpcPayloadHeaderProto includes callId, but in rpc response
callId is included in RpcResponseHeaderProto, and there is also
HadoopRpcRequestProto, this is just too confusing.
3. The rpc system is not fully protobuf based, there are still some Writables:
RpcRequestWritable and RpcResponseWritable.
rpc response exception name and stack trace string.
And RpcRequestWritable uses protobuf style varint32 prefix, but
RpcResponseWritable uses int32 prefix, why this inconsistency?
Currently rpc request is splitted into length, PayLoadHeader and PayLoad, and
response into RpcResponseHeader, response and error message.
I think wrap request and response into single RequstProto and ResponseProto is
better, cause this gives a formal complete wire format definition,
or developer need to look into the source code and hard coding the
communication format.
These issues above make it very confusing and hard for developers to use these
rpc interfaces.
Some of these issues can be solved without breaking compatibility, but some can
not, but at least we need to know what will be changed and what will stay
stable?
Priority: Major (was: Minor)
Hadoop Flags: Incompatible change
Summary: Clean up RPC protocol for consistency (was: Some minor
issus in protobuf based ipc)
> Clean up RPC protocol for consistency
> --------------------------------------
>
> Key: HADOOP-8990
> URL: https://issues.apache.org/jira/browse/HADOOP-8990
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Binglin Chang
> Assignee: Sanjay Radia
>
> There are a few inconsistencies in the current RPC protocol that makes new
> client implementation (especially non-blocking) unnecessarily hard. For
> example:
> # proto file naming
> RpcPayloadHeader.proto include not only RpcPayLoadHeaderProto, but also
> RpcResponseHeaderProto, which is irrelevant to the file name.
> hadoop_rpc.proto only include HadoopRpcRequestProto, and the filename
> "hadoop_rpc" is strange comparing to other .proto file names.
> How about merge those two file into HadoopRpc.proto?
> # proto class naming
> In rpc request RpcPayloadHeaderProto includes callId, but in rpc response
> callId is included in RpcResponseHeaderProto, and there is also
> HadoopRpcRequestProto, this is just too confusing.
> # The rpc system is not fully protobuf based, there are still some Writables:
> RpcRequestWritable, RpcResponseWritable, rpc response exception name, stack
> trace string and sasl request/response.
> RpcRequestWritable uses protobuf style varint32 prefix, but
> RpcResponseWritable uses int32 prefix, why this inconsistency?
> Currently rpc request is splitted into length, PayLoadHeader and PayLoad, and
> response into RpcResponseHeader, response and error message. Wrapping request
> and response into single RequestProto and ResponseProto would be better,
> since this gives a formal complete wire format definition.
> These issues above make it very confusing and hard for developers to use
> these rpc interfaces.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
