[
https://issues.apache.org/jira/browse/HDFS-2060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047324#comment-13047324
]
Todd Lipcon commented on HDFS-2060:
-----------------------------------
Here's my proposal:
- in a HADOOP jira, modify ObjectWritable to be able to serialize protocol
buffers (ie the {{Message}}) type. This allows coexistence of writables and
protobufs within a single protocol so we can do a staged switchover without a
wholesale switch in rpc engine implementation.
- for each of the RPCs in ClientProtocol:
-- define a new protobuf for the request params and the resposne params. EG
{{SetReplicationRequestProto}} and {{SetReplicationResponseProto}}.
-- add a new RPC {{SetReplicationResponseProto
setReplication(SetReplicationResponseProto param)}}
-- implement that RPC by simply writing the boilerplate wrap/unwrap code
to/from the original parameters
-- deprecate the old call
-- switch callers internal to HDFS to the new proto-based call
The nice thing about this approach is that it (a) maintains API
backwards-compatibility for one version for folks directly using protocol
proxies to talk to the NN, and (b) allows us to do a staged switchover, one RPC
at a time.
It doesn't address the ability to evolve the underlying IPC/RPC transport, but
that could be tackled in parallel or sequenced after this.
> DFS client RPCs using protobufs
> -------------------------------
>
> Key: HDFS-2060
> URL: https://issues.apache.org/jira/browse/HDFS-2060
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
>
> The most important place for wire-compatibility in DFS is between clients and
> the cluster, since lockstep upgrade is very difficult and a single client may
> want to talk to multiple server versions. So, I'd like to focus this JIRA on
> making the RPCs between the DFS client and the NN/DNs wire-compatible using
> protocol buffer based serialization.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
