[
https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15589177#comment-15589177
]
Daryn Sharp commented on HDFS-11026:
------------------------------------
IMHO, a major upgrade shouldn't be a sufficient reason to introduce avoidable
protocol incompatibilities. It'll guarantee that 3.x is DOA for large
production deployments.
Yarn isn't exactly a good counter-example. As you point out, yarn made the
change to support RU so the cost/benefit was justified. What is the compelling
case that breaks the crucial feature that justified the yarn incompatibility?
The main difference between hdfs and yarn is scope of impact. The impact from
the yarn incompatibility was confined to the cluster being upgraded. This hdfs
incompatibility would impact clients on other clusters. Imagine trying to get
the stars to align with customers across many clusters to give a green light
for their SLAs possibly being impacted. Not once, but every time you upgrade
dozens of clusters.
In the end, the DN will have to support dual decoding for inter-cluster
clients, super long running clients holding old tokens, the balancer, etc. If
we go forward with this change, I'd like/prefer to see a designated 2.x
"minimum release" before a 3.x upgrade. That designated release would add the
latent support for PB tokens.
> Convert BlockTokenIdentifier to use Protobuf
> --------------------------------------------
>
> Key: HDFS-11026
> URL: https://issues.apache.org/jira/browse/HDFS-11026
> Project: Hadoop HDFS
> Issue Type: Task
> Components: hdfs, hdfs-client
> Affects Versions: 2.9.0, 3.0.0-alpha1
> Reporter: Ewan Higgs
> Fix For: 3.0.0-alpha2
>
> Attachments: blocktokenidentifier-protobuf.patch
>
>
> {{BlockTokenIdentifier}} currently uses a {{DataInput}}/{{DataOutput}}
> (basically a {{byte[]}}) and manual serialization to get data into and out of
> the encrypted buffer (in {{BlockKeyProto}}). Other TokenIdentifiers (e.g.
> {{ContainerTokenIdentifier}}, {{AMRMTokenIdentifier}}) use Protobuf. The
> {{BlockTokenIdenfitier}} should use Protobuf as well so it can be expanded
> more easily and will be consistent with the rest of the system.
> NB: Release of this will require a version update since 2.8.x won't be able
> to decipher {{BlockKeyProto.keyBytes}} from 2.8.y.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
