[ https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15589177#comment-15589177 ]
Daryn Sharp commented on HDFS-11026: ------------------------------------ IMHO, a major upgrade shouldn't be a sufficient reason to introduce avoidable protocol incompatibilities. It'll guarantee that 3.x is DOA for large production deployments. Yarn isn't exactly a good counter-example. As you point out, yarn made the change to support RU so the cost/benefit was justified. What is the compelling case that breaks the crucial feature that justified the yarn incompatibility? The main difference between hdfs and yarn is scope of impact. The impact from the yarn incompatibility was confined to the cluster being upgraded. This hdfs incompatibility would impact clients on other clusters. Imagine trying to get the stars to align with customers across many clusters to give a green light for their SLAs possibly being impacted. Not once, but every time you upgrade dozens of clusters. In the end, the DN will have to support dual decoding for inter-cluster clients, super long running clients holding old tokens, the balancer, etc. If we go forward with this change, I'd like/prefer to see a designated 2.x "minimum release" before a 3.x upgrade. That designated release would add the latent support for PB tokens. > Convert BlockTokenIdentifier to use Protobuf > -------------------------------------------- > > Key: HDFS-11026 > URL: https://issues.apache.org/jira/browse/HDFS-11026 > Project: Hadoop HDFS > Issue Type: Task > Components: hdfs, hdfs-client > Affects Versions: 2.9.0, 3.0.0-alpha1 > Reporter: Ewan Higgs > Fix For: 3.0.0-alpha2 > > Attachments: blocktokenidentifier-protobuf.patch > > > {{BlockTokenIdentifier}} currently uses a {{DataInput}}/{{DataOutput}} > (basically a {{byte[]}}) and manual serialization to get data into and out of > the encrypted buffer (in {{BlockKeyProto}}). Other TokenIdentifiers (e.g. > {{ContainerTokenIdentifier}}, {{AMRMTokenIdentifier}}) use Protobuf. The > {{BlockTokenIdenfitier}} should use Protobuf as well so it can be expanded > more easily and will be consistent with the rest of the system. > NB: Release of this will require a version update since 2.8.x won't be able > to decipher {{BlockKeyProto.keyBytes}} from 2.8.y. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org