[ https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596667#comment-15596667 ]
Chris Douglas commented on HDFS-11026: -------------------------------------- The length prefix is a [WritableUtils|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/WritableUtils.java#L271] vint. The {{expiryDate}} field should start with a negative byte, since any valid {{expiryDate}} requires more than one byte to represent. In contrast, the [encoding|https://developers.google.com/protocol-buffers/docs/encoding] for PB prefixes each field with {{(field_number << 3) | wire_type}}, so as long as the first field has a {{field_number}} less than 16, the first byte will be positive. [~daryn], would this be an acceptable way to distinguish the two token types? [~anu] in HDFS-11010 [cited|https://issues.apache.org/jira/browse/HDFS-11010?focusedCommentId=15576699&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15576699] incompatible changes in DFSClient that have already affected backwards compatibility. Even if we adopt an approach that makes this switch redundant (e.g., separate 2.x/3.x ports), we can undo it without affecting clients. Thanks [~owen.omalley] (who knows too much about the internals of PB) > Convert BlockTokenIdentifier to use Protobuf > -------------------------------------------- > > Key: HDFS-11026 > URL: https://issues.apache.org/jira/browse/HDFS-11026 > Project: Hadoop HDFS > Issue Type: Task > Components: hdfs, hdfs-client > Affects Versions: 2.9.0, 3.0.0-alpha1 > Reporter: Ewan Higgs > Fix For: 3.0.0-alpha2 > > Attachments: blocktokenidentifier-protobuf.patch > > > {{BlockTokenIdentifier}} currently uses a {{DataInput}}/{{DataOutput}} > (basically a {{byte[]}}) and manual serialization to get data into and out of > the encrypted buffer (in {{BlockKeyProto}}). Other TokenIdentifiers (e.g. > {{ContainerTokenIdentifier}}, {{AMRMTokenIdentifier}}) use Protobuf. The > {{BlockTokenIdenfitier}} should use Protobuf as well so it can be expanded > more easily and will be consistent with the rest of the system. > NB: Release of this will require a version update since 2.8.x won't be able > to decipher {{BlockKeyProto.keyBytes}} from 2.8.y. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org