[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ewan Higgs updated HDFS-11639: ------------------------------ Attachment: HDFS-11639-HDFS-9806.003.patch Attaching an updated patch that addresses some of [~virajith]'s comments: {quote} - In ProvidedBlocksBuilder#newLocatedBlock, the fileRegion should be resolved only if the block has PROVIDED locations (i.e., hasProvidedLocations is true). When dfs.namenode.provided.enabled is set to true, all LocatedBlock are created in this method, and for non-provided blocks, a resolution of BlockAlias is needed. - PBHelperClient#convertLocatedBlockProto() and PBHelperClient#convertLocatedBlock() should be modified to decode/encode the BlockAlias bytes. - How about decoding the blockAlias bytes in DataXceiver#readBlock using a new DataTransferProtoUtil#blockAliasFromProto(bytes[] blockAlias method instead of using the BlockAlias#builder()? The former will be in-line with the way the protobufs are decoded in DataXceiver. Further, if in the future a different BlockAlias is used, the current implementation of using the FileRegion#Builder in DataXceiver#readBlock will be hard to extend (will end up being try FileRegion#Builder, if null try BlockAliasXX#Builder and so on). - Similar to passing on BlockAlias from DataXceiver#readBlock to BlockSender, it should be passed along from DataXceiver#readBlock to BlockReceiver. However, we would not need it till we have writes implemented. - DataStreamer#blockAlias will never be non-null. I think it should be initialized in DFSOutputStream. {quote} This also added BlockAlias to the transferBlocks message but it isn't incorporated in since this is a write message. e.g. {{DNA_TRANSFER}} has not yet been updated. > [READ] Encode the BlockAlias in the client protocol > --------------------------------------------------- > > Key: HDFS-11639 > URL: https://issues.apache.org/jira/browse/HDFS-11639 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs > Reporter: Ewan Higgs > Assignee: Ewan Higgs > Attachments: HDFS-11639-HDFS-9806.001.patch, > HDFS-11639-HDFS-9806.002.patch, HDFS-11639-HDFS-9806.003.patch > > > As part of the {{PROVIDED}} storage type, we have a {{BlockAlias}} type which > encodes information about where the data comes from. i.e. URI, offset, > length, and nonce value. This data should be encoded in the protocol > ({{LocatedBlockProto}} and the {{BlockTokenIdentifier}}) when a block is > available using the PROVIDED storage type. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org