[ https://issues.apache.org/jira/browse/HDFS-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118798#comment-14118798 ]
Colin Patrick McCabe commented on HDFS-6921: -------------------------------------------- Interesting discussion. I don't think adding this field will cause DistCp to fail. DistCp doesn't currently check this field, so it will have no idea whether it's there or not. It is a little concerning that FileStatus#read(FileStatus.write) will no longer return the original object (we can't "round trip" it) but this is already true of many (all?) the subclasses of FileStatus, like LocatedFileStatus. They just don't bother serializing the new fields they add so they already have this problem. I filed HDFS-6984 to remove the Writable interface from FileStatus completely in Hadoop 3.0. In the meantime, we could support "round tripping" FileStatus by packing the isLazyPersist bit into the sign bit of the replication field. Would that address the compatibility concerns? bq. Another issue is that lazy persist should be internal to the HDFS itself, it is much better to keep it fully inside. If there is nothing in FileStatus, how can users find out this information? Perhaps by using a extended attribute? That might actually be a good choice. > Add LazyPersist flag to FileStatus > ---------------------------------- > > Key: HDFS-6921 > URL: https://issues.apache.org/jira/browse/HDFS-6921 > Project: Hadoop HDFS > Issue Type: Sub-task > Affects Versions: HDFS-6581 > Reporter: Arpit Agarwal > Assignee: Arpit Agarwal > Fix For: HDFS-6581 > > Attachments: HDFS-6921.01.patch, HDFS-6921.02.patch > > > A new flag will be added to FileStatus to indicate that a file can be lazily > persisted to disk i.e. trading reduced durability for better write > performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)