[ https://issues.apache.org/jira/browse/HDFS-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453510#comment-13453510 ]
Kihwal Lee commented on HDFS-3731: ---------------------------------- Since Bobby is busy is busy with the 0.23.3 release, I am taking over the porting. The test failure happens because all RWR blocks are thrown away. When namenode sees OP_ADD on an existing file while reading edits, it turns the inode into "under construction", but does not do anything to blocks. Since it thinks there is no blocks under construction, the reported RWR blocks are considered corrupt. This only happens in 0.22 and 0.23. In 2.x and trunk, it was fixed as part of HDFS-1623 (HA). This is unacceptable behavior if append/hsync is used. I will file a separate jira for 0.22 and 0.23 and make it as a dependency of this jira. > 2.0 release upgrade must handle blocks being written from 1.0 > ------------------------------------------------------------- > > Key: HDFS-3731 > URL: https://issues.apache.org/jira/browse/HDFS-3731 > Project: Hadoop HDFS > Issue Type: Bug > Components: data-node > Affects Versions: 2.0.0-alpha > Reporter: Suresh Srinivas > Assignee: Kihwal Lee > Priority: Blocker > Fix For: 2.0.2-alpha > > Attachments: hadoop1-bbw.tgz, HDFS-3731.002.patch, HDFS-3731.003.patch > > > Release 2.0 upgrades must handle blocks being written to (bbw) files from 1.0 > release. Problem reported by Brahma Reddy. > The {{DataNode}} will only have one block pool after upgrading from a 1.x > release. (This is because in the 1.x releases, there were no block pools-- > or equivalently, everything was in the same block pool). During the upgrade, > we should hardlink the block files from the {{blocksBeingWritten}} directory > into the {{rbw}} directory of this block pool. Similarly, on {{-finalize}}, > we should delete the {{blocksBeingWritten}} directory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira