[ https://issues.apache.org/jira/browse/HDFS-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14056477#comment-14056477 ]
Kihwal Lee commented on HDFS-6618: ---------------------------------- With the new patch {{removedINodes}} is passed to {{FSNamesystem#removePathAndBlocks()}} while in the write lock. The method was modified to conditionally acquire the directory lock. I didn't move the removal to {{FSDirectory}}, since we may want to do something with the inodes in {{FSNamesystem}} later as a part of failure handling in a separate jira. > Remove deleted INodes from INodeMap right away > ---------------------------------------------- > > Key: HDFS-6618 > URL: https://issues.apache.org/jira/browse/HDFS-6618 > Project: Hadoop HDFS > Issue Type: Bug > Affects Versions: 2.5.0 > Reporter: Kihwal Lee > Assignee: Kihwal Lee > Priority: Blocker > Attachments: HDFS-6618.AbstractList.patch, > HDFS-6618.inodeRemover.patch, HDFS-6618.inodeRemover.v2.patch, > HDFS-6618.patch, HDFS-6618.simpler.patch > > > After HDFS-6527, we have not seen the edit log corruption for weeks on > multiple clusters until yesterday. Previously, we would see it within 30 > minutes on a cluster. > But the same condition was reproduced even with HDFS-6527. The only > explanation is that the RPC handler thread serving {{addBlock()}} was > accessing stale parent value. Although nulling out parent is done inside the > {{FSNamesystem}} and {{FSDirectory}} write lock, there is no memory barrier > because there is no "synchronized" block involved in the process. > I suggest making parent volatile. -- This message was sent by Atlassian JIRA (v6.2#6252)