[ https://issues.apache.org/jira/browse/HADOOP-12678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
madhumita chakraborty updated HADOOP-12678: ------------------------------------------- Attachment: HADOOP-12678.004.patch > Handle empty rename pending metadata file during atomic rename in redo path > --------------------------------------------------------------------------- > > Key: HADOOP-12678 > URL: https://issues.apache.org/jira/browse/HADOOP-12678 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure > Reporter: madhumita chakraborty > Assignee: madhumita chakraborty > Priority: Critical > Attachments: HADOOP-12678.001.patch, HADOOP-12678.002.patch, > HADOOP-12678.003.patch, HADOOP-12678.004.patch > > > Handle empty rename pending metadata file during atomic rename in redo path > During atomic rename we create metadata file for rename(-renamePending.json). > We create that in 2 steps > 1. We create an empty blob corresponding to the .json file in its real > location > 2. We create a scratch file to which we write the contents of the rename > pending which is then copied over into the blob described in 1 > If process crash occurs after step 1 and before step 2 is complete - we will > be left with a zero size blob corresponding to the pending rename metadata > file. > This kind of scenario can happen in the /hbase/.tmp folder because it is > considered a candidate folder for atomic rename. Now when HMaster starts up > it executes listStatus on the .tmp folder to clean up pending data. At this > stage due to the lazy pending rename complete process we look for these json > files. On seeing an empty file the process simply throws a fatal exception > assuming something went wrong. -- This message was sent by Atlassian JIRA (v6.3.4#6332)