[ https://issues.apache.org/jira/browse/HIVE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain resolved HIVE-3251. ------------------------------ Resolution: Fixed Hadoop Flags: Reviewed Committed. Thanks Tim > Hive doesn't remove scrach directories while killing running MR job > ------------------------------------------------------------------- > > Key: HIVE-3251 > URL: https://issues.apache.org/jira/browse/HIVE-3251 > Project: Hive > Issue Type: Bug > Components: Server Infrastructure > Reporter: Gang Tim Liu > Assignee: Gang Tim Liu > Attachments: HIVE-3151.patch > > > While killing running MR job, hive doesn't clean up scratch directory > (mapred.cache.files). So that, afterwards, scratch directory is left there in > hdfs. HDFS name node doesn't know it and try to do lease recovery. while such > instances happen more, it will eventually crash namenode. > The fix is to leverage hdfs clean up functionality. While creating scratch > dirs, hive registers it to hdfs cleanup hook. While killing happens, hdfs > will clean them up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira