intermediate files of killed tasks not purged

2009-04-28 Thread Sandhya E
Hi Under hadoop-tmp-dir/mapred/local there are directories like attempt_200904262046_0026_m_02_0 Each of these directories contains files of format: intermediate.1 intermediate.2 intermediate.3 intermediate.4 intermediate.5 There are many directories in this format. All these correspond to

Re: intermediate files of killed tasks not purged

2009-04-28 Thread Edward J. Yoon
Hi, It seems related with https://issues.apache.org/jira/browse/HADOOP-4654. On Tue, Apr 28, 2009 at 4:01 PM, Sandhya E sandhyabhas...@gmail.com wrote: Hi Under hadoop-tmp-dir/mapred/local there are directories like attempt_200904262046_0026_m_02_0 Each of these directories contains

Re: intermediate files of killed tasks not purged

2009-04-28 Thread Amareshwari Sriramadasu
Hi Sandhya, Which version of HADOOP are you using? There could be attempt_id directories in mapred/local, pre 0.17. Now, there should not be any such directories. From version 0.17 onwards, the attempt directories will be present only at mapred/local/taskTracker/jobCache/jobid/attempid . If

Re: intermediate files of killed tasks not purged

2009-04-28 Thread Amareshwari Sriramadasu
Again, where are you seeing the attemptid directories? are they at mapred/local/attemptid or at mapred/local/taskTracker/jobCache/jobid/attempid. If you are seeing files at mapred/local/attemptid, then it is bug. Please raise a jira and attach tasktracker logs if possible. If not,

Re: intermediate files of killed tasks not purged

2009-04-28 Thread Sandhya E
Attempt directories are in hadoop-tmp/mapred/local I grep'd for one of the attempt that has left over in mapred/local in tasktracker logs: 09/04/27 21:07:19 INFO mapred.TaskTracker: LaunchTaskAction: attempt_200902120108_44218_r_00_0 09/04/27 21:07:29 INFO mapred.TaskTracker: