Can you check the job configuration for these ~100 jobs? Do they have keep.failed.task.files set to true? If so, these files won't be deleted. If it doesn't, it could be a bug.
Sharing your configs for these jobs will definitely help. Thanks, +Vinod On Wed, Jan 9, 2013 at 6:41 AM, Ivan Tretyakov <itretya...@griddynamics.com>wrote: > Hello! > > I've found that jobcache directory became very large on our cluster, e.g.: > > # du -sh /data?/mapred/local/taskTracker/user/jobcache > 465G /data1/mapred/local/taskTracker/user/jobcache > 464G /data2/mapred/local/taskTracker/user/jobcache > 454G /data3/mapred/local/taskTracker/user/jobcache > > And it stores information for about 100 jobs: > > # ls -1 /data?/mapred/local/taskTracker/persona/jobcache/ | sort | uniq | > wc -l >
