Pig is supposed to remove all these temporary files, as long as the
java process finishes in such a way that it has a chance to clean up
(ie, no one does a kill -9 on it or something). Can you file a JIRA
with a reproducible case so we can track this down and fix it?
Alan.
On Nov 22, 2010, at 2:20 PM, rakesh kothari wrote:
This is is happening on PIG 0.7 as well. When I use PigServer to
launch pig jobs from Java lots of tmp files (e.g. directory /tmp/
temp-203305867) gets created that are not deleted at the end of the
Job.
"hadoop.tmp.dir" points to some other directory "/tmp/hadoop-userA".
Any ideas on how to get path to these tmp folders so that I can
explicitly delete them at the end of the Job ?
Thanks,
-Rakesh
Date: Mon, 25 Oct 2010 20:30:24 +0200
From: [email protected]
To: [email protected]
Subject: Re: How to make PIG delete its temporary files ?
Thanks for your answer, I will investigate on that.
Le 25/10/10 08:26, Rekha Joshi a écrit :
Hi Vincent,
I recall it happens under some specific condition and is resolved
in later versions of pig/hadoop by adding an explicit cleanup
before final exit.
As a workaround in 0.6 you can code finally{} in your java file to
remove files from your user specific folder under
hadoop.tmp.dir.HTH!
Thanks& Regards,
/Rekha.
On 10/24/10 5:24 AM, "Vincent Barat"<[email protected]> wrote:
I forgot to mention that I use PIG 0.6.0, it may help !
Le 23/10/10 13:29, Vincent Barat a écrit :
Hello,
I face an issue with PIG temporary files: they are not deleted
once a job is terminated.
I got my HDFS storage full of PIG temporary files.
I use PIG from Java using a PigServer object.
Is there is way to ask PIG to cleanup all that it created to
perform a job ?
Thanks a lot
