It seems that pig generates some folders/files under "/tmp" in HDFS for pig
jobs. I remember that hadoop saves such intermediate results (map output, etc.)
in non-hdfs folders, which are specified in mapred-site.xml. So, is there a way
to tell pig to store such data to a non-hdfs folder?
Thanks
ty the same way as above. I am
not sure how they all interrelate.
There is also a tmpdir variable for the JVM, I am not sure what reads that. I
just set them all the same.
- Original Message
> From: jiang licht
> To: pig-user@hadoop.apache.org
> Sent: Mon, September 13, 2010 5:
Michael
--- On Mon, 9/13/10, Mr. Jan Walter wrote:
From: Mr. Jan Walter
Subject: Re: specify temp folder?
To: pig-user@hadoop.apache.org
Date: Monday, September 13, 2010, 5:01 PM
Set the following parameter in your workers' mapred-site.xml, and change the
value to what you want:
mapr
in HDFS (maybe just in "interactive mode",
not sure if I remember correctly, will check this), which means they get
replicated and use much more space.
Thanks,
Michael
--- On Mon, 9/13/10, Mr. Jan Walter wrote:
From: Mr. Jan Walter
Subject: Re: specify temp folder?
To: pig-user@
;, not sure if I remember correctly, will check
this), which means they get replicated and use much more space.
Thanks,
Michael
--- On Mon, 9/13/10, Mr. Jan Walter wrote:
From: Mr. Jan Walter
Subject: Re: specify temp folder?
To: pig-user@hadoop.apache.org
Date: Monday, September 13, 2010, 5
