I have a pig script. If I don't set any codec for Map output for hadoop
cluster, no problem. Now I made the following compression settings, the job
failed and the error message is shown below. I guess there are some other
settings that should be correctly set together with using the compression.
Hi,
Can you please let us know what platform you are running on your hadoop
machines?
For gzip and lzo to work, you need supported hadoop native libraries ( I
remember reading on this somewhere in hadoop wiki :) )
Amogh
On 2/23/10 8:16 AM, "jiang licht" wrote:
I have a pig script. If I don't
ssion setting on
Fedora 8 and 0.19.1 without any problem. So, I think it might depends on some
other settings (wrt what spill is about).
Thanks,
Michael
--- On Mon, 2/22/10, Amogh Vasekar wrote:
From: Amogh Vasekar
Subject: Re: java.io.IOException: Spill failed when using w/ GzipCodec fo
hael
--- On Mon, 2/22/10, Amogh Vasekar wrote:
From: Amogh Vasekar
Subject: Re: java.io.IOException: Spill failed when using w/ GzipCodec for Map
output
To: "common-user@hadoop.apache.org"
Date: Monday, February 22, 2010, 11:27 PM
Hi,
Can you please let us know what platform you are ru
Thanks, Amogh. Good to know :)
Michael
--- On Tue, 2/23/10, Amogh Vasekar wrote:
From: Amogh Vasekar
Subject: Re: java.io.IOException: Spill failed when using w/ GzipCodec for Map
output
To: "common-user@hadoop.apache.org"
Date: Tuesday, February 23, 2010, 1:45 AM
Hi,
Certainly
