Using LinuxTaskController -- Local Dir Permissions Error

2009-10-23 Thread Aaron Kimball
Hi all, I'm trying to get the LinuxTaskController working (on the svn trunk) on a pseudo-distributed cluster. It's being quite frustrating. I compiled common, hdfs, and mapred jars with 'ant jar' and copied everything together into the same directory structure. I then ran: $ cd

DefaultCodec vs. LZO

2009-10-23 Thread Zheng Shao
We are using Hadoop 0.20 and mapred.map.output.compression.Codec is set to DefaultCodec. We tried LZO but the performance seems very similar to DefaultCodec. I heard of a lot of good words about LZO. So did anybody compare LZO with DefaultCodec? Is there a big difference? We are running CentOS

Re: DefaultCodec vs. LZO

2009-10-23 Thread He Yongqiang
I did a search, and find the below two links, hope they are helpful. http://blog.oskarsson.nu/2009/03/hadoop-feat-lzo-save-disk-space-and.html http://mail.jabber.org/pipermail/standards/2005-October/008768.html I think Hadoop’s DefaultCodec is using native Zlib when possible. Thanks, Yongqiang

Re: DefaultCodec vs. LZO

2009-10-23 Thread Hong Tang
You mean this: http://code.google.com/p/hadoop-gpl-compression/issues/detail?id=17 ? Based on the description, the bug affects LzopCompressor, and should have no impact on the packaging part, right? -Hong On Oct 23, 2009, at 2:17 PM, Todd Lipcon wrote: I would, except currently the github