Re: EC2, Max tasks, under utilized?

2009-06-23 Thread Hong Tang
Do you use block compression in sequence file? How large is your total dataset? On Jun 23, 2009, at 7:50 AM, Saptarshi Guha wrote: Hello, I should also point out that I'm using a SequenceFileInputFormat. Regards Saptarshi Guha On Tue, Jun 23, 2009 at 10:43 AM, Saptarshi Guha wrote: Hello

Re: Fastlz coming?

2009-06-04 Thread Hong Tang
Using com.hadoop.compression.lzo.LzoCodec is not much different from using other codecs: adding the hadoop-gpl-compression-0.1.0-dev.jar in your classpath, and add the path to the native library libgplcompression.so in system property java.library.path. Hope this helps, Hong On Jun 4, 200

Re: "Filesystem closed" errors

2008-11-25 Thread Hong Tang
Does your code ever call fs.close()? If so, https://issues.apache.org/ jira/browse/HADOOP-4655 might be relevant to your problem. On Nov 25, 2008, at 9:07 PM, David B. Ritch wrote: Do you have speculative execution enabled? I've seen error messages like this caused by speculative execution.