Hi All, Could I try other codec besides lzo using hadoop-common TFile property "io.compression.codec.lzo.class"? If not, please ignore following information.
I was trying "io.compression.codec.lzo.class" with value "com.ibm.biginsights.compress.CmxCodec" (not the default "org.apache.hadoop.io.compress.LzoCodec"). The jar for this CmxCodec is called ibm-compression.jar. But I always encountered following errors: "java.io.IOException: LZO codec class not specified. Did you forget to set property io.compression.codec.lzo.class?" I lunched mapreduce jobs and from "job*.xml" I could find this property was set correctly. And also could find ibm-compression.jar in "java.class.path". From following code, seems "io.compression.codec.lzo.class" does not work for other codec besides default. Hadoop will always treat it as defaultClazz. I think it is a bug and opened this jira. Please help give a glance and give your comments. Thanks 77 private static final String defaultClazz = 78 "org.apache.hadoop.io.compress.LzoCodec"; 82 public synchronized boolean isSupported() { 83 if (!checked) { 84 checked = true; 85 String extClazz = 86 (conf.get(CONF_LZO_CLASS) == null ? System 87 .getProperty(CONF_LZO_CLASS) : null); 88 String clazz = (extClazz != null) ? extClazz : defaultClazz;