Hi All,
Could I try other codec besides lzo using hadoop-common TFile property
"io.compression.codec.lzo.class"? If not, please ignore following information.
I was trying "io.compression.codec.lzo.class" with value
"com.ibm.biginsights.compress.CmxCodec" (not the default
"org.apache.hadoop.io.compress.LzoCodec"). The jar for this CmxCodec is called
ibm-compression.jar. But I always encountered following errors:
"java.io.IOException: LZO codec class not specified. Did you forget to set
property io.compression.codec.lzo.class?"
I lunched mapreduce jobs and from "job*.xml" I could find this property was set
correctly. And also could find ibm-compression.jar in "java.class.path".
From following code, seems "io.compression.codec.lzo.class" does not work for
other codec besides default. Hadoop will always treat it as defaultClazz. I
think it is a bug and opened this jira. Please help give a glance and give your
comments. Thanks
77 private static final String defaultClazz =
78 "org.apache.hadoop.io.compress.LzoCodec";
82 public synchronized boolean isSupported() {
83 if (!checked) {
84 checked = true;
85 String extClazz =
86 (conf.get(CONF_LZO_CLASS) == null ? System
87 .getProperty(CONF_LZO_CLASS) : null);
88 String clazz = (extClazz != null) ? extClazz : defaultClazz;