[ https://issues.apache.org/jira/browse/MAPREDUCE-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163952#comment-13163952 ]
Alejandro Abdelnur commented on MAPREDUCE-3515: ----------------------------------------------- Wing Yew, would you please verify if the build of Hadoop you were testing includes MAPREDUCE-3500? MAPREDUCE-3500 should have fixed that problem. > hadoop 0.23: native compression libraries not being loaded > ---------------------------------------------------------- > > Key: MAPREDUCE-3515 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3515 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 > Affects Versions: 0.23.0 > Reporter: Wing Yew Poon > > I installed the hadoop package from the Bigtop hadoop 0.23 branch. Among > other files, the package installs > /usr/lib/hadoop/lib/native > /usr/lib/hadoop/lib/native/libhadoop.a > /usr/lib/hadoop/lib/native/libhadoop.so.1 > /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0 > /usr/lib/hadoop/lib/native/libhdfs.a > I ran a simple job using compression: > hadoop jar /usr/lib/hadoop/hadoop-mapreduce-examples.jar wordcount -D > mapreduce.output.fileoutputformat.compress=true -D > mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec > examples/text wordcount-gz > I see > 11/12/06 13:42:06 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > 11/12/06 13:42:06 WARN snappy.LoadSnappy: Snappy native library not loaded > and at the end, I see > -rw-r--r-- 1 root supergroup 0 2011-12-06 13:42 > wordcount-gz/_SUCCESS > -rw-r--r-- 1 root supergroup 46228 2011-12-06 13:42 > wordcount-gz/part-r-00000.gz > so the output is compressed, but from the log message, I assume that the > native library is not being loaded and that the java gzip is being used. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira