Yes, The steps I followed where:
1-Intall lzo 2.06 in a machine with the same kernel as my nodes.
2-Compile there lzo 0.4.15 (in /lib replaced cdh3u3 per my hadoop 0.20.2
release)
3-Replace hadoop-lzo-0.4.9.jar for the now compiled hadoop-lzo-0.4.15.jar in
the hadoop lib directory of all my nodes and master
4-Put de generated native files in the native lib directory of all the nodes
and master
5-In my jar job, replaced the jar library hadoop-lzo-0.4.9.jar for
hadoop-lzo-0.4.15.jar
And sometimes when a job is running I get (4 times so the job gets killed):
...org.apache.hadoop.mapred.ReduceTask: Shuffling 3188320 bytes (1025174 raw
bytes) into RAM from attempt_201202291221_1501_m_000480_0
2012-03-02 02:32:55,496 INFO org.apache.hadoop.mapred.ReduceTask: Task
attempt_201202291221_1501_r_000105_0: Failed fetch #1 from
attempt_201202291221_1501_m_000046_0
2012-03-02 02:32:55,496 WARN org.apache.hadoop.mapred.ReduceTask:
attempt_201202291221_1501_r_000105_0 adding host hadoop-01.backend to
penalty box, next contact in 4 seconds
2012-03-02 02:32:55,496 INFO org.apache.hadoop.mapred.ReduceTask:
attempt_201202291221_1501_r_000105_0: Got 1 map-outputs from previous
failures
2012-03-02 02:32:55,497 FATAL org.apache.hadoop.mapred.TaskRunner:
attempt_201202291221_1501_r_000105_0 : Map output copy failure :
java.lang.InternalError: lzo1x_decompress returned: -8
at
com.hadoop.compression.lzo.LzoDecompressor.decompressBytesDirect(Native
Method)
at
com.hadoop.compression.lzo.LzoDecompressor.decompress(LzoDecompressor.java:305)
at
org.apache.hadoop.io.compress.BlockDecompressorStream.decompress(BlockDecompressorStream.java:76)
at
org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:75)
at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(ReduceTask.java:1553)
at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutput(ReduceTask.java:1432)
at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1285)
at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1216)
--
View this message in context:
http://lucene.472066.n3.nabble.com/LZO-exception-decompressing-returned-8-tp3783652p3792505.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
