The documentation put out by Cloudera for running Accumulo on CDH used to have a section on getting LZO working[1].
One thing I see in those docs that I don't see you doing an equivalent of is setting LD_LIBRARY_PATH to point to the LZO libraries. They also specify io.compression.codec.lzo.class as a command line argument for the service startup rather than putting it in core-site.xml, but it's not clear to me if that's necessary. [1]: http://www.cloudera.com/documentation/other/accumulo/1-6-0/PDF/Apache-Accumulo-Installation-Guide.pdf On Tue, Feb 7, 2017 at 5:18 AM, Massimilian Mattetti <[email protected]> wrote: > Hi all, > > I got stuck trying to enable the LZO compression on a table. I installed the > native-lzo library on each tablet server (sudo apt-get install liblzo2-dev) > and set the compression codecs libraries in the Hadoop core-site.xml: > <property> > <name>io.compression.codecs</name> > <value>org.apache.hadoop.io.compress.GzipCodec, > org.apache.hadoop.io.compress.DefaultCodec, > org.apache.hadoop.io.compress.BZip2Codec, > com.hadoop.compression.lzo.LzoCodec, > com.hadoop.compression.lzo.LzopCodec > </value> > </property> > <property> > <name>io.compression.codec.lzo.class</name> > <value>com.hadoop.compression.lzo.LzoCodec</value> > </property> > > I also added the path to the library to the general.classpaths property of > accumulo-site.xml: > <property> > <name>general.classpaths</name> > <value> > .... > > /usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.5.0.0-1245.jar, > > /usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64/libgplcompression.so, > /lib/x86_64-linux-gnu/liblzo2.so.2 > </value> > </property> > > I tested that the lzo library is working correctly compressing a file via a > MapReduce Job, but when I asked Accumulo to compact the table on which I > enabled the LZO compression I got this error on each tablet server: > MajC Unexpected exception, extent = 1z; ? ??????????<; ? ??????????. > java.lang.RuntimeException: native-lzo library not available > at > com.hadoop.compression.lzo.LzoCodec.getCompressorType(LzoCodec.java:155) > at > org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150) > at > org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:165) > at > org.apache.accumulo.core.file.rfile.bcfile.Compression$Algorithm.getCompressor(Compression.java:502) > at > org.apache.accumulo.core.file.rfile.bcfile.BCFile$Writer$WBlockState.<init>(BCFile.java:192) > at > org.apache.accumulo.core.file.rfile.bcfile.BCFile$Writer.prepareDataBlock(BCFile.java:487) > at > org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile$Writer.prepareDataBlock(CachableBlockFile.java:89) > at > org.apache.accumulo.core.file.rfile.RFile$LocalityGroupWriter.append(RFile.java:429) > at > org.apache.accumulo.core.file.rfile.RFile$Writer.append(RFile.java:596) > at > org.apache.accumulo.core.file.BloomFilterLayer$Writer.append(BloomFilterLayer.java:141) > at > org.apache.accumulo.tserver.tablet.Compactor.compactLocalityGroup(Compactor.java:367) > at > org.apache.accumulo.tserver.tablet.Compactor.call(Compactor.java:223) > at > org.apache.accumulo.tserver.tablet.Tablet._majorCompact(Tablet.java:1850) > at > org.apache.accumulo.tserver.tablet.Tablet.majorCompact(Tablet.java:1967) > at > org.apache.accumulo.tserver.tablet.CompactionRunner.run(CompactionRunner.java:44) > at > org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at > org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35) > at java.lang.Thread.run(Thread.java:745) > What am I missing? > Thanks > > Regards, > Max > -- busbey
