[
https://issues.apache.org/jira/browse/FLUME-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852692#comment-15852692
]
qstar commented on FLUME-1875:
------------------------------
I would think that is not the problem, I set to lzo and write to hdfs
successfully.
1.flume conf: output.sinks.sink-hdfs.hdfs.codeC = lzo
2.then copy hadoop core-site.xml to flume/conf which must contain two
properties below:
<property>
<name>io.compression.codecs</name>
<value>org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DeflateCodec,com.hadoop.compression.lzo
.LzoCodec</value>
</property>
<property>
<name>io.compression.codec.lzo.class</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>
3.make sure hadoop-lzo-*.jar is in hadoop/lib
4.then maybe restart flume, it'll work
> Setting lzo in hdfs.codeC throws IllegalArgumentException
> ---------------------------------------------------------
>
> Key: FLUME-1875
> URL: https://issues.apache.org/jira/browse/FLUME-1875
> Project: Flume
> Issue Type: Bug
> Reporter: Hari Shreedharan
>
> The User guide says lzo is a valid compression option, but using that throws:
> {code}
> 13/01/24 16:09:55 ERROR node.AbstractConfigurationProvider: Sink sink1 has
> been removed due to an error during configuration
> java.lang.IllegalArgumentException: Unsupported compression codec lzo.
> Please choose from: [None, BZip2Codec, DefaultCodec, DeflateCodec, GzipCodec,
> Lz4Codec, SnappyCodec]
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.getCodec(HDFSEventSink.java:317)
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.configure(HDFSEventSink.java:222)
> at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
> at
> org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:418)
> at
> org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:103)
> at
> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {code}
> Maybe it should be lz4?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
