jack li created FLUME-1379:
------------------------------
Summary: source's type is avro and sink's type is hdfs but sink
don't to work
Key: FLUME-1379
URL: https://issues.apache.org/jira/browse/FLUME-1379
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: v1.1.0, v1.3.0
Environment: rhel5.7 ,jdk1.6,yum installed flume-ng
1.1.0+120-1.cdh4.0.0 ,mvn install 1.3.
Reporter: jack li
Fix For: v1.3.0
hello everyone!
1, source's type is exec and sink's type is hdfs .the event can write to the
hdfs.But when I change the source's type is avro ,and no change sink's type,no
event write to hdfs.
2, flume.conf
#List sources, sinks and channels in the agent
weblog-agent.sources = tail
weblog-agent.sinks = avro-forward-sink01
weblog-agent.channels = jdbc-channel01
#webblog-agent sources config
weblog-agent.sources.tail.type = avro
weblog-agent.sources.tail.port = 41414
weblog-agent.sources.tail.bind = 0.0.0.0
#avro sink properties
weblog-agent.sinks.avro-forward-sink01.channel = jdbc-channel01
weblog-agent.sinks.avro-forward-sink01.type = hdfs
weblog-agent.sinks.avro-forward-sink01.hdfs.path =
hdfs://hadoop-01:8020/flume/web
weblog-agent.sinks.avro-forward-sink01.hdfs.rollInterval = 120
#channels config
weblog-agent.channels.jdbc-channel01.type = memory
3,flume.log
2012-07-18 21:03:18,731 INFO lifecycle.LifecycleSupervisor: Starting lifecycle
supervisor 1
2012-07-18 21:03:18,734 INFO node.FlumeNode: Flume node starting - weblog-agent
2012-07-18 21:03:18,738 INFO nodemanager.DefaultLogicalNodeManager: Node
manager starting
2012-07-18 21:03:18,738 INFO lifecycle.LifecycleSupervisor: Starting lifecycle
supervisor 10
2012-07-18 21:03:18,738 INFO properties.PropertiesFileConfigurationProvider:
Configuration provider starting
2012-07-18 21:03:18,740 INFO properties.PropertiesFileConfigurationProvider:
Reloading configuration file:flume.conf
2012-07-18 21:03:18,748 INFO conf.FlumeConfiguration: Added sinks:
avro-forward-sink01 Agent: weblog-agent
2012-07-18 21:03:18,748 INFO conf.FlumeConfiguration:
Processing:avro-forward-sink01
2012-07-18 21:03:18,748 INFO conf.FlumeConfiguration:
Processing:avro-forward-sink01
2012-07-18 21:03:18,748 INFO conf.FlumeConfiguration:
Processing:avro-forward-sink01
2012-07-18 21:03:18,748 INFO conf.FlumeConfiguration:
Processing:avro-forward-sink01
2012-07-18 21:03:18,748 INFO conf.FlumeConfiguration:
Processing:avro-forward-sink01
2012-07-18 21:03:18,784 INFO conf.FlumeConfiguration: Post-validation flume
configuration contains configuration for agents: [weblog-agent]
2012-07-18 21:03:18,785 INFO properties.PropertiesFileConfigurationProvider:
Creating channels
2012-07-18 21:03:18,857 INFO instrumentation.MonitoredCounterGroup: Monitoried
counter group for type: CHANNEL, name: jdbc-channel01, registered successfully.
2012-07-18 21:03:18,857 INFO properties.PropertiesFileConfigurationProvider:
created channel jdbc-channel01
2012-07-18 21:03:18,872 INFO instrumentation.MonitoredCounterGroup: Monitoried
counter group for type: SOURCE, name: tail, registered successfully.
2012-07-18 21:03:18,886 INFO sink.DefaultSinkFactory: Creating instance of
sink: avro-forward-sink01, type: hdfs
2012-07-18 21:03:19,142 INFO hdfs.HDFSEventSink: Hadoop Security enabled: false
2012-07-18 21:03:19,144 INFO instrumentation.MonitoredCounterGroup: Monitoried
counter group for type: SINK, name: avro-forward-sink01, registered
successfully.
2012-07-18 21:03:19,146 INFO nodemanager.DefaultLogicalNodeManager: Starting
new configuration:{ sourceRunners:{tail=EventDrivenSourceRunner: { source:Avro
source tail: { bindAddress: localhost, port: 41414 } }}
sinkRunners:{avro-forward-sink01=SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@57a7ddcf counterGroup:{
name:null counters:{} } }}
channels:{jdbc-channel01=org.apache.flume.channel.MemoryChannel@4dd36dfe} }
2012-07-18 21:03:19,146 INFO nodemanager.DefaultLogicalNodeManager: Starting
Channel jdbc-channel01
2012-07-18 21:03:19,147 INFO nodemanager.DefaultLogicalNodeManager: Waiting for
channel: jdbc-channel01 to start. Sleeping for 500 ms
2012-07-18 21:03:19,147 INFO instrumentation.MonitoredCounterGroup: Component
type: CHANNEL, name: jdbc-channel01 started
2012-07-18 21:03:19,648 INFO nodemanager.DefaultLogicalNodeManager: Starting
Sink avro-forward-sink01
2012-07-18 21:03:19,649 INFO nodemanager.DefaultLogicalNodeManager: Starting
Source tail
2012-07-18 21:03:19,650 INFO source.AvroSource: Starting Avro source tail: {
bindAddress: localhost, port: 41414 }...
2012-07-18 21:03:19,656 INFO instrumentation.MonitoredCounterGroup: Component
type: SINK, name: avro-forward-sink01 started
2012-07-18 21:03:20,096 INFO instrumentation.MonitoredCounterGroup: Component
type: SOURCE, name: tail started
2012-07-18 21:03:20,096 INFO source.AvroSource: Avro source tail started.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
