Hi there
I am new to flume, and trying to use flume to tail a logfile and push the
messages via flume to sparkstreaming. However , I got an exception like below:
16/08/26 15:55:24 ERROR flume.SinkRunner: Unable to deliver event. Exception
follows.
org.apache.flume.EventDeliveryException: Failed to send events
at
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.flume.FlumeException: NettyAvroRpcClient { host:
master60, port: 31002 }: RPC connection error
at
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:182)
at
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:121)
at
org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:638)
at
org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:89)
at org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:127)
at
org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:211)
at
org.apache.flume.sink.AbstractRpcSink.verifyConnection(AbstractRpcSink.java:272)
at
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:349)
... 3 more
Caused by: java.io.IOException: Error connecting to master60/10.0.10.60:31002
at
org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
at
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:168)
... 10 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:496)
at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:452)
at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:365)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Here is my agent config in the conf file
# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444
# Describe the sink
a1.sinks.k1.type = avro
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
a1.sinks.k1.hostname = master60
a1.sinks.k1.port = 31002
I don't know why my flume sink can not bind to the port 31002 which I specified.
Any idea will be appreciated.
San.Luo
