Hi, Isn't it possible that your spark streaming application is not running when flume tries to send events to it? That might be the reason of flume not being able to connect to the port 31002. According to the Spark streaming + Flume integration guide ( http://spark.apache.org/docs/latest/streaming-flume-integration.html) when you use the push-based approach "the streaming application needs to be up, with the receiver scheduled and listening on the chosen port, for Flume to be able push data."
On the other hand you might want to consider using the more reliable pull-based approach. Regards, Denes On Mon, Aug 29, 2016 at 4:02 AM 罗辉 <[email protected]> wrote: > Hi Iain, > > telnet is ok, if I switch the sink to logger. Please check below log: > > source: > > [hadoop@master60 flume]$ telnet localhost 44444 > > Trying ::1... > > telnet: connect to address ::1: Connection refused > > Trying 127.0.0.1... > > Connected to localhost. > > Escape character is '^]'. > > abc > > OK > > abc > > OK > > gggd > > OK > > > > > > Sink: > > 16/08/29 09:57:16 INFO node.Application: Starting Sink k1 > > 16/08/29 09:57:16 INFO node.Application: Starting Source r1 > > 16/08/29 09:57:16 INFO source.NetcatSource: Source starting > > 16/08/29 09:57:16 INFO source.NetcatSource: Created > serverSocket:sun.nio.ch.ServerSocketChannelImpl[/127.0.0.1:44444] > > 16/08/29 09:58:10 INFO sink.LoggerSink: Event: { headers:{} body: 61 62 63 > 0D abc. } > > 16/08/29 09:58:14 INFO sink.LoggerSink: Event: { headers:{} body: 61 62 63 > 0D abc. } > > 16/08/29 09:58:16 INFO sink.LoggerSink: Event: { headers:{} body: 67 67 67 > 64 0D gggd. } > > > > Seems it is ok when the sink is logger , but when I change the sink type > from logger to avro. It failed with previous exception. > > > > > > > > *发件人:* iain wright [mailto:[email protected]] > *发送时间:* 2016年8月27日 0:13 > *收件人:* [email protected] > *主题:* Re: Flume sink exeption:org.apache.flume.EventDeliveryException: > Failed to send events > > > > Can you telnet master60 31002 from the server the flume agent is running > on? > > > Sent from my iPhone > > > On Aug 26, 2016, at 1:00 AM, 罗辉 <[email protected]> wrote: > > 31002 > >
