Re: Flume ERROR

Anandkumar Lakshmanan Tue, 11 Aug 2015 06:55:38 -0700

Hi,


Yes it is admin related work.

You can flush the firewall rules by running this command.

Sudo iptables -F

or you can open the specific port which flume requires to do its process.

Thanks
Anand.

On 08/07/2015 07:50 PM, Nikhil Gs wrote:


Thanks AnandKumar.

Flushing the firewall rules is admin related work right?

Redhat 4.7 is our OS.

Linux version 2.6.32-504.3.3.el6.x86_64([email protected]<mailto:[email protected]>) (gcc version4.4.7 20120313 (Red Hat 4.4.7-9) (GCC) ) #1 SMP Fri Dec 12 16:05:43EST 2014


Regards,
Nik.

On Fri, Aug 7, 2015 at 9:21 AM, Anandkumar Lakshmanan<[email protected] <mailto:[email protected]>> wrote:



    Hi,

    Flush the firewall rules and start flume.

    What OS are you using?

    Thanks
    Anand.


    On 08/07/2015 07:24 PM, Nikhil Gs wrote:

    Hello AnandKumar,

    The issue is again repeating. Yesterday the files were deleting
    from spool but the data was not available in the destination
    Hbase tables, but now this morning what I have noticed is the log
    is same.

    Earlier, as you have suggested, "verify the firewall settings. It
    blocks the connection it seems."
    Can you be more specific, I mean how can verify the firewall
    settings. Any suggestion?

    Thanks in advance.

    8:45:01.040 AM      ERROR   org.apa
    che.flume.SinkRunner        
    Unable to deliver event. Exception follows.
    org.apache.flume.EventDeliveryException: Failed to send events
        at 
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392)
        at 
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:745)
    Caused by: org.apache.flume.FlumeException: NettyAvroRpcClient { 
host:sdldalplhdw02.suddenlink.cequel3.com  
<http://sdldalplhdw02.suddenlink.cequel3.com>, port: 40002 }: RPC connection 
error
        at 
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:182)
        at 
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:121)
        at 
org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:638)
        at 
org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:89)
        at org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:127)
        at 
org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:211)
        at 
org.apache.flume.sink.AbstractRpcSink.verifyConnection(AbstractRpcSink.java:272)
        at 
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:349)
        ... 3 more
    Caused by: java.io.IOException: Error connecting 
tosdldalplhdw02.suddenlink.cequel3.com/10.48.210.244:40002  
<http://sdldalplhdw02.suddenlink.cequel3.com/10.48.210.244:40002>
        at 
org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:292)
        at 
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:206)
        at 
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:155)
        at 
org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:168)
        ... 10 more
    Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:148)
        at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:104)
        at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:78)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
        at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:41)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        ... 1 more


    Regards,
    Nik.

    On Fri, Aug 7, 2015 at 8:28 AM, Nikhil Gs
    <[email protected] <mailto:[email protected]>> wrote:

        Hello Anandkumar,

        Thank you for your time and also for the reply.

        Now, flume acts differently. I don't see any log errors, in
        fact my log now looks as below;

        2015-08-06 22:51:14,426 INFO