AvroSource is failing on ClosedChannelException
-----------------------------------------------
Key: FLUME-861
URL: https://issues.apache.org/jira/browse/FLUME-861
Project: Flume
Issue Type: Bug
Affects Versions: NG alpha 2
Reporter: Jarek Jarcec Cecho
Attachments: hddev01.log, hddev01.properties, hddev02.log,
hddev02.properties
I've noticed that flume-ng is not reconnecting avro source/sink combination
after restart of the sink part.
Steps to reproduce:
1) Run flume-ng on two nodes
2) Start sending data
3) Turn off sink instance (I used CTRL+C)
4) Turn it on again
5) Avro source part will fail on:
2011-11-25 02:22:23,720 (Thread-1) [WARN -
org.apache.avro.ipc.NettyTransceiver$NettyClientAvroHandler.exceptionCaught(NettyTransceiver.java:257)]
Unexpected exception from downstream.
java.nio.channels.ClosedChannelException
(entire error message is present in hddev01.log)
I've used two centos boxes where first was acting as an agent, reading local
log file using ExecSource. Second machine was acting as a collector, waiting
for input events and storing them on HDFS. Both machines were connected using
AVRO sink+source combination. You can find configuration files for both
machines with their logs as well attached to this JIRA bug.
I've executed both flume-ng instances using following commands:
./bin/flume-ng node --conf conf/ --classpath flume-ng.jar --f
conf/configuration.properties -n hddev01 > hddev01.log 2>&1
./bin/flume-ng node --conf conf/ --classpath flume-ng.jar --f
conf/configuration.properties -n hddev02 > hddev02.log 2>&1
Input file was created using following small bash script (it was executed
after flume-ng was successfully started):
for i in `seq -w 01 180`; do echo $i; echo Yoda-$i >> /var/log/jarcec; sleep
1s; done
Please note that I applied patch from FLUME-858 in order to get DataStream
file type working for this test, however I was observing same behavior without
it as well (e.g. using default sequence file)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
