[
https://issues.apache.org/jira/browse/FLUME-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15289528#comment-15289528
]
Jarek Jarcec Cecho commented on FLUME-2905:
-------------------------------------------
My local build is failing with:
{code}
Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 6.481 sec <<<
FAILURE!
testSourceStoppedOnFlumeException(org.apache.flume.source.TestNetcatSource)
Time elapsed: 6 sec <<< ERROR!
java.lang.IllegalStateException: No channel processor configured
at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.source.AbstractSource.start(AbstractSource.java:45)
at org.apache.flume.source.NetcatSource.start(NetcatSource.java:192)
at
org.apache.flume.source.TestNetcatSource.startSource(TestNetcatSource.java:344)
at
org.apache.flume.source.TestNetcatSource.testSourceStoppedOnFlumeException(TestNetcatSource.java:314)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
{code}
Wondering if you saw that in your environment as well?
> NetcatSource - Socket not closed when an exception is encountered during
> start() leading to file descriptor leaks
> -----------------------------------------------------------------------------------------------------------------
>
> Key: FLUME-2905
> URL: https://issues.apache.org/jira/browse/FLUME-2905
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.6.0
> Reporter: Siddharth Ahuja
> Assignee: Siddharth Ahuja
> Attachments: FLUME-2905-0.patch, FLUME-2905-1.patch
>
>
> During the flume agent start-up, the flume configuration containing the
> NetcatSource is parsed and the source's start() is called. If there is an
> issue while binding the channel's socket to a local address to configure the
> socket to listen for connections following exception is thrown but the socket
> open just before is not closed.
> {code}
> 2016-05-01 03:04:37,273 ERROR org.apache.flume.lifecycle.LifecycleSupervisor:
> Unable to start EventDrivenSourceRunner: {
> source:org.apache.flume.source.NetcatSource{name:src-1,state:IDLE} } -
> Exception follows.
> org.apache.flume.FlumeException: java.net.BindException: Address already in
> use
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:173)
> at
> org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
> at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
> at org.apache.flume.source.NetcatSource.start(NetcatSource.java:167)
> ... 9 more
> {code}
> The source's start() is then called again leading to another socket being
> opened but not closed and so on. This leads to file descriptor (socket) leaks.
> This can be easily reproduced as follows:
> 1. Set Netcat as the source in flume agent configuration.
> 2. Set the bind port for the netcat source to a port which is already in use.
> e.g. in my case I used 50010 which is the port for DataNode's XCeiver
> Protocol in use by the HDFS service.
> 3. Start flume agent and perform "lsof -p <flume_process_id> | wc -l". Notice
> the file descriptors keep on growing due to socket leaks with errors like:
> "can't identify protocol".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
