[
https://issues.apache.org/jira/browse/FLUME-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228280#comment-13228280
]
fang fang chen commented on FLUME-1025:
---------------------------------------
Root cause:
In flume-0.9.4, "flume.node.http.autofindport" does not work because flume
agent use com.cloudera.util.InternalHttpServer which did work based on
"autofindport" value. This is different with flume-0.9.1(use StatusHttpServer):
public void start() throws IOException {
try {
while (true) {
try {
LOG.info("Status server available at http://"; + NetUtils.localhost()
+ ":" + listener.getPort());
webServer.start();
break;
} catch (MultiException ex) {
// if the multi exception contains ONLY a bind exception,
// then try the next port number.
boolean needNewPort = false;
if (ex.size() == 1) {
Exception sub = ex.getException(0);
if (sub instanceof java.net.BindException) {
if (!findPort)
throw sub; // java.net.BindException
needNewPort = true;
}
}
if (!needNewPort)
throw ex;
listener.setPort(listener.getPort() + 1);
}
}
} catch (IOException ie) {
throw ie;
} catch (Exception e) {
IOException ie = new IOException("Problem starting http server");
ie.initCause(e);
throw ie;
}
}
> [flume-0.9.4]: alwayse encountered "Address already in use" during start more
> than 1 agent in one node
> ------------------------------------------------------------------------------------------------------
>
> Key: FLUME-1025
> URL: https://issues.apache.org/jira/browse/FLUME-1025
> Project: Flume
> Issue Type: Bug
> Components: Shell
> Affects Versions: v0.9.4
> Reporter: fang fang chen
> Priority: Blocker
>
> This problem did not exists in flume-0.9.1. The following steps works in
> flume-0.9.1 but not in 0.9.4.
> Steps:
> 1. flume node information:
> node1 (name node, flume master)
> node2 (data node, flume node)
> node3 (data node, flume node)
> 2. try to start an agent 'host1' on node3
> ./flume node no_watch host1
> There are exception:
> 2012-03-09 09:58:04,603 [main] INFO mortbay.log: jetty-6.1.26
> 2012-03-09 09:58:04,635 [main] INFO mortbay.log: Extract /fl
> ume/webapps/flumeagent.war to
> /tmp/Jetty_0_0_0_0_35862_flumeagent.war__flumeagen
> t__4cgrz3/webapp
> 2012-03-09 09:58:04,917 [main] WARN mortbay.log: failed
> SelectChannelConnector@0
> .0.0.0:35862
> java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind(Native Method)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:
> 149)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:71)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnec
> tor.java:216)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.doStart(SelectChannelCon
> nector.java:315)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:
> 50)
> at org.mortbay.jetty.Server.doStart(Server.java:235)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:
> 50)
> at
> com.cloudera.util.InternalHttpServer.start(InternalHttpServer.java:18
> 8)
> at com.cloudera.flume.agent.FlumeNode.start(FlumeNode.java:275)
> at com.cloudera.flume.agent.FlumeNode.setup(FlumeNode.java:504)
> at com.cloudera.flume.agent.FlumeNode.main(FlumeNode.java:665)
> 2012-03-09 09:58:04,918 [main] WARN mortbay.log: failed Server@4a774a77
> java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind(Native Method)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:
> 149)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:71)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnec
> tor.java:216)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.doStart(SelectChannelCon
> nector.java:315)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:
> 50)
> at org.mortbay.jetty.Server.doStart(Server.java:235)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:
> 50)
> at
> com.cloudera.util.InternalHttpServer.start(InternalHttpServer.java:18
> 8)
> at com.cloudera.flume.agent.FlumeNode.start(FlumeNode.java:275)
> at com.cloudera.flume.agent.FlumeNode.setup(FlumeNode.java:504)
> at com.cloudera.flume.agent.FlumeNode.main(FlumeNode.java:665)
> 2012-03-09 09:58:04,918 [main] WARN util.InternalHttpServer: Caught exception
> du
> ring HTTP server start.
> java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind(Native Method)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:
> 149)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:71)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnec
> tor.java:216)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.doStart(SelectChannelCon
> nector.java:315)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:
> 50)
> at org.mortbay.jetty.Server.doStart(Server.java:235)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:
> 50)
> at
> com.cloudera.util.InternalHttpServer.start(InternalHttpServer.java:18
> 8)
> at com.cloudera.flume.agent.FlumeNode.start(FlumeNode.java:275)
> at com.cloudera.flume.agent.FlumeNode.setup(FlumeNode.java:504)
> at com.cloudera.flume.agent.FlumeNode.main(FlumeNode.java:665)
> 2012-03-09 09:58:04,918 [main] ERROR agent.FlumeNode: Unexpected
> exception/error
> thrown! Unable to start HTTP server
> com.cloudera.util.InternalHttpServer$InternalHttpServerException: Unable to
> star
> t HTTP server
> at
> com.cloudera.util.InternalHttpServer.start(InternalHttpServer.java:19
> 4)
> at com.cloudera.flume.agent.FlumeNode.start(FlumeNode.java:275)
> at com.cloudera.flume.agent.FlumeNode.setup(FlumeNode.java:504)
> at com.cloudera.flume.agent.FlumeNode.main(FlumeNode.java:665)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind(Native Method)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:
> 149)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:71)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnec
> tor.java:216)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.doStart(SelectChannelCon
> nector.java:315)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:
> 50)
> at org.mortbay.jetty.Server.doStart(Server.java:235)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:
> 50)
> at
> com.cloudera.util.InternalHttpServer.start(InternalHttpServer.java:18
> 8)
> ... 3 more
> 2012-03-09 09:58:04,949 [Heartbeat] INFO agent.MultiMasterRPC: No active
> master
> RPC connection
> 2012-03-09 09:58:05,041 [main] INFO agent.LogicalNodeManager: creating new
> logic
> al node node3
> 2012-03-09 09:58:05,045 [main] WARN agent.FlumeNode: Flume is using Hadoop
> core
> 0.20.2 which does not support Security / Authentication:
> org.apache.hadoop.secur
> ity.UserGroupInformation.isSecurityEnabled()
> 2012-03-09 09:58:05,139 [Heartbeat] INFO agent.ThriftMasterRPC: Connected to
> mas
> ter at node1:35872
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
