Thirft server doesn't check if port is free
-------------------------------------------
Key: FLUME-874
URL: https://issues.apache.org/jira/browse/FLUME-874
Project: Flume
Issue Type: Improvement
Components: Sinks+Sources
Environment: RHEL 5
Reporter: Ossi L
When starting a collector with multiple flows / logical nodes configured, there
might be conflict with overlapping ports.
2011-12-08 13:12:00,470 INFO org.mortbay.log: Extract
/usr/lib/flume/webapps/flumeagent.war to
/tmp/Jetty_0_0_0_0_35862_flumeagent.war__flumeagent__4cgrz3/webapp
2011-12-08 13:12:00,680 INFO org.mortbay.log: Started
[email protected]:35862
2011-12-08 13:12:00,680 INFO com.cloudera.util.InternalHttpServer: Server
started on port 35862
2011-12-08 13:12:05,785 INFO com.cloudera.flume.agent.LogicalNodeManager:
Loading node name with FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011'
snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011'
flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink(
"hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-"
)' }
2011-12-08 13:12:05,787 INFO com.cloudera.flume.agent.LogicalNode: Node config
successfully set to FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011'
snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011'
flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink(
"hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-"
)' }
2011-12-08 13:12:05,798 INFO
com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking thread
pool server on port 35862...
org.apache.thrift.transport.TTransportException: Could not create ServerSocket
on address 0.0.0.0/0.0.0.0:35862.
So, first flume start jetty on port 35862 and then it tries to start thrift
server on the same port and it naturally fails.
Same problem will be with master if that server is running multiple logical
nodes / flows.
And of course with any other service which happens to be using ports near above
range.
This can be avoided by configuring http port of nodes and master to be than
35853, but it doesn't fix problem with some other services.
So, thrift should check if the port is free or not and bind only after that, if
port is free.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
