[
https://issues.apache.org/jira/browse/CAMEL-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409283#comment-13409283
]
Claus Ibsen commented on CAMEL-5433:
------------------------------------
And example of thread dump during that retry logic
{code}
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <7fb6998a8> (a sun.nio.ch.Util$2)
- locked <7fb699898> (a java.util.Collections$UnmodifiableSet)
- locked <7fb699690> (a sun.nio.ch.KQueueSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at
org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:332)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:203)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:656)
at
org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:434)
- locked <7fb679d80> (a org.apache.hadoop.ipc.Client$Connection)
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:560)
- locked <7fb679d80> (a org.apache.hadoop.ipc.Client$Connection)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:184)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1202)
at org.apache.hadoop.ipc.Client.call(Client.java:1046)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at $Proxy87.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
at
org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
at org.apache.camel.component.hdfs.HdfsInfo.<init>(HdfsInfo.java:34)
at
org.apache.camel.component.hdfs.HdfsOutputStream.createOutputStream(HdfsOutputStream.java:48)
at
org.apache.camel.component.hdfs.HdfsProducer.doStart(HdfsProducer.java:93)
at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:60)
at
org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:62)
at
org.apache.camel.impl.ProducerCache.doGetProducer(ProducerCache.java:387)
{code}
> camel-hdfs - Allow to startup async as hdfs file system has hardcoded 15 min
> retry in case no connection
> --------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-5433
> URL: https://issues.apache.org/jira/browse/CAMEL-5433
> Project: Camel
> Issue Type: Improvement
> Components: camel-hdfs
> Affects Versions: 2.10.0
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Fix For: 2.10.1, 2.11
>
>
> When start a Camel app with camel-hdfs (either as producer or consumer), then
> getting a hdfs file system object may have Hadoop block for up till 15
> minutes where it tries to obtain a connection to the cluster.
> This makes the app appear as blocking and people dont want their apps to
> block for 15 min on startup.
> We should let the hadoop hdfs connection happing in an async thread, so the
> app can startup. Configurable so you can turn back the old behavior.
> Hadoop dosn't allow to configure a lower timeout etc this as its hardcoded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
