[ https://issues.apache.org/jira/browse/FLINK-5685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16204695#comment-16204695 ]
Stephan Ewen commented on FLINK-5685: ------------------------------------- Is there any update on this issue, or did increasing the number of file handles solve the issue? We are also updating to a newer akka version for the next release, which may have an impact. > Connection leak in Taskmanager > ------------------------------ > > Key: FLINK-5685 > URL: https://issues.apache.org/jira/browse/FLINK-5685 > Project: Flink > Issue Type: Bug > Components: Distributed Coordination > Reporter: Andrey > Priority: Critical > > Steps to reproduce: > * setup cluster with the following configuration: 1 job manager, 2 task > managers > * job manager starts rejecting connection attempts from task manager. > {code} > 2017-01-30 03:24:42,908 INFO > org.apache.flink.runtime.taskmanager.TaskManager - Trying to > register at JobManager akka.tcp://flink@ip:6123/user/jobmanager (attempt > 4326, timeout: 30 seconds) > 2017-01-30 03:24:42,913 WARN Remoting > - Tried to associate with unreachable remote address > [akka.tcp://flink@ip:6123]. Address is now gated for 5000 ms, all messages to > this > address will be delivered to dead letters. Reason: The remote system has > quarantined this system. No further associations to the remote system are > possible until this system is restarted. > {code} > * task manager tries multiple times. (looks like it doens't close connection > after failure) > * job manager unable to process any messages. In logs: > {code} > 2017-01-30 03:25:12,932 WARN > org.jboss.netty.channel.socket.nio.AbstractNioSelector - Failed to > accept a connection. > java.io.IOException: Too many open files > at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) > at > sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) > at > sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) > at > org.jboss.netty.channel.socket.nio.NioServerBoss.process(NioServerBoss.java:100) > at > org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) > at > org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)