Stephan Ewen created FLINK-8125:
-----------------------------------
Summary: Support limiting the number of open FileSystem connections
Key: FLINK-8125
URL: https://issues.apache.org/jira/browse/FLINK-8125
Project: Flink
Issue Type: Improvement
Components: Core
Reporter: Stephan Ewen
Assignee: Stephan Ewen
Fix For: 1.4.1, 1.5.0
We need a way to limit the number of streams that Flink FileSystems
concurrently open.
For example, for very small HDFS clusters with few RPC handlers, a large Flink
job trying to build up many connections during a checkpoint causes failures due
to rejected connections.
I propose to add a file system that can wrap another existing file system The
file system may track the progress of streams and close streams that have been
inactive for too long, to avoid locked streams of taking up the complete pool.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
