The mechanism to close idle connections already exists. It doesn't
mean you can just use it as is in existing connections. So if you want
to go and fix that, you're going to have to figure out that part. Or
figure out a different solution.
Either way, file a bug so that this is properly tracked.
Yeh monitor is present but for some cases like long running job I found App
master is idle.so it will end up closing the App master’s channel so job
will not be completed.
So needed a mechanism to close only invalid connections .
On Wed, 28 Feb 2018 at 10:54 PM, Marcelo Vanzin
Spark already has code to monitor idle connections and close them.
That's in TransportChannelHandler.java.
If there's anything to do here, it's to allow all users of the
transport library to support the "close idle connections" feature of
that class.
On Wed, Feb 28, 2018 at 9:07 AM,
In case of client mode App Master and Driver are in different JVM process,the
port opened by the Driver is vulnerable for flooding attacks as it is not
closing the IDLE connections.
I am thinking to fix this issue using below mechanism
1.Expose configuration to close the IDLE connections as
SparkSubmit will open the port to communicate with the APP Master and
executors.
This port is not closing the IDLE connections,so it is vulnerable for DOS
attack,I did telnet IP port and this connection is not closed.
In order to fix this I tried to Handle in the *userEventTriggered * of