Help Required - Unable to run spark-submit on YARN client mode

[Debabrata Ghosh]

Hi Everyone,
I have been trying to run spark-shell in YARN client mode, but am getting
lot of ClosedChannelException errors, however the program works fine on
local mode.  I am using spark 2.2.0 build for Hadoop 2.7.3.  If you are
familiar with this error, please can you help with the possible resolution.

Any help would be greatly appreciated!

Here is the error message:

18/05/08 00:01:18 ERROR TransportClient: Failed to send RPC
7905321254854295784 to /9.30.94.43:60220:
java.nio.channels.ClosedChannelException
java.nio.channels.ClosedChannelException
        at
io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source)
18/05/08 00:01:18 ERROR YarnSchedulerBackend$YarnSchedulerEndpoint: Sending
RequestExecutors(5,0,Map(),Set()) to AM was unsuccessful
java.io.IOException: Failed to send RPC 7905321254854295784 to /
9.30.94.43:60220: java.nio.channels.ClosedChannelException
        at
org.apache.spark.network.client.TransportClient.lambda$sendRpc$2(TransportClient.java:237)
        at
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
        at
io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481)
        at
io.netty.util.concurrent.DefaultPromise.access$000(DefaultPromise.java:34)
        at
io.netty.util.concurrent.DefaultPromise$1.run(DefaultPromise.java:431)
        at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:446)
        at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
        at
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.channels.ClosedChannelException

Cheers,

Debu