Hi Team,
getting this error if we put sc.stop() in application.. can we remove it from application but i read if yu dont explicitly stop using sc.stop the yarn application will not get registered in history service.. SO what to do ? WARN Dispatcher: Message RemoteProcessDisconnected dropped. java.lang.IllegalStateException: RpcEnv already stopped. at org.apache.spark.rpc.netty.Dispatcher.postMessage(Dispatcher.scala:159) at org.apache.spark.rpc.netty.Dispatcher.postToAll(Dispatcher.scala:109) at org.apache.spark.rpc.netty.NettyRpcHandler.connectionTerminated(NettyRpcEnv.scala:630) at org.apache.spark.network.server.TransportRequestHandler.channelUnregistered(TransportRequestHandler.java:94)