HeartSaVioR commented on a change in pull request #27010: [SPARK-30313][CORE]
Ensure EndpointRef is available MasterWebUI/WorkerPage
URL: https://github.com/apache/spark/pull/27010#discussion_r362999859
##########
File path: core/src/main/scala/org/apache/spark/rpc/netty/Dispatcher.scala
##########
@@ -68,11 +59,34 @@ private[netty] class Dispatcher(nettyEnv: NettyRpcEnv,
numUsableCores: Int) exte
if (stopped) {
throw new IllegalStateException("RpcEnv has been stopped")
}
- if (endpoints.putIfAbsent(name, getMessageLoop(name, endpoint)) != null)
{
+ if (endpoints.containsKey(name)) {
throw new IllegalArgumentException(s"There is already an RpcEndpoint
called $name")
}
+
+ // This must be done before assigning RpcEndpoint to MessageLoop, as
MessageLoop sets Inbox be
+ // active when registering, and endpointRef must be put into
endpointRefs before onStart is
+ // called.
+ endpointRefs.put(endpoint, endpointRef)
+
+ var messageLoop: MessageLoop = null
+ try {
+ messageLoop = endpoint match {
+ case e: IsolatedRpcEndpoint =>
+ new DedicatedMessageLoop(name, e, this)
+ case _ =>
+ sharedLoop.register(name, endpoint)
+ sharedLoop
+ }
+ endpoints.put(name, messageLoop)
+ } catch {
+ case NonFatal(e) =>
+ endpointRefs.remove(endpoint)
+ if (messageLoop != null &&
messageLoop.isInstanceOf[DedicatedMessageLoop]) {
Review comment:
You're right. I'll remove it.
Btw, do we decide to just ignore leaking thread pool? The previous change I
reverted was required to deal with it, as thread pool shouldn't be initialized
in constructor. I guess it might be yes, as you've mentioned it as "small
thing", but that's only the matter of git rebase so please let me know if we
would want to address it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
