Github user CodingCat commented on a diff in the pull request:
https://github.com/apache/spark/pull/4063#discussion_r23058033
--- Diff:
core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala
---
@@ -52,6 +52,16 @@ private[spark] class CoarseGrainedExecutorBackend(
driver ! RegisterExecutor(executorId, hostPort, cores)
context.system.eventStream.subscribe(self,
classOf[RemotingLifecycleEvent])
}
+
+ override def isValidMessage(msg: Any) = msg match {
+ case disEvt: DisassociatedEvent =>
+ if (disEvt.remoteAddress == driver.anchorPath.address) {
--- End diff --
no...it's hard to get a universal condition to check whether
DisassociatedEvent is valid or not
e.g. for worker, the condition is "x.remoteAddress == masterAddress"
for master, x.remoteAddress is in driverAddresses || x.remoteAddress is in
workerAddresses
In current implementation, Master/Worker/CoarseGrainedSchedulerBackend
maintain in-memory data structures, which are luckily related to
x.remoteAddress. So even they received DisassociatedEvent from irrelevant
remoteAddresses, they just ignore that by running
"RemoteAddress-Related-Map.get(remoteAddress).foreach{...}"
ExecutorBackend does not get that luck, so we have to check explicitly in
the way proposed in this PR....
I made it as an abstract method in case we will introduce more actor-based
components in future so that we can extend that
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
