uros-b commented on code in PR #58437:
URL: https://github.com/apache/spark/pull/58437#discussion_r3904591194
##########
core/src/main/scala/org/apache/spark/MapOutputTracker.scala:
##########
@@ -1044,19 +1059,41 @@ private[spark] class MapOutputTrackerMaster(
/**
* Removes all shuffle outputs associated with this host. Note that this
will also remove
* outputs which are served by an external shuffle server (if one exists).
+ *
+ * When `skipReliablyStored` is true (executor/worker loss rather than a
fetch failure),
+ * shuffles whose output is reliably stored off-executor are left intact,
since losing the host
+ * does not lose their output.
*/
- def removeOutputsOnHost(host: String): Unit = {
- shuffleStatuses.valuesIterator.foreach { _.removeOutputsOnHost(host) }
+ def removeOutputsOnHost(host: String): Unit =
+ removeOutputsOnHost(host, skipReliablyStored = false)
+
+ def removeOutputsOnHost(host: String, skipReliablyStored: Boolean): Unit = {
+ shuffleStatuses.valuesIterator.foreach { status =>
+ if (!(skipReliablyStored && status.isReliablyStored)) {
+ status.removeOutputsOnHost(host)
+ }
+ }
incrementEpoch()
Review Comment:
removeOutputsOnHost/removeOutputsOnExecutor unconditionally call
incrementEpoch() even when skipReliablyStored = true and every shuffle is
reliably stored (nothing was actually removed from the tracker). The epoch bump
causes every executor worker to invalidate its cached map-output info and issue
a round-trip to the driver on next use, even though the tracker state is
unchanged. In the mixed-configuration scenario this optimization targets
(remote shuffle service with frequent spot-instance turnover) this produces
spurious cache churn on every executor loss. Fix: track whether any status's
removal call changed state and guard incrementEpoch() accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
