Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22335#discussion_r215360036
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -1549,7 +1549,7 @@ private[spark] class BlockManager(
// TODO: Avoid a linear scan by creating another mapping of RDD.id to
blocks.
logInfo(s"Removing RDD $rddId")
val blocksToRemove =
blockInfoManager.entries.flatMap(_._1.asRDDId).filter(_.rddId == rddId)
- blocksToRemove.foreach { blockId => removeBlock(blockId, tellMaster =
false) }
--- End diff --
I don't think this should be changing. Instead, the `onUnpersistRDD`
callback in the listener should be updating the executor data it's tracking
when the RDD is being unpersisted. That's similar to what I understand was
being done in 2.2.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
