Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2076#issuecomment-52908740 @srowen yes! Not only in "StorageTab", "ExectutorTab" may also lose some rdd-infos which have been overwritten by following rdd in a same task. "StorageTab": when multiple stages run simultaneously, completed stage will remove rdd-info which belong to other stages which are still running. "ExectutorTab": In a dependency chain of rdds, task can only update the latest rdd info. Like the following example: val r1 = sc.paralize(..).cache() val r2 = r1.map(...).cache() val n = r2.count() Currently, "ExecutorTab" show a wrong info about the "Memory Used", it only shows the usage of r2, but not r1 and r2.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org