Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22004#discussion_r207753917 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2369,39 +2369,12 @@ class DAGSchedulerSuite extends SparkFunSuite with LocalSparkContext with TimeLi assert(scheduler.getShuffleDependencies(rddE) === Set(shuffleDepA, shuffleDepC)) } - test("SPARK-17644: After one stage is aborted for too many failed attempts, subsequent stages" + + test("SPARK-17644: After one stage is aborted for too many failed attempts, subsequent stages " + "still behave correctly on fetch failures") { - // Runs a job that always encounters a fetch failure, so should eventually be aborted --- End diff -- Yeah you've said it more correctly, it's really the body of the lambda capturing references, and it just happens to be to an enclosing class of the test method in this case. This is the kind of the I wonder if we *do* have to do, like in the old closure cleaner. Because LMF closures capture far less to begin with, it's much less of an issue. I also remember that closure cleaning such things got dicey because it's not clear when it's just OK to null some object's field. It may not be possible to do reasonably.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org