[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r643501449 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -1743,6 +1762,10 @@ private[spark] class DAGScheduler( }

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r643429058 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -1743,6 +1762,10 @@ private[spark] class DAGScheduler( }

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r643429058 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -1743,6 +1762,10 @@ private[spark] class DAGScheduler( }

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r641257596 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -2000,6 +2023,147 @@ private[spark] class DAGScheduler( } }

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r641118134 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -2000,6 +2023,147 @@ private[spark] class DAGScheduler( } }

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r640374790 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -2136,9 +2137,24 @@ private[spark] class DAGScheduler( } }

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r640363202 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -2136,9 +2137,24 @@ private[spark] class DAGScheduler( } }

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r640073914 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -2136,9 +2137,24 @@ private[spark] class DAGScheduler( } }

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r639138475 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -2060,17 +2060,18 @@ private[spark] class DAGScheduler(

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r626227795 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -689,7 +716,7 @@ private[spark] class DAGScheduler( dep

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r626227795 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -689,7 +716,7 @@ private[spark] class DAGScheduler( dep

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r626226635 ## File path: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ## @@ -3393,6 +3406,271 @@ class DAGSchedulerSuite extends

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r626178921 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -254,6 +259,28 @@ private[spark] class DAGScheduler( private val