Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21754#discussion_r207192928 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -117,10 +118,6 @@ class ExchangeCoordinator( */ def estimatePartitionStartIndices( mapOutputStatistics: Array[MapOutputStatistics]): Array[Int] = { - // If we have mapOutputStatistics.length < numExchange, it is because we do not submit - // a stage when the number of partitions of this dependency is 0. - assert(mapOutputStatistics.length <= numExchanges) --- End diff -- To pass the existing tests for `estimatePartitionStartIndices`, e.g., https://github.com/apache/spark/pull/21754/files#diff-3cd46a3f60c5352282bd3f2c9efff7fcR61. As another approach, we might add a dummy `ShuffleExchange` in `ExhcnageCoordinator` there. But, building `ShuffleExchange` is troublesome in the test suite without `SharedSparkSession`.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org