[GitHub] [spark] cloud-fan commented on a change in pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

cloud-fan commented on a change in pull request #28109: URL: https://github.com/apache/spark/pull/28109#discussion_r415645175 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CustomShuffleReaderExec.scala ## @@ -121,12 +122,22 @@ case class CustomS

[GitHub] [spark] cloud-fan commented on a change in pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

cloud-fan commented on a change in pull request #28109: URL: https://github.com/apache/spark/pull/28109#discussion_r415519376 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/metric/SQLMetrics.scala ## @@ -87,6 +87,7 @@ object SQLMetrics { private val TI

[GitHub] [spark] cloud-fan commented on a change in pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

cloud-fan commented on a change in pull request #28109: URL: https://github.com/apache/spark/pull/28109#discussion_r415519482 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CustomShuffleReaderExec.scala ## @@ -153,7 +170,9 @@ case class CustomShu

[GitHub] [spark] cloud-fan commented on a change in pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

cloud-fan commented on a change in pull request #28109: URL: https://github.com/apache/spark/pull/28109#discussion_r415519133 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CustomShuffleReaderExec.scala ## @@ -112,6 +113,16 @@ case class CustomSh