[GitHub] [spark] zhengruifeng commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

zhengruifeng commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-775778416 ``` scala> spark.sql("CREATE TABLE t (key bigint, value string) USING parquet") res0: org.apache.spark.sql.DataFrame = [] scala> spark.sql("SELECT COUNT(*) FROM

[GitHub] [spark] zhengruifeng commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

zhengruifeng commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-775732036 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] zhengruifeng commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

zhengruifeng commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-773112265 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] zhengruifeng commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

zhengruifeng commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-773112511 related to https://github.com/apache/spark/pull/31409 This is an automated message from the Apache Git

[GitHub] [spark] zhengruifeng commented on pull request #31468: [SPARK-34353][SQL] CollectLimitExec avoid shuffle if input rdd has single partition

zhengruifeng commented on pull request #31468: URL: https://github.com/apache/spark/pull/31468#issuecomment-773112265 > We may have more operators that adding shuffle in the doExecute method instead of the planner @cloud-fan shuffle only is directly added in the `doExecute` method