[jira] [Commented] (SPARK-35989) Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled
[ https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382925#comment-17382925 ] Dongjoon Hyun commented on SPARK-35989: --- This is assigned to [~ulysses]. > Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled > -- > > Key: SPARK-35989 > URL: https://issues.apache.org/jira/browse/SPARK-35989 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.2.0 >Reporter: XiDuo You >Assignee: XiDuo You >Priority: Major > Fix For: 3.2.0 > > > The shuffle origin is `REPARTITION_BY_NUM` if user specify an exact partition > number with repartition, then we should not do any change of the number. That > said, the shuffle output partitioning number should be always same with user > expected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-35989) Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled
[ https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373408#comment-17373408 ] Apache Spark commented on SPARK-35989: -- User 'ulysses-you' has created a pull request for this issue: https://github.com/apache/spark/pull/33188 > Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled > -- > > Key: SPARK-35989 > URL: https://issues.apache.org/jira/browse/SPARK-35989 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.2.0 >Reporter: XiDuo You >Priority: Major > > The shuffle origin is `REPARTITION_BY_NUM` if user specify an exact partition > number with repartition, then we should not do any change of the number. That > said, the shuffle output partitioning number should be always same with user > expected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
