[jira] [Updated] (SPARK-35725) Support repartition expand partitions in AQE

[ https://issues.apache.org/jira/browse/SPARK-35725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35725: -- Summary: Support repartition expand partitions in AQE (was: Support repartition expand partitions)

[jira] [Updated] (SPARK-35725) Support repartition expand partitions

[ https://issues.apache.org/jira/browse/SPARK-35725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35725: -- Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > Support repartition expand

[jira] [Created] (SPARK-35725) Support repartition expand partitions

XiDuo You created SPARK-35725: - Summary: Support repartition expand partitions Key: SPARK-35725 URL: https://issues.apache.org/jira/browse/SPARK-35725 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-35687) PythonUDFSuite move assume into its methods

XiDuo You created SPARK-35687: - Summary: PythonUDFSuite move assume into its methods Key: SPARK-35687 URL: https://issues.apache.org/jira/browse/SPARK-35687 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33832) Add an option in AQE to mitigate skew even if it causes an new shuffle

[ https://issues.apache.org/jira/browse/SPARK-33832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359190#comment-17359190 ] XiDuo You commented on SPARK-33832: --- hi [~ekoifman] I created a PR for this issue, do you mind taking

[jira] [Updated] (SPARK-35675) EnsureRequirements remove shuffle should respect PartitioningCollection

[ https://issues.apache.org/jira/browse/SPARK-35675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35675: -- Description: Currently EnsureRequirements only check if child has semantic equal HashPartitioning

[jira] [Created] (SPARK-35675) EnsureRequirements remove shuffle should respect PartitioningCollection

XiDuo You created SPARK-35675: - Summary: EnsureRequirements remove shuffle should respect PartitioningCollection Key: SPARK-35675 URL: https://issues.apache.org/jira/browse/SPARK-35675 Project: Spark

[jira] [Created] (SPARK-35629) Drop database should check if exists

XiDuo You created SPARK-35629: - Summary: Drop database should check if exists Key: SPARK-35629 URL: https://issues.apache.org/jira/browse/SPARK-35629 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-35608) Support AQE optimizer side transformUpWithPruning

XiDuo You created SPARK-35608: - Summary: Support AQE optimizer side transformUpWithPruning Key: SPARK-35608 URL: https://issues.apache.org/jira/browse/SPARK-35608 Project: Spark Issue Type:

[jira] [Created] (SPARK-35585) Support propagate empty relation through project/filter

XiDuo You created SPARK-35585: - Summary: Support propagate empty relation through project/filter Key: SPARK-35585 URL: https://issues.apache.org/jira/browse/SPARK-35585 Project: Spark Issue

[jira] [Created] (SPARK-35552) Make query stage materialized more readable

XiDuo You created SPARK-35552: - Summary: Make query stage materialized more readable Key: SPARK-35552 URL: https://issues.apache.org/jira/browse/SPARK-35552 Project: Spark Issue Type:

[jira] [Reopened] (SPARK-35442) Eliminate unnecessary join through Aggregate

[ https://issues.apache.org/jira/browse/SPARK-35442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You reopened SPARK-35442: --- > Eliminate unnecessary join through Aggregate > > >

[jira] [Created] (SPARK-35540) Make config maxShuffledHashJoinLocalMapThreshold fallback to advisoryPartitionSizeInBytes

XiDuo You created SPARK-35540: - Summary: Make config maxShuffledHashJoinLocalMapThreshold fallback to advisoryPartitionSizeInBytes Key: SPARK-35540 URL: https://issues.apache.org/jira/browse/SPARK-35540

[jira] [Updated] (SPARK-35282) Support AQE side shuffled hash join formula

[ https://issues.apache.org/jira/browse/SPARK-35282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35282: -- Description: Use AQE runtime statistics to decide if we can use shuffled hash join instead of sort

[jira] [Resolved] (SPARK-35442) Eliminate unnecessary join through Aggregate

[ https://issues.apache.org/jira/browse/SPARK-35442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You resolved SPARK-35442. --- Resolution: Duplicate > Eliminate unnecessary join through Aggregate >

[jira] [Updated] (SPARK-35455) Enhance EliminateUnnecessaryJoin

[ https://issues.apache.org/jira/browse/SPARK-35455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35455: -- Priority: Major (was: Minor) > Enhance EliminateUnnecessaryJoin > >

[jira] [Updated] (SPARK-35455) Enhance EliminateUnnecessaryJoin

[ https://issues.apache.org/jira/browse/SPARK-35455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35455: -- Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > Enhance

[jira] [Created] (SPARK-35455) Enhance EliminateUnnecessaryJoin

XiDuo You created SPARK-35455: - Summary: Enhance EliminateUnnecessaryJoin Key: SPARK-35455 URL: https://issues.apache.org/jira/browse/SPARK-35455 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35442) Eliminate unnecessary join through Aggregate

[ https://issues.apache.org/jira/browse/SPARK-35442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35442: -- Description: If Aggregate and Join have the same output partitioning, the plan will look like:

[jira] [Updated] (SPARK-35442) Eliminate unnecessary join through Aggregate

[ https://issues.apache.org/jira/browse/SPARK-35442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35442: -- Description: If Aggregate and Join have the same output partitioning, the plan will look like:

[jira] [Updated] (SPARK-35442) Eliminate unnecessary join through Aggregate

[ https://issues.apache.org/jira/browse/SPARK-35442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-35442: -- Description: If Aggregate and Join have the same output partitioning, the plan will look like:

[jira] [Created] (SPARK-35442) Eliminate unnecessary join through Aggregate

XiDuo You created SPARK-35442: - Summary: Eliminate unnecessary join through Aggregate Key: SPARK-35442 URL: https://issues.apache.org/jira/browse/SPARK-35442 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35332) Not Coalesce shuffle partitions when cache table

[ https://issues.apache.org/jira/browse/SPARK-35332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344501#comment-17344501 ] XiDuo You commented on SPARK-35332: --- [~luxianghao] Now you can `set

[jira] [Created] (SPARK-35376) Fallback config should override defaultValue

XiDuo You created SPARK-35376: - Summary: Fallback config should override defaultValue Key: SPARK-35376 URL: https://issues.apache.org/jira/browse/SPARK-35376 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35332) Not Coalesce shuffle partitions when cache table

[ https://issues.apache.org/jira/browse/SPARK-35332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341244#comment-17341244 ] XiDuo You commented on SPARK-35332: --- Adding a new cache-specific option in a CACHE statement seems a

[jira] [Commented] (SPARK-35332) Not Coalesce shuffle partitions when cache table

[ https://issues.apache.org/jira/browse/SPARK-35332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341107#comment-17341107 ] XiDuo You commented on SPARK-35332: --- The reason is Spark force disable the AQE during executing the

<    1   2   3   4   5