[jira] [Assigned] (SPARK-39667) Add another workaround when there is not enough memory to build and broadcast the table

2022-07-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39667: --- Assignee: Yuming Wang > Add another workaround when there is not enough memory to build

[jira] [Resolved] (SPARK-39726) Change the default value of spark.sql.execution.topKSortFallbackThreshold to 800000

2022-07-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39726. - Resolution: Not A Problem > Change the default value of

[jira] [Resolved] (SPARK-39709) The result of executeCollect and doExecute of TakeOrderedAndProjectExec should be the same

2022-07-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39709. - Resolution: Not A Bug > The result of executeCollect and doExecute of TakeOrderedAndProjectExec

[jira] [Created] (SPARK-39726) Change the default value of spark.sql.execution.topKSortFallbackThreshold to 800000

2022-07-08 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39726: --- Summary: Change the default value of spark.sql.execution.topKSortFallbackThreshold to 80 Key: SPARK-39726 URL: https://issues.apache.org/jira/browse/SPARK-39726

[jira] [Created] (SPARK-39709) The result of executeCollect and doExecute of TakeOrderedAndProjectExec should be the same

2022-07-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39709: --- Summary: The result of executeCollect and doExecute of TakeOrderedAndProjectExec should be the same Key: SPARK-39709 URL: https://issues.apache.org/jira/browse/SPARK-39709

[jira] [Created] (SPARK-39698) Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold

2022-07-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39698: --- Summary: Use TakeOrderedAndProject if maxRows below the topKSortFallbackThreshold Key: SPARK-39698 URL: https://issues.apache.org/jira/browse/SPARK-39698 Project:

[jira] [Resolved] (SPARK-39606) Use child stats to estimate order operator

2022-07-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39606. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36994

[jira] [Assigned] (SPARK-39606) Use child stats to estimate order operator

2022-07-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39606: --- Assignee: Yuming Wang > Use child stats to estimate order operator >

[jira] [Created] (SPARK-39667) Add another workaround when there is not enough memory to build and broadcast the table

2022-07-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39667: --- Summary: Add another workaround when there is not enough memory to build and broadcast the table Key: SPARK-39667 URL: https://issues.apache.org/jira/browse/SPARK-39667

[jira] [Assigned] (SPARK-39631) Update FilterPushdownBenchmark results

2022-07-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39631: --- Assignee: Yuming Wang > Update FilterPushdownBenchmark results >

[jira] [Resolved] (SPARK-39631) Update FilterPushdownBenchmark results

2022-07-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39631. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37022

[jira] [Created] (SPARK-39655) Add a config to limit CartesianProductExec's partition number

2022-07-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39655: --- Summary: Add a config to limit CartesianProductExec's partition number Key: SPARK-39655 URL: https://issues.apache.org/jira/browse/SPARK-39655 Project: Spark

[jira] [Created] (SPARK-39642) Improve Expand statistics estimation

2022-06-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39642: --- Summary: Improve Expand statistics estimation Key: SPARK-39642 URL: https://issues.apache.org/jira/browse/SPARK-39642 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-39640) Improve window statistics estimation

2022-06-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39640: --- Summary: Improve window statistics estimation Key: SPARK-39640 URL: https://issues.apache.org/jira/browse/SPARK-39640 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-39639) Fix possible null pointer in MySQLDialect listIndexes

2022-06-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39639: Fix Version/s: (was: 3.4.0) > Fix possible null pointer in MySQLDialect listIndexes >

[jira] [Created] (SPARK-39637) Involve excludedTpcdsQueries in plan related tests

2022-06-29 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39637: --- Summary: Involve excludedTpcdsQueries in plan related tests Key: SPARK-39637 URL: https://issues.apache.org/jira/browse/SPARK-39637 Project: Spark Issue Type:

[jira] [Created] (SPARK-39631) Update FilterPushdownBenchmark results

2022-06-28 Thread Yuming Wang (Jira)
Title: Message Title Yuming Wang created

[jira] [Resolved] (SPARK-39604) Miss UT for DerbyDialet's getCatalystType

2022-06-27 Thread Yuming Wang (Jira)
Title: Message Title Yuming Wang resolved

[jira] [Assigned] (SPARK-39604) Miss UT for DerbyDialet's getCatalystType

2022-06-27 Thread Yuming Wang (Jira)
Title: Message Title Yuming Wang assigned

[jira] [Created] (SPARK-39606) Use child stats to estimate order operator

2022-06-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39606: --- Summary: Use child stats to estimate order operator Key: SPARK-39606 URL: https://issues.apache.org/jira/browse/SPARK-39606 Project: Spark Issue Type:

[jira] [Updated] (SPARK-39605) PySpark df.count() operation works fine on DBR 7.3 LTS but fails in DBR 10.4 LTS

2022-06-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39605: Fix Version/s: (was: 3.0.1) > PySpark df.count() operation works fine on DBR 7.3 LTS but

[jira] [Updated] (SPARK-39605) PySpark df.count() operation works fine on DBR 7.3 LTS but fails in DBR 10.4 LTS

2022-06-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39605: Target Version/s: (was: 3.2.1) > PySpark df.count() operation works fine on DBR 7.3 LTS but

[jira] [Created] (SPARK-39600) Enhance pushdown limit through window

2022-06-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39600: --- Summary: Enhance pushdown limit through window Key: SPARK-39600 URL: https://issues.apache.org/jira/browse/SPARK-39600 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-39447) Only non-broadcast query stage can propagate empty relation

2022-06-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39447. - Resolution: Duplicate > Only non-broadcast query stage can propagate empty relation >

[jira] [Resolved] (SPARK-39449) Propagate empty relation through Window

2022-06-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39449. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36848

[jira] [Assigned] (SPARK-39449) Propagate empty relation through Window

2022-06-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39449: --- Assignee: Yuming Wang > Propagate empty relation through Window >

[jira] [Assigned] (SPARK-39511) Push limit 1 to right side if join type is LeftSemiOrAnti and join condition is empty

2022-06-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39511: --- Assignee: Yuming Wang > Push limit 1 to right side if join type is LeftSemiOrAnti and join

[jira] [Resolved] (SPARK-39511) Push limit 1 to right side if join type is LeftSemiOrAnti and join condition is empty

2022-06-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39511. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36909

[jira] [Resolved] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

2022-06-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38614. - Fix Version/s: 3.2.2 3.4.0 3.3.1 Assignee: Bruce

[jira] [Resolved] (SPARK-39556) Propagate empty relation if maxRows is 0

2022-06-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39556. - Resolution: Not A Problem It already supported through OptimizeLimitZero and 

[jira] [Created] (SPARK-39556) Propagate empty relation if maxRows is 0

2022-06-22 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39556: --- Summary: Propagate empty relation if maxRows is 0 Key: SPARK-39556 URL: https://issues.apache.org/jira/browse/SPARK-39556 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

2022-06-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557179#comment-17557179 ] Yuming Wang commented on SPARK-38614: - Thank you for reporting this issue. Workaround: {code:sql}

[jira] [Updated] (SPARK-39539) Overflow on converting valid Milliseconds to Microseconds

2022-06-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39539: Fix Version/s: (was: 3.2.0) > Overflow on converting valid Milliseconds to Microseconds >

[jira] [Created] (SPARK-39535) Make SchemaPruning only pruning if it contains nested column

2022-06-20 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39535: --- Summary: Make SchemaPruning only pruning if it contains nested column Key: SPARK-39535 URL: https://issues.apache.org/jira/browse/SPARK-39535 Project: Spark

[jira] [Assigned] (SPARK-39445) Remove the window if windowExpressions is empty in column pruning

2022-06-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39445: --- Assignee: Yuming Wang > Remove the window if windowExpressions is empty in column pruning

[jira] [Resolved] (SPARK-39445) Remove the window if windowExpressions is empty in column pruning

2022-06-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39445. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36842

[jira] [Resolved] (SPARK-39530) Fix KafkaTestUtils to support IPv6

2022-06-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39530. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36923

[jira] [Assigned] (SPARK-39530) Fix KafkaTestUtils to support IPv6

2022-06-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39530: --- Assignee: Dongjoon Hyun > Fix KafkaTestUtils to support IPv6 >

[jira] [Commented] (SPARK-39481) Pandas UDF executed twice if used in projection followed by filter

2022-06-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556219#comment-17556219 ] Yuming Wang commented on SPARK-39481: - Copy the code from description: {code:python} from

[jira] [Commented] (SPARK-39513) HiveMetastore serializable exception with Spark 3.3.0

2022-06-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556050#comment-17556050 ] Yuming Wang commented on SPARK-39513: - [~tcarland] What is your java version? > HiveMetastore

[jira] [Resolved] (SPARK-39514) LauncherBackendSuite should add java.net.preferIPv6Addresses conf

2022-06-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39514. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36911

[jira] [Assigned] (SPARK-39514) LauncherBackendSuite should add java.net.preferIPv6Addresses conf

2022-06-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39514: --- Assignee: Dongjoon Hyun > LauncherBackendSuite should add java.net.preferIPv6Addresses

[jira] [Created] (SPARK-39511) Push limit 1 to right side if join type is LeftSemiOrAnti and join condition is empty

2022-06-18 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39511: --- Summary: Push limit 1 to right side if join type is LeftSemiOrAnti and join condition is empty Key: SPARK-39511 URL: https://issues.apache.org/jira/browse/SPARK-39511

[jira] [Created] (SPARK-39449) Propagate empty relation through Window

2022-06-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39449: --- Summary: Propagate empty relation through Window Key: SPARK-39449 URL: https://issues.apache.org/jira/browse/SPARK-39449 Project: Spark Issue Type:

[jira] [Updated] (SPARK-39448) Add ReplaceCTERefWithRepartition into nonExcludableRules list

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39448: Description: A unit test to test excluded rules: {code:scala} /* * Licensed to the Apache

[jira] [Updated] (SPARK-39448) Add ReplaceCTERefWithRepartition into nonExcludableRules list

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39448: Description: A unit test to test excluded rules: {code:scala} /* * Licensed to the Apache

[jira] [Created] (SPARK-39448) Add ReplaceCTERefWithRepartition into nonExcludableRules list

2022-06-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39448: --- Summary: Add ReplaceCTERefWithRepartition into nonExcludableRules list Key: SPARK-39448 URL: https://issues.apache.org/jira/browse/SPARK-39448 Project: Spark

[jira] [Updated] (SPARK-39447) Only non-broadcast query stage can propagate empty relation

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39447: Summary: Only non-broadcast query stage can propagate empty relation (was: Only non-broadcast

[jira] [Created] (SPARK-39447) Only non-broadcast can propagate empty relation

2022-06-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39447: --- Summary: Only non-broadcast can propagate empty relation Key: SPARK-39447 URL: https://issues.apache.org/jira/browse/SPARK-39447 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-39325) Improve MapOutputTracker convertMapStatuses performance

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39325: --- Assignee: Wan Kun > Improve MapOutputTracker convertMapStatuses performance >

[jira] [Resolved] (SPARK-39325) Improve MapOutputTracker convertMapStatuses performance

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39325. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36709

[jira] [Resolved] (SPARK-39436) graph.connectedComponents(maxIterations) get ArrayIndexOutOfBoundsException: -1

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39436. - Resolution: Duplicate > graph.connectedComponents(maxIterations) get

[jira] [Created] (SPARK-39445) Remove the window if windowExpressions is empty in column pruning

2022-06-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39445: --- Summary: Remove the window if windowExpressions is empty in column pruning Key: SPARK-39445 URL: https://issues.apache.org/jira/browse/SPARK-39445 Project: Spark

[jira] [Updated] (SPARK-39444) Add OptimizeSubqueries into nonExcludableRules list

2022-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39444: Issue Type: Bug (was: Improvement) > Add OptimizeSubqueries into nonExcludableRules list >

[jira] [Created] (SPARK-39444) Add OptimizeSubqueries into nonExcludableRules list

2022-06-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39444: --- Summary: Add OptimizeSubqueries into nonExcludableRules list Key: SPARK-39444 URL: https://issues.apache.org/jira/browse/SPARK-39444 Project: Spark Issue

[jira] [Created] (SPARK-39440) Add a config to disable event timeline

2022-06-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39440: --- Summary: Add a config to disable event timeline Key: SPARK-39440 URL: https://issues.apache.org/jira/browse/SPARK-39440 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-39428) use code block for `Coalesce Hints for SQL Queries`

2022-06-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39428: --- Assignee: qian > use code block for `Coalesce Hints for SQL Queries` >

[jira] [Resolved] (SPARK-39428) use code block for `Coalesce Hints for SQL Queries`

2022-06-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39428. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36820

[jira] [Created] (SPARK-39438) Add a threshold to not in line CTE

2022-06-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39438: --- Summary: Add a threshold to not in line CTE Key: SPARK-39438 URL: https://issues.apache.org/jira/browse/SPARK-39438 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-39400) spark-sql remain hive resource download dir after exit

2022-06-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39400. - Resolution: Fixed Issue resolved by pull request 36786

[jira] [Assigned] (SPARK-39400) spark-sql remain hive resource download dir after exit

2022-06-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39400: --- Assignee: angerszhu > spark-sql remain hive resource download dir after exit >

[jira] [Created] (SPARK-39408) Update the buildKeys for DynamicPruningSubquery.withNewPlan

2022-06-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39408: --- Summary: Update the buildKeys for DynamicPruningSubquery.withNewPlan Key: SPARK-39408 URL: https://issues.apache.org/jira/browse/SPARK-39408 Project: Spark

[jira] [Created] (SPARK-39402) Optimize ReplaceCTERefWithRepartition

2022-06-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39402: --- Summary: Optimize ReplaceCTERefWithRepartition Key: SPARK-39402 URL: https://issues.apache.org/jira/browse/SPARK-39402 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-39401) Replace withView with withTempView in CTEInlineSuite

2022-06-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39401: --- Summary: Replace withView with withTempView in CTEInlineSuite Key: SPARK-39401 URL: https://issues.apache.org/jira/browse/SPARK-39401 Project: Spark Issue

[jira] [Resolved] (SPARK-39395) Spark Thriftserver enabled LDAP,Error using beeline connection: error code 49 - invalid credentials

2022-06-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39395. - Resolution: Duplicate > Spark Thriftserver enabled LDAP,Error using beeline connection: error

[jira] [Commented] (SPARK-39386) Flaky Test: BloomFilterAggregateQuerySuite

2022-06-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17550334#comment-17550334 ] Yuming Wang commented on SPARK-39386: - [~dongjoon] Could you provide your environment. I can't

[jira] [Commented] (SPARK-34631) Caught Hive MetaException when query by partition (partition col start with ‘$’)

2022-06-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17550298#comment-17550298 ] Yuming Wang commented on SPARK-34631: - Please backport SPARK-36137 to workaround this issue. >

[jira] [Assigned] (SPARK-39377) Normalize expr ids in ListQuery and Exists expressions

2022-06-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39377: --- Assignee: Yuming Wang > Normalize expr ids in ListQuery and Exists expressions >

[jira] [Resolved] (SPARK-39377) Normalize expr ids in ListQuery and Exists expressions

2022-06-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39377. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36764

[jira] [Created] (SPARK-39377) Normalize expr ids in ListQuery and Exists expressions

2022-06-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39377: --- Summary: Normalize expr ids in ListQuery and Exists expressions Key: SPARK-39377 URL: https://issues.apache.org/jira/browse/SPARK-39377 Project: Spark Issue

[jira] [Resolved] (SPARK-39368) Move RewritePredicateSubquery into InjectRuntimeFilter

2022-06-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39368. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36755

[jira] [Assigned] (SPARK-39368) Move RewritePredicateSubquery into InjectRuntimeFilter

2022-06-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39368: --- Assignee: Yuming Wang > Move RewritePredicateSubquery into InjectRuntimeFilter >

[jira] [Created] (SPARK-39374) Improve error message for user specified column list

2022-06-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39374: --- Summary: Improve error message for user specified column list Key: SPARK-39374 URL: https://issues.apache.org/jira/browse/SPARK-39374 Project: Spark Issue

[jira] [Commented] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17545726#comment-17545726 ] Yuming Wang commented on SPARK-29260: - Please backport HIVE-8472 to your Hive metastore service if

[jira] [Resolved] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-29260. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36750

[jira] [Assigned] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-29260: --- Assignee: Chao Sun > Enable supported Hive metastore versions once it support altering

[jira] [Commented] (SPARK-39368) Move RewritePredicateSubquery into InjectRuntimeFilter

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17545719#comment-17545719 ] Yuming Wang commented on SPARK-39368: - https://github.com/apache/spark/pull/36755 > Move

[jira] [Created] (SPARK-39368) Move RewritePredicateSubquery into InjectRuntimeFilter

2022-06-02 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39368: --- Summary: Move RewritePredicateSubquery into InjectRuntimeFilter Key: SPARK-39368 URL: https://issues.apache.org/jira/browse/SPARK-39368 Project: Spark Issue

[jira] [Commented] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17545264#comment-17545264 ] Yuming Wang commented on SPARK-29260: - The syntax supports it, but it doesn't change location:

[jira] [Commented] (SPARK-39354) The analysis exception is incorrect

2022-06-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17544914#comment-17544914 ] Yuming Wang commented on SPARK-39354: - Yes. It is only exception message related. > The analysis

[jira] [Commented] (SPARK-39354) The analysis exception is incorrect

2022-06-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17544894#comment-17544894 ] Yuming Wang commented on SPARK-39354: - [~maxgekk], I think this is a blocker issue for 3.3.0

[jira] [Created] (SPARK-39354) The analysis exception is incorrect

2022-06-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39354: --- Summary: The analysis exception is incorrect Key: SPARK-39354 URL: https://issues.apache.org/jira/browse/SPARK-39354 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39338. - Fix Version/s: 3.4.0 Assignee: Yuming Wang Resolution: Fixed > Remove dynamic

[jira] [Updated] (SPARK-39203) Fix remote table location based on database location

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39203: Affects Version/s: 3.2.0 3.1.1 3.1.0

[jira] [Resolved] (SPARK-39203) Fix remote table location based on database location

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39203. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36625

[jira] [Assigned] (SPARK-39203) Fix remote table location based on database location

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39203: --- Assignee: Yuming Wang > Fix remote table location based on database location >

[jira] [Commented] (SPARK-37609) Transient StackOverflowError on DataFrame from Catalyst QueryPlan

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17544204#comment-17544204 ] Yuming Wang commented on SPARK-37609: - [~angerszhuuu] How to reproduce this issue? > Transient

[jira] [Updated] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39338: Issue Type: Bug (was: Improvement) > Remove dynamic pruning subquery if pruningKey's references

[jira] [Updated] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39338: Affects Version/s: 3.3.0 > Remove dynamic pruning subquery if pruningKey's references is empty >

[jira] [Created] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39338: --- Summary: Remove dynamic pruning subquery if pruningKey's references is empty Key: SPARK-39338 URL: https://issues.apache.org/jira/browse/SPARK-39338 Project: Spark

[jira] [Resolved] (SPARK-39308) Upgrade parquet to 1.12.3

2022-05-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39308. - Fix Version/s: 3.4.0 Assignee: Yuming Wang Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-39308) Upgrade parquet to 1.12.3

2022-05-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39308: --- Summary: Upgrade parquet to 1.12.3 Key: SPARK-39308 URL: https://issues.apache.org/jira/browse/SPARK-39308 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-39277) Make Optimizer extends SQLConfHelper

2022-05-24 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39277: --- Summary: Make Optimizer extends SQLConfHelper Key: SPARK-39277 URL: https://issues.apache.org/jira/browse/SPARK-39277 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-38506) Push partial aggregation through join

2022-05-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541486#comment-17541486 ] Yuming Wang edited comment on SPARK-38506 at 5/24/22 2:40 PM: -- Benchmark

[jira] [Commented] (SPARK-38506) Push partial aggregation through join

2022-05-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541486#comment-17541486 ] Yuming Wang commented on SPARK-38506: - Benchmark result: |SQL|Before(ms)|With Partial Aggregation

[jira] [Created] (SPARK-39251) Simplify MultiLike if remainPatterns is empty

2022-05-22 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39251: --- Summary: Simplify MultiLike if remainPatterns is empty Key: SPARK-39251 URL: https://issues.apache.org/jira/browse/SPARK-39251 Project: Spark Issue Type:

[jira] [Commented] (SPARK-39241) Spark SQL 'Like' operator behaves wrongly while filtering on partitioned column after Spark 3.1

2022-05-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540601#comment-17540601 ] Yuming Wang commented on SPARK-39241: - I can't reproduce this issue: {code:scala} spark.sql(

[jira] [Updated] (SPARK-39248) Decimal divide much slower than multiply

2022-05-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39248: Description: How to reproduce this issue: {code:scala} import

[jira] [Created] (SPARK-39248) Decimal divide much slower than multiply

2022-05-21 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39248: --- Summary: Decimal divide much slower than multiply Key: SPARK-39248 URL: https://issues.apache.org/jira/browse/SPARK-39248 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19609) Broadcast joins should pushdown join constraints as Filter to the larger relation

2022-05-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539884#comment-17539884 ] Yuming Wang commented on SPARK-19609: - [~devict] Do you have benchmark number if pushdown join

<    2   3   4   5   6   7   8   9   10   11   >