[jira] [Commented] (SPARK-40493) Revert "[SPARK-33861][SQL] Simplify conditional in predicate"

2022-09-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606907#comment-17606907 ] Yuming Wang commented on SPARK-40493: - Issue resolved by pull request 37729

[jira] [Resolved] (SPARK-40493) Revert "[SPARK-33861][SQL] Simplify conditional in predicate"

2022-09-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40493. - Resolution: Fixed > Revert "[SPARK-33861][SQL] Simplify conditional in predicate" >

[jira] [Updated] (SPARK-40493) Revert "[SPARK-33861][SQL] Simplify conditional in predicate"

2022-09-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40493: Fix Version/s: 3.2.3 > Revert "[SPARK-33861][SQL] Simplify conditional in predicate" >

[jira] [Updated] (SPARK-40493) Revert "[SPARK-33861][SQL] Simplify conditional in predicate"

2022-09-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40493: Fix Version/s: 3.3.1 > Revert "[SPARK-33861][SQL] Simplify conditional in predicate" >

[jira] [Updated] (SPARK-40493) Revert "[SPARK-33861][SQL] Simplify conditional in predicate"

2022-09-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40493: Affects Version/s: 3.2.2 3.2.1 3.2.0 > Revert

[jira] [Created] (SPARK-40493) Revert "[SPARK-33861][SQL] Simplify conditional in predicate"

2022-09-19 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40493: --- Summary: Revert "[SPARK-33861][SQL] Simplify conditional in predicate" Key: SPARK-40493 URL: https://issues.apache.org/jira/browse/SPARK-40493 Project: Spark

[jira] [Resolved] (SPARK-39512) Document the Spark Docker container release process

2022-09-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39512. - Fix Version/s: 3.4.0 Assignee: Holden Karau Resolution: Fixed > Document the

[jira] [Assigned] (SPARK-40461) Set upperbound for pyzmq 24.0.0 for Python linter

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-40461: --- Assignee: Hyukjin Kwon > Set upperbound for pyzmq 24.0.0 for Python linter >

[jira] [Assigned] (SPARK-40149) Star expansion after outer join asymmetrically includes joining key

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-40149: --- Assignee: Wenchen Fan > Star expansion after outer join asymmetrically includes joining

[jira] [Updated] (SPARK-39656) Fix wrong namespace in DescribeNamespaceExec

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39656: Fix Version/s: (was: 3.1.4) (was: 3.4.0) (was:

[jira] [Updated] (SPARK-40469) Avoid creating directory failures

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40469: Component/s: Spark Core (was: Build) > Avoid creating directory failures >

[jira] [Updated] (SPARK-40469) Avoid creating directory failures

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40469: Description: {noformat} java.nio.file.NoSuchFileException:

[jira] [Updated] (SPARK-40469) Avoid creating directory failures

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40469: Summary: Avoid creating directory failures (was: Upgrade Scala to 2.12.17) > Avoid creating

[jira] [Reopened] (SPARK-40469) Upgrade Scala to 2.12.17

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-40469: - > Upgrade Scala to 2.12.17 > > > Key: SPARK-40469 >

[jira] [Resolved] (SPARK-40469) Upgrade Scala to 2.12.17

2022-09-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40469. - Resolution: Duplicate > Upgrade Scala to 2.12.17 > > >

[jira] [Created] (SPARK-40469) Upgrade Scala to 2.12.17

2022-09-15 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40469: --- Summary: Upgrade Scala to 2.12.17 Key: SPARK-40469 URL: https://issues.apache.org/jira/browse/SPARK-40469 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-40463) Update gpg's keyserver

2022-09-15 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40463: --- Summary: Update gpg's keyserver Key: SPARK-40463 URL: https://issues.apache.org/jira/browse/SPARK-40463 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-40412) limit(x,y) + 子查询 出现数据丢失和乱序问题

2022-09-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40412. - Resolution: Invalid Spark SQL does not support \{{limit n, m}}. Please contact the huawei

[jira] [Updated] (SPARK-40178) Rebalance/Repartition Hints Not Working in PySpark

2022-09-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40178: Target Version/s: (was: 3.2.0, 3.2.1, 3.3.0, 3.2.2, 3.4.0, 3.3.1) > Rebalance/Repartition Hints

[jira] [Updated] (SPARK-40178) Rebalance/Repartition Hints Not Working in PySpark

2022-09-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40178: Fix Version/s: (was: 3.2.0) (was: 3.3.0) (was:

[jira] [Updated] (SPARK-40177) Simplify join condition of form (a==b) || (a==null&==null) to a<=>b

2022-09-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40177: Target Version/s: (was: 3.3.1) > Simplify join condition of form (a==b) || (a==null&==null) to

[jira] [Updated] (SPARK-40177) Simplify join condition of form (a==b) || (a==null&==null) to a<=>b

2022-09-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40177: Fix Version/s: (was: 3.3.1) > Simplify join condition of form (a==b) || (a==null&==null) to

[jira] [Updated] (SPARK-40412) limit(x,y) + 子查询 出现数据丢失和乱序问题

2022-09-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40412: Fix Version/s: (was: 2.4.5) > limit(x,y) + 子查询 出现数据丢失和乱序问题 > > >

[jira] [Commented] (SPARK-40412) limit(x,y) + 子查询 出现数据丢失和乱序问题

2022-09-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603436#comment-17603436 ] Yuming Wang commented on SPARK-40412: - Could you test the latest Spark? > limit(x,y) + 子查询

[jira] [Updated] (SPARK-40412) limit(x,y) + 子查询 出现数据丢失和乱序问题

2022-09-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40412: Target Version/s: (was: 2.4.5) > limit(x,y) + 子查询 出现数据丢失和乱序问题 > >

[jira] [Resolved] (SPARK-40401) Remove the support of deprecated `spark.akka.*` config

2022-09-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40401. - Fix Version/s: 3.4.0 Assignee: Dongjoon Hyun Resolution: Fixed > Remove the

[jira] [Resolved] (SPARK-40378) What is React Native.

2022-09-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40378. - Resolution: Invalid > What is React Native. > - > > Key:

[jira] [Updated] (SPARK-40378) What is React Native.

2022-09-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40378: Target Version/s: (was: 3.1.2) > What is React Native. > - > >

[jira] [Updated] (SPARK-40378) What is React Native.

2022-09-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40378: Fix Version/s: (was: 3.1.3) > What is React Native. > - > >

[jira] [Commented] (SPARK-40351) Spark Sum increases the precision of DecimalType arguments by 10

2022-09-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600535#comment-17600535 ] Yuming Wang commented on SPARK-40351: -

[jira] [Created] (SPARK-40322) Fix all dead links

2022-09-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40322: --- Summary: Fix all dead links Key: SPARK-40322 URL: https://issues.apache.org/jira/browse/SPARK-40322 Project: Spark Issue Type: Bug Components:

[jira] [Comment Edited] (SPARK-33861) Simplify conditional in predicate

2022-09-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599868#comment-17599868 ] Yuming Wang edited comment on SPARK-33861 at 9/3/22 8:54 AM: - Note that only

[jira] [Resolved] (SPARK-33861) Simplify conditional in predicate

2022-09-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33861. - Resolution: Won't Fix Note that only 3.2.0, 3.2.1 and 3.3.0 include this optimization. We

[jira] [Reopened] (SPARK-33861) Simplify conditional in predicate

2022-09-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-33861: - Assignee: (was: Yuming Wang) > Simplify conditional in predicate >

[jira] [Updated] (SPARK-33861) Simplify conditional in predicate

2022-09-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33861: Fix Version/s: (was: 3.2.0) > Simplify conditional in predicate >

[jira] [Resolved] (SPARK-40312) Add missing configuration documentation in Spark History Server

2022-09-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40312. - Fix Version/s: 3.4.0 Assignee: dzcxzl Resolution: Fixed > Add missing

[jira] [Assigned] (SPARK-40228) Don't simplify multiLike if child is not attribute

2022-09-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-40228: --- Assignee: Yuming Wang > Don't simplify multiLike if child is not attribute >

[jira] [Resolved] (SPARK-40228) Don't simplify multiLike if child is not attribute

2022-09-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40228. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37672

[jira] [Commented] (SPARK-40303) The performance will be worse after codegen

2022-09-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17598831#comment-17598831 ] Yuming Wang commented on SPARK-40303: - cc [~cloud_fan] [~joshrosen] [~rednaxelafx] > The

[jira] [Created] (SPARK-40303) The performance will be worse after codegen

2022-09-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40303: --- Summary: The performance will be worse after codegen Key: SPARK-40303 URL: https://issues.apache.org/jira/browse/SPARK-40303 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-40040) Push local limit to both sides if join condition is empty

2022-08-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40040. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37475

[jira] [Assigned] (SPARK-40040) Push local limit to both sides if join condition is empty

2022-08-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-40040: --- Assignee: Yuming Wang > Push local limit to both sides if join condition is empty >

[jira] [Created] (SPARK-40283) Update mima's previousSparkVersion to 3.3.0

2022-08-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40283: --- Summary: Update mima's previousSparkVersion to 3.3.0 Key: SPARK-40283 URL: https://issues.apache.org/jira/browse/SPARK-40283 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39971. - Resolution: Not A Problem > ANALYZE TABLE makes some queries run forever >

[jira] [Assigned] (SPARK-40207) Specify the column name when the data type is not supported by datasource

2022-08-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-40207: --- Assignee: Yi kaifei (was: Apache Spark) > Specify the column name when the data type is

[jira] [Resolved] (SPARK-40207) Specify the column name when the data type is not supported by datasource

2022-08-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40207. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37574

[jira] [Resolved] (SPARK-34777) [UI] StagePage input size/records not show when records greater than zero

2022-08-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-34777. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35498

[jira] [Assigned] (SPARK-34777) [UI] StagePage input size/records not show when records greater than zero

2022-08-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-34777: --- Assignee: Zhongwei Zhu > [UI] StagePage input size/records not show when records greater

[jira] [Updated] (SPARK-40248) Use larger number of bits to build bloom filter

2022-08-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40248: Component/s: SQL (was: Optimizer) > Use larger number of bits to build bloom

[jira] [Created] (SPARK-40248) Use larger number of bits to build bloom filter

2022-08-28 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40248: --- Summary: Use larger number of bits to build bloom filter Key: SPARK-40248 URL: https://issues.apache.org/jira/browse/SPARK-40248 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-40241) Correct the link of GenericUDTF

2022-08-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40241. - Fix Version/s: 3.3.1 3.1.4 3.2.3 3.4.0

[jira] [Assigned] (SPARK-40241) Correct the link of GenericUDTF

2022-08-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-40241: --- Assignee: Ruifeng Zheng > Correct the link of GenericUDTF >

[jira] [Created] (SPARK-40243) Enhance Hive UDF support documentation

2022-08-27 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40243: --- Summary: Enhance Hive UDF support documentation Key: SPARK-40243 URL: https://issues.apache.org/jira/browse/SPARK-40243 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-40228) Don't simplify multiLike if child is not attribute

2022-08-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40228: --- Summary: Don't simplify multiLike if child is not attribute Key: SPARK-40228 URL: https://issues.apache.org/jira/browse/SPARK-40228 Project: Spark Issue Type:

[jira] [Updated] (SPARK-40223) Cannot alter table with locale tr

2022-08-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40223: Description: How to reproduce this issue: {code:scala} test("Test update stats with locale tr")

[jira] [Created] (SPARK-40223) Cannot alter table with locale tr

2022-08-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40223: --- Summary: Cannot alter table with locale tr Key: SPARK-40223 URL: https://issues.apache.org/jira/browse/SPARK-40223 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-40212) SparkSQL castPartValue does not properly handle byte & short

2022-08-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17584613#comment-17584613 ] Yuming Wang commented on SPARK-40212: - How to reproduce this issue? > SparkSQL castPartValue does

[jira] [Updated] (SPARK-40207) Specify the column name when the data type is not supported by datasource

2022-08-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40207: Fix Version/s: (was: 3.4.0) > Specify the column name when the data type is not supported by

[jira] [Resolved] (SPARK-39637) Involve excludedTpcdsQueries in plan related tests

2022-08-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39637. - Resolution: Won't Fix > Involve excludedTpcdsQueries in plan related tests >

[jira] [Comment Edited] (SPARK-39915) Dataset.repartition(N) may not create N partitions

2022-08-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17582402#comment-17582402 ] Yuming Wang edited comment on SPARK-39915 at 8/20/22 10:41 PM: --- The reason

[jira] [Commented] (SPARK-39915) Dataset.repartition(N) may not create N partitions

2022-08-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17582402#comment-17582402 ] Yuming Wang commented on SPARK-39915: - The reason is that it will return empty local relation since

[jira] [Commented] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17582211#comment-17582211 ] Yuming Wang commented on SPARK-39971: - It is difficult to read. Could you set

[jira] [Resolved] (SPARK-40080) Adds additional EliminateSorts before LimitPushDown

2022-08-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40080. - Resolution: Duplicate > Adds additional EliminateSorts before LimitPushDown >

[jira] [Updated] (SPARK-40050) Enhance EliminateSorts to support removing sorts via LocalLimit

2022-08-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40050: Summary: Enhance EliminateSorts to support removing sorts via LocalLimit (was: Eliminate the

[jira] [Assigned] (SPARK-40050) Eliminate the Sort if there is a LocalLimit between Join and Sort

2022-08-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-40050: --- Assignee: Yuming Wang > Eliminate the Sort if there is a LocalLimit between Join and Sort

[jira] [Resolved] (SPARK-40050) Eliminate the Sort if there is a LocalLimit between Join and Sort

2022-08-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40050. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37519

[jira] [Resolved] (SPARK-40133) Regenerate excludedTpcdsQueries's golden files if regenerateGoldenFiles is true

2022-08-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40133. - Fix Version/s: 3.4.0 Assignee: Yuming Wang Resolution: Fixed Resolved by

[jira] [Created] (SPARK-40133) Regenerate excludedTpcdsQueries's golden files if regenerateGoldenFiles is true

2022-08-17 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40133: --- Summary: Regenerate excludedTpcdsQueries's golden files if regenerateGoldenFiles is true Key: SPARK-40133 URL: https://issues.apache.org/jira/browse/SPARK-40133

[jira] [Updated] (SPARK-40050) Eliminate the Sort if there is a LocalLimit between Join and Sort

2022-08-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40050: Summary: Eliminate the Sort if there is a LocalLimit between Join and Sort (was: Eliminate sort

[jira] [Updated] (SPARK-40080) Adds additional EliminateSorts before LimitPushDown

2022-08-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40080: Summary: Adds additional EliminateSorts before LimitPushDown (was: Move EliminateSorts before

[jira] [Created] (SPARK-40099) Merge adjacent CaseWhen branches if their values are the same

2022-08-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40099: --- Summary: Merge adjacent CaseWhen branches if their values are the same Key: SPARK-40099 URL: https://issues.apache.org/jira/browse/SPARK-40099 Project: Spark

[jira] [Created] (SPARK-40080) Move EliminateSorts before LimitPushDown

2022-08-15 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40080: --- Summary: Move EliminateSorts before LimitPushDown Key: SPARK-40080 URL: https://issues.apache.org/jira/browse/SPARK-40080 Project: Spark Issue Type:

[jira] [Updated] (SPARK-40050) Eliminate sort if parent is local limit

2022-08-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40050: Description: It seems we can remove Sort operator: {code:scala} val projectPlan =

[jira] [Created] (SPARK-40050) Eliminate sort if parent is local limit

2022-08-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40050: --- Summary: Eliminate sort if parent is local limit Key: SPARK-40050 URL: https://issues.apache.org/jira/browse/SPARK-40050 Project: Spark Issue Type:

[jira] [Updated] (SPARK-40040) Push local limit to both sides if join condition is empty

2022-08-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40040: Summary: Push local limit to both sides if join condition is empty (was: Push local limit

[jira] [Created] (SPARK-40040) Push local limit through outer join if join condition is empty

2022-08-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40040: --- Summary: Push local limit through outer join if join condition is empty Key: SPARK-40040 URL: https://issues.apache.org/jira/browse/SPARK-40040 Project: Spark

[jira] [Created] (SPARK-40017) Adjust advisory partition size in bytes based on the following operators

2022-08-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40017: --- Summary: Adjust advisory partition size in bytes based on the following operators Key: SPARK-40017 URL: https://issues.apache.org/jira/browse/SPARK-40017 Project:

[jira] [Created] (SPARK-39989) Support estimate column statistics if it is foldable expression

2022-08-05 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39989: --- Summary: Support estimate column statistics if it is foldable expression Key: SPARK-39989 URL: https://issues.apache.org/jira/browse/SPARK-39989 Project: Spark

[jira] [Created] (SPARK-39978) Make filtered distinct count more accurate

2022-08-04 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39978: --- Summary: Make filtered distinct count more accurate Key: SPARK-39978 URL: https://issues.apache.org/jira/browse/SPARK-39978 Project: Spark Issue Type:

[jira] [Updated] (SPARK-39978) Make filtered distinct count more accurate

2022-08-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39978: Attachment: data_dim_001.snappy.parquet > Make filtered distinct count more accurate >

[jira] [Commented] (SPARK-39921) SkewJoin--Stream side skew in BroadcastJoin

2022-08-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575108#comment-17575108 ] Yuming Wang commented on SPARK-39921: - +1. Would you like to file a PR? > SkewJoin--Stream side

[jira] [Commented] (SPARK-39971) ANALYZE TABLE makes some queries run forever

2022-08-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575097#comment-17575097 ] Yuming Wang commented on SPARK-39971: - [~felipepessoto] Could you provide the query plan?

[jira] [Resolved] (SPARK-39946) Upgrade jersey to 2.36

2022-08-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39946. - Resolution: Duplicate > Upgrade jersey to 2.36 > -- > >

[jira] [Assigned] (SPARK-39860) More expressions should extend Predicate

2022-07-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39860: --- Assignee: Yuming Wang > More expressions should extend Predicate >

[jira] [Resolved] (SPARK-39860) More expressions should extend Predicate

2022-07-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39860. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37274

[jira] [Commented] (SPARK-39896) The structural integrity of the plan is broken after UnwrapCastInBinaryComparison

2022-07-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571950#comment-17571950 ] Yuming Wang commented on SPARK-39896: - cc [~fchen] > The structural integrity of the plan is broken

[jira] [Created] (SPARK-39896) The structural integrity of the plan is broken after UnwrapCastInBinaryComparison

2022-07-27 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39896: --- Summary: The structural integrity of the plan is broken after UnwrapCastInBinaryComparison Key: SPARK-39896 URL: https://issues.apache.org/jira/browse/SPARK-39896

[jira] [Comment Edited] (SPARK-39750) Enable spark.sql.cbo.enabled by default

2022-07-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571308#comment-17571308 ] Yuming Wang edited comment on SPARK-39750 at 7/26/22 9:36 AM: -- A special

[jira] [Commented] (SPARK-39750) Enable spark.sql.cbo.enabled by default

2022-07-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571308#comment-17571308 ] Yuming Wang commented on SPARK-39750: - A special case, this is part of tpc-ds q47: {code:sql}

[jira] [Created] (SPARK-39860) More expressions should extend Predicate

2022-07-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39860: --- Summary: More expressions should extend Predicate Key: SPARK-39860 URL: https://issues.apache.org/jira/browse/SPARK-39860 Project: Spark Issue Type:

[jira] [Updated] (SPARK-39851) Improve join stats estimation if one side can keep uniqueness

2022-07-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39851: Summary: Improve join stats estimation if one side can keep uniqueness (was: Fix join stats

[jira] [Created] (SPARK-39851) Fix join stats estimation if one side can keep uniqueness

2022-07-24 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39851: --- Summary: Fix join stats estimation if one side can keep uniqueness Key: SPARK-39851 URL: https://issues.apache.org/jira/browse/SPARK-39851 Project: Spark

[jira] [Resolved] (SPARK-39312) Use Parquet in predicate for Spark In filter

2022-07-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39312. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36696

[jira] [Assigned] (SPARK-39312) Use Parquet in predicate for Spark In filter

2022-07-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39312: --- Assignee: Huaxin Gao > Use Parquet in predicate for Spark In filter >

[jira] [Commented] (SPARK-39753) Broadcast joins should pushdown join constraints as Filter to the larger relation

2022-07-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566130#comment-17566130 ] Yuming Wang commented on SPARK-39753: - [~devict] So the build side needs to be executed twice.  >

[jira] [Commented] (SPARK-39753) Broadcast joins should pushdown join constraints as Filter to the larger relation

2022-07-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17565410#comment-17565410 ] Yuming Wang commented on SPARK-39753: - [~devict] I do not think {{lhs.a == rhs.a}} can be written as

[jira] [Updated] (SPARK-39752) Spark job failed with 10M rows data with Broken pipe error

2022-07-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39752: Fix Version/s: (was: 3.0.2) > Spark job failed with 10M rows data with Broken pipe error >

[jira] [Created] (SPARK-39750) Enable spark.sql.cbo.enabled by default

2022-07-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39750: --- Summary: Enable spark.sql.cbo.enabled by default Key: SPARK-39750 URL: https://issues.apache.org/jira/browse/SPARK-39750 Project: Spark Issue Type:

[jira] [Updated] (SPARK-39741) Support url encode/decode as built-in function

2022-07-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39741: Fix Version/s: (was: 3.4.0) > Support url encode/decode as built-in function >

[jira] [Resolved] (SPARK-39667) Add another workaround when there is not enough memory to build and broadcast the table

2022-07-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39667. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37069

<    1   2   3   4   5   6   7   8   9   10   >