[jira] [Assigned] (SPARK-34514) Push down limit for LEFT SEMI and LEFT ANTI join

2022-01-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-34514: --- Assignee: Cheng Su (was: Apache Spark) > Push down limit for LEFT SEMI and LEFT ANTI join

[jira] [Created] (SPARK-37915) Push down deterministic projection through SQL UNION

2022-01-14 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37915: --- Summary: Push down deterministic projection through SQL UNION Key: SPARK-37915 URL: https://issues.apache.org/jira/browse/SPARK-37915 Project: Spark Issue

[jira] [Updated] (SPARK-37899) EliminateInnerJoin to support convert inner join to left semi join

2022-01-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37899: Summary: EliminateInnerJoin to support convert inner join to left semi join (was:

[jira] [Created] (SPARK-37899) EliminateInnerJoin

2022-01-13 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37899: --- Summary: EliminateInnerJoin Key: SPARK-37899 URL: https://issues.apache.org/jira/browse/SPARK-37899 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-37828) Push down filters through RebalancePartitions

2022-01-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-37828. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35118

[jira] [Assigned] (SPARK-37828) Push down filters through RebalancePartitions

2022-01-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-37828: --- Assignee: Yuming Wang > Push down filters through RebalancePartitions >

[jira] [Assigned] (SPARK-37838) Upgrade scalatestplus artifacts to 3.3.0-SNAP3

2022-01-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-37838: --- Assignee: Yuming Wang > Upgrade scalatestplus artifacts to 3.3.0-SNAP3 >

[jira] [Resolved] (SPARK-37838) Upgrade scalatestplus artifacts to 3.3.0-SNAP3

2022-01-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-37838. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35128

[jira] [Created] (SPARK-37838) Upgrade scalatestplus artifacts to 3.3.0-SNAP3

2022-01-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37838: --- Summary: Upgrade scalatestplus artifacts to 3.3.0-SNAP3 Key: SPARK-37838 URL: https://issues.apache.org/jira/browse/SPARK-37838 Project: Spark Issue Type:

[jira] [Created] (SPARK-37828) Push down filters through RebalancePartitions

2022-01-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37828: --- Summary: Push down filters through RebalancePartitions Key: SPARK-37828 URL: https://issues.apache.org/jira/browse/SPARK-37828 Project: Spark Issue Type:

[jira] [Updated] (SPARK-37807) Hivethriftserver2 send proper exception message

2022-01-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37807: Fix Version/s: (was: 3.3.0) > Hivethriftserver2 send proper exception message >

[jira] [Created] (SPARK-37642) Fall back to SortMergeJoin if BroadcastHashJoin has OOM issue

2021-12-14 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37642: --- Summary: Fall back to SortMergeJoin if BroadcastHashJoin has OOM issue Key: SPARK-37642 URL: https://issues.apache.org/jira/browse/SPARK-37642 Project: Spark

[jira] [Commented] (SPARK-37626) Upgrade libthrift to 1.15.0

2021-12-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458173#comment-17458173 ] Yuming Wang commented on SPARK-37626: - We need to upgrade to 0.16.0 because we need [this

[jira] [Resolved] (SPARK-37565) Upgrade mysql-connector-java to 8.0.27

2021-12-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-37565. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34827

[jira] [Assigned] (SPARK-37565) Upgrade mysql-connector-java to 8.0.27

2021-12-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-37565: --- Assignee: Yuming Wang > Upgrade mysql-connector-java to 8.0.27 >

[jira] [Commented] (SPARK-37609) Transient StackOverflowError on DataFrame from Catalyst QueryPlan

2021-12-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457481#comment-17457481 ] Yuming Wang commented on SPARK-37609: - How to reproduce this issue? > Transient StackOverflowError

[jira] [Commented] (SPARK-37603) org.apache.spark.sql.catalyst.expressions.ListQuery; no valid constructor

2021-12-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456920#comment-17456920 ] Yuming Wang commented on SPARK-37603: - How to reproduce this issue. >

[jira] [Created] (SPARK-37565) Upgrade mysql-connector-java to 8.0.27

2021-12-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37565: --- Summary: Upgrade mysql-connector-java to 8.0.27 Key: SPARK-37565 URL: https://issues.apache.org/jira/browse/SPARK-37565 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-37549) Support set parallel through data source properties

2021-12-05 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37549: --- Summary: Support set parallel through data source properties Key: SPARK-37549 URL: https://issues.apache.org/jira/browse/SPARK-37549 Project: Spark Issue

[jira] [Created] (SPARK-37535) Change default spark.io.compression.codec to zstd

2021-12-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37535: --- Summary: Change default spark.io.compression.codec to zstd Key: SPARK-37535 URL: https://issues.apache.org/jira/browse/SPARK-37535 Project: Spark Issue Type:

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2021-12-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452730#comment-17452730 ] Yuming Wang commented on SPARK-18105: - Workaround this issue by set spark.io.compression.codec=zstd.

[jira] [Resolved] (SPARK-37270) Incorect result of filter using isNull condition

2021-11-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-37270. - Fix Version/s: 3.2.1 Resolution: Fixed Issue resolved by pull request 34627

[jira] [Assigned] (SPARK-37270) Incorect result of filter using isNull condition

2021-11-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-37270: --- Assignee: Yuming Wang > Incorect result of filter using isNull condition >

[jira] [Assigned] (SPARK-37345) Add java.security.jgss/sun.security.krb5 to DEFAULT_MODULE_OPTIONS

2021-11-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-37345: --- Assignee: Yuming Wang > Add java.security.jgss/sun.security.krb5 to DEFAULT_MODULE_OPTIONS

[jira] [Resolved] (SPARK-37345) Add java.security.jgss/sun.security.krb5 to DEFAULT_MODULE_OPTIONS

2021-11-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-37345. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34615

[jira] [Updated] (SPARK-37347) Spark Thrift Server (STS) driver fullFC becourse of timeoutExecutor not shutdown correctly

2021-11-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37347: Target Version/s: (was: 3.1.2, 3.2.0) > Spark Thrift Server (STS) driver fullFC becourse of

[jira] [Updated] (SPARK-37347) Spark Thrift Server (STS) driver fullFC becourse of timeoutExecutor not shutdown correctly

2021-11-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37347: Fix Version/s: (was: 3.1.3) (was: 3.2.1) > Spark Thrift Server (STS)

[jira] [Commented] (SPARK-37345) Add java.security.jgss/sun.security.krb5 to DEFAULT_MODULE_OPTIONS

2021-11-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1728#comment-1728 ] Yuming Wang commented on SPARK-37345: - Related changes:

[jira] [Created] (SPARK-37345) Add java.security.jgss/sun.security.krb5 to DEFAULT_MODULE_OPTIONS

2021-11-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37345: --- Summary: Add java.security.jgss/sun.security.krb5 to DEFAULT_MODULE_OPTIONS Key: SPARK-37345 URL: https://issues.apache.org/jira/browse/SPARK-37345 Project: Spark

[jira] [Commented] (SPARK-37270) Incorect result of filter using isNull condition

2021-11-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443018#comment-17443018 ] Yuming Wang commented on SPARK-37270: - Thank you [~bersprockets] I will fix it soon. > Incorect

[jira] [Updated] (SPARK-37292) Removes outer join if it only has DISTINCT on streamed side with alias

2021-11-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37292: Description: {code:scala} spark.range(200L).selectExpr("id AS a").createTempView("t1")

[jira] [Created] (SPARK-37292) Removes outer join if it only has DISTINCT on streamed side with alias

2021-11-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37292: --- Summary: Removes outer join if it only has DISTINCT on streamed side with alias Key: SPARK-37292 URL: https://issues.apache.org/jira/browse/SPARK-37292 Project: Spark

[jira] [Resolved] (SPARK-35933) PartitionFilters and pushFilters not applied to window functions

2021-11-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-35933. - Resolution: Not A Problem > PartitionFilters and pushFilters not applied to window functions >

[jira] [Created] (SPARK-37226) Filter push down through window

2021-11-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37226: --- Summary: Filter push down through window Key: SPARK-37226 URL: https://issues.apache.org/jira/browse/SPARK-37226 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-37172) Push down filters having both partitioning and non-partitioning columns

2021-10-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17436574#comment-17436574 ] Yuming Wang commented on SPARK-37172: - Is there a data source can handle these filters? > Push down

[jira] [Commented] (SPARK-32268) Bloom Filter Join

2021-10-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435782#comment-17435782 ] Yuming Wang commented on SPARK-32268: - [~Penglei Shi] Please add my WeChat: yumwang666. We can

[jira] [Resolved] (SPARK-37109) Install Java 17 on all of the Jenkins workers

2021-10-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-37109. - Resolution: Won't Do OK. I do not know this plan. > Install Java 17 on all of the Jenkins

[jira] [Resolved] (SPARK-37110) Add Java 17 support for spark pull request builds

2021-10-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-37110. - Resolution: Won't Do OK. I do not know this plan. > Add Java 17 support for spark pull request

[jira] [Updated] (SPARK-37110) Add Java 17 support for spark pull request builds

2021-10-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37110: Summary: Add Java 17 support for spark pull request builds (was: Add java17 support for spark

[jira] [Created] (SPARK-37110) Add java17 support for spark pull request builds

2021-10-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37110: --- Summary: Add java17 support for spark pull request builds Key: SPARK-37110 URL: https://issues.apache.org/jira/browse/SPARK-37110 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37109) Install Java 17 on all of the Jenkins workers

2021-10-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433799#comment-17433799 ] Yuming Wang commented on SPARK-37109: - cc [~shaneknapp] > Install Java 17 on all of the Jenkins

[jira] [Updated] (SPARK-37109) Install Java 17 on all of the Jenkins workers

2021-10-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37109: Summary: Install Java 17 on all of the Jenkins workers (was: Add Java 17 in our Jenkins) >

[jira] [Created] (SPARK-37109) Add Java 17 in our Jenkins

2021-10-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37109: --- Summary: Add Java 17 in our Jenkins Key: SPARK-37109 URL: https://issues.apache.org/jira/browse/SPARK-37109 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-37101) In class ShuffleBlockPusher, use config instead of key

2021-10-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37101: Affects Version/s: (was: 3.2.0) 3.3.0 > In class ShuffleBlockPusher,

[jira] [Resolved] (SPARK-37041) Backport HIVE-15025: Secure-Socket-Layer (SSL) support for HMS

2021-10-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-37041. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34312

[jira] [Assigned] (SPARK-37041) Backport HIVE-15025: Secure-Socket-Layer (SSL) support for HMS

2021-10-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-37041: --- Assignee: Yuming Wang > Backport HIVE-15025: Secure-Socket-Layer (SSL) support for HMS >

[jira] [Created] (SPARK-37041) Backport HIVE-15025: Secure-Socket-Layer (SSL) support for HMS

2021-10-18 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37041: --- Summary: Backport HIVE-15025: Secure-Socket-Layer (SSL) support for HMS Key: SPARK-37041 URL: https://issues.apache.org/jira/browse/SPARK-37041 Project: Spark

[jira] [Created] (SPARK-36840) Support DPP if there is no selective predicate on the filtering side

2021-09-24 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36840: --- Summary: Support DPP if there is no selective predicate on the filtering side Key: SPARK-36840 URL: https://issues.apache.org/jira/browse/SPARK-36840 Project: Spark

[jira] [Commented] (SPARK-36763) Pull out ordering expressions

2021-09-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17418386#comment-17418386 ] Yuming Wang commented on SPARK-36763: - Benchmark and benchmark result: {code:scala} import

[jira] [Resolved] (SPARK-25936) InsertIntoDataSourceCommand does not use Cached Data

2021-09-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-25936. - Resolution: Not A Bug > InsertIntoDataSourceCommand does not use Cached Data >

[jira] [Created] (SPARK-36796) Make all unit tests pass on Java 17

2021-09-17 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36796: --- Summary: Make all unit tests pass on Java 17 Key: SPARK-36796 URL: https://issues.apache.org/jira/browse/SPARK-36796 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36780) Make dev/mima runs on Java 17

2021-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36780: Summary: Make dev/mima runs on Java 17 (was: Make dev/mima runs on Java 17+) > Make dev/mima

[jira] [Updated] (SPARK-36777) Move Java 17 on GitHub Actions from EA to LTS release

2021-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36777: Affects Version/s: (was: 3.3.0) 3.2.0 > Move Java 17 on GitHub Actions

[jira] [Created] (SPARK-36780) Make dev/mima runs on Java 17+

2021-09-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36780: --- Summary: Make dev/mima runs on Java 17+ Key: SPARK-36780 URL: https://issues.apache.org/jira/browse/SPARK-36780 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-36777) Move Java 17 on GitHub Actions from EA to LTS release

2021-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-36777. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 34017

[jira] [Created] (SPARK-36763) Pull out ordering expressions

2021-09-15 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36763: --- Summary: Pull out ordering expressions Key: SPARK-36763 URL: https://issues.apache.org/jira/browse/SPARK-36763 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-36716) Join estimation support LeftExistence join type

2021-09-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36716: Attachment: (was: image-2021-09-10-17-17-33-082.png) > Join estimation support LeftExistence

[jira] [Created] (SPARK-36716) Join estimation support LeftExistence join type

2021-09-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36716: --- Summary: Join estimation support LeftExistence join type Key: SPARK-36716 URL: https://issues.apache.org/jira/browse/SPARK-36716 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-35967) Update nullability based on column statistics

2021-09-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-35967. - Resolution: Won't Fix > Update nullability based on column statistics >

[jira] [Resolved] (SPARK-36376) Collapse repartitions if there is a project between them

2021-09-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-36376. - Resolution: Won't Fix > Collapse repartitions if there is a project between them >

[jira] [Created] (SPARK-36703) Remove the Sort if it is the child of RepartitionByExpression

2021-09-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36703: --- Summary: Remove the Sort if it is the child of RepartitionByExpression Key: SPARK-36703 URL: https://issues.apache.org/jira/browse/SPARK-36703 Project: Spark

[jira] [Updated] (SPARK-36444) Remove OptimizeSubqueries from batch of PartitionPruning

2021-08-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36444: Affects Version/s: 3.0.3 3.1.2 > Remove OptimizeSubqueries from batch of

[jira] [Updated] (SPARK-36444) Remove OptimizeSubqueries from batch of PartitionPruning

2021-08-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36444: Affects Version/s: (was: 3.3.0) 3.2.0 > Remove OptimizeSubqueries from

[jira] [Resolved] (SPARK-36444) Remove OptimizeSubqueries from batch of PartitionPruning

2021-08-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-36444. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33664

[jira] [Assigned] (SPARK-36444) Remove OptimizeSubqueries from batch of PartitionPruning

2021-08-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-36444: --- Assignee: Yuming Wang > Remove OptimizeSubqueries from batch of PartitionPruning >

[jira] [Commented] (SPARK-34276) Check the unreleased/unresolved JIRAs/PRs of Parquet 1.11

2021-08-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396450#comment-17396450 ] Yuming Wang commented on SPARK-34276: - I think so. cc [~smilegator] > Check the

[jira] [Commented] (SPARK-34276) Check the unreleased/unresolved JIRAs/PRs of Parquet 1.11

2021-08-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396435#comment-17396435 ] Yuming Wang commented on SPARK-34276: - We have used parquet 1.11/1.12 in the production environment

[jira] [Commented] (SPARK-34276) Check the unreleased/unresolved JIRAs/PRs of Parquet 1.11

2021-08-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396433#comment-17396433 ] Yuming Wang commented on SPARK-34276: - It seems that there is no unreleased/unresolved JIRAs/PRs of

[jira] [Resolved] (SPARK-36359) Coalesce drop all expressions after the first non nullable expression

2021-08-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-36359. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33590

[jira] [Assigned] (SPARK-36359) Coalesce drop all expressions after the first non nullable expression

2021-08-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-36359: --- Assignee: Yuming Wang > Coalesce drop all expressions after the first non nullable

[jira] [Commented] (SPARK-36444) Remove OptimizeSubqueries from batch of PartitionPruning

2021-08-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394641#comment-17394641 ] Yuming Wang commented on SPARK-36444: - Another case: {code:scala} sql("create table t1 using parquet

[jira] [Updated] (SPARK-36444) Remove OptimizeSubqueries from batch of PartitionPruning

2021-08-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36444: Description: To support this case: {code:scala} sql( """ |SELECT date_id,

[jira] [Created] (SPARK-36444) Remove OptimizeSubqueries from batch of PartitionPruning

2021-08-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36444: --- Summary: Remove OptimizeSubqueries from batch of PartitionPruning Key: SPARK-36444 URL: https://issues.apache.org/jira/browse/SPARK-36444 Project: Spark Issue

[jira] [Updated] (SPARK-36359) Coalesce drop all expressions after the first non nullable expression

2021-08-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36359: Summary: Coalesce drop all expressions after the first non nullable expression (was: Coalesce

[jira] [Assigned] (SPARK-36373) DecimalPrecision only add necessary cast

2021-08-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-36373: --- Assignee: Yuming Wang > DecimalPrecision only add necessary cast >

[jira] [Resolved] (SPARK-36373) DecimalPrecision only add necessary cast

2021-08-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-36373. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33602

[jira] [Created] (SPARK-36376) Collapse repartitions if there is a project between them

2021-08-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36376: --- Summary: Collapse repartitions if there is a project between them Key: SPARK-36376 URL: https://issues.apache.org/jira/browse/SPARK-36376 Project: Spark Issue

[jira] [Created] (SPARK-36373) DecimalPrecision only add necessary cast

2021-08-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36373: --- Summary: DecimalPrecision only add necessary cast Key: SPARK-36373 URL: https://issues.apache.org/jira/browse/SPARK-36373 Project: Spark Issue Type:

[jira] [Created] (SPARK-36359) Coalesce returns the first expression if it is non nullable

2021-07-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36359: --- Summary: Coalesce returns the first expression if it is non nullable Key: SPARK-36359 URL: https://issues.apache.org/jira/browse/SPARK-36359 Project: Spark

[jira] [Commented] (SPARK-36290) Push down join condition evaluation

2021-07-29 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389715#comment-17389715 ] Yuming Wang commented on SPARK-36290: - {code:java} spark.sql("create table t1 using parquet select

[jira] [Created] (SPARK-36290) Push down join condition evaluation

2021-07-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36290: --- Summary: Push down join condition evaluation Key: SPARK-36290 URL: https://issues.apache.org/jira/browse/SPARK-36290 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-31809) Infer IsNotNull for non null intolerant child of null intolerant in join condition

2021-07-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386991#comment-17386991 ] Yuming Wang commented on SPARK-31809: - {code:java} spark.sql("create table t1 using parquet select

[jira] [Created] (SPARK-36280) Remove redundant aliases after RewritePredicateSubquery

2021-07-23 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36280: --- Summary: Remove redundant aliases after RewritePredicateSubquery Key: SPARK-36280 URL: https://issues.apache.org/jira/browse/SPARK-36280 Project: Spark Issue

[jira] [Resolved] (SPARK-30186) support Dynamic Partition Pruning in Adaptive Execution

2021-07-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-30186. - Fix Version/s: 3.2.0 Assignee: Ke Jia Resolution: Fixed > support Dynamic

[jira] [Created] (SPARK-36245) Deduplicate the right side of left semi/anti join

2021-07-21 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36245: --- Summary: Deduplicate the right side of left semi/anti join Key: SPARK-36245 URL: https://issues.apache.org/jira/browse/SPARK-36245 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36238) Spark UI load event timeline too slow for huge stage

2021-07-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36238: Attachment: (was: screenshot-1.png) > Spark UI load event timeline too slow for huge stage >

[jira] [Resolved] (SPARK-36183) Push down limit 1 through Aggregate

2021-07-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-36183. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33397

[jira] [Assigned] (SPARK-36183) Push down limit 1 through Aggregate

2021-07-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-36183: --- Assignee: Yuming Wang > Push down limit 1 through Aggregate >

[jira] [Assigned] (SPARK-36093) The result incorrect if the partition path case is inconsistent

2021-07-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-36093: --- Assignee: angerszhu (was: Apache Spark) > The result incorrect if the partition path case

[jira] [Created] (SPARK-36194) Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

2021-07-17 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36194: --- Summary: Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side Key: SPARK-36194 URL:

[jira] [Created] (SPARK-36183) Push down limit 1 through Aggregate

2021-07-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36183: --- Summary: Push down limit 1 through Aggregate Key: SPARK-36183 URL: https://issues.apache.org/jira/browse/SPARK-36183 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-36162) extractJoinKeysWithColStats support EqualNullSafe

2021-07-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36162: Description: sql("select * from date_dim join item on d_date_sk = i_item_sk").explain("cost")

[jira] [Created] (SPARK-36162) extractJoinKeysWithColStats support EqualNullSafe

2021-07-15 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36162: --- Summary: extractJoinKeysWithColStats support EqualNullSafe Key: SPARK-36162 URL: https://issues.apache.org/jira/browse/SPARK-36162 Project: Spark Issue Type:

[jira] [Created] (SPARK-36155) Eliminate join base uniqueness

2021-07-15 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36155: --- Summary: Eliminate join base uniqueness Key: SPARK-36155 URL: https://issues.apache.org/jira/browse/SPARK-36155 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-36093) The result incorrect if the partition path case is inconsistent

2021-07-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36093: Labels: correctness (was: ) > The result incorrect if the partition path case is inconsistent >

[jira] [Created] (SPARK-36093) The result incorrect if the partition path case is inconsistent

2021-07-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36093: --- Summary: The result incorrect if the partition path case is inconsistent Key: SPARK-36093 URL: https://issues.apache.org/jira/browse/SPARK-36093 Project: Spark

[jira] [Created] (SPARK-36086) The case of the delta table is inconsistent with parquet

2021-07-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36086: --- Summary: The case of the delta table is inconsistent with parquet Key: SPARK-36086 URL: https://issues.apache.org/jira/browse/SPARK-36086 Project: Spark Issue

[jira] [Created] (SPARK-36080) Broadcast join outer join stream side

2021-07-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36080: --- Summary: Broadcast join outer join stream side Key: SPARK-36080 URL: https://issues.apache.org/jira/browse/SPARK-36080 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35991) Add PlanStability suite for TPCH

2021-07-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-35991: Affects Version/s: (was: 3.1.2) (was: 3.2.0)

[jira] [Updated] (SPARK-35991) Add PlanStability suite for TPCH

2021-07-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-35991: Issue Type: Improvement (was: Bug) > Add PlanStability suite for TPCH >

<    4   5   6   7   8   9   10   11   12   13   >