[jira] [Created] (SPARK-39217) Makes DPP support the pruning side has Union

2022-05-17 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39217: --- Summary: Makes DPP support the pruning side has Union Key: SPARK-39217 URL: https://issues.apache.org/jira/browse/SPARK-39217 Project: Spark Issue Type:

[jira] [Created] (SPARK-39203) Fix remote table location based on database location

2022-05-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39203: --- Summary: Fix remote table location based on database location Key: SPARK-39203 URL: https://issues.apache.org/jira/browse/SPARK-39203 Project: Spark Issue

[jira] [Resolved] (SPARK-39120) Prunes the duplicate columns from child of UnaryNode

2022-05-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39120. - Resolution: Won't Do > Prunes the duplicate columns from child of UnaryNode >

[jira] [Updated] (SPARK-39173) The error message is different if disable broadcast join

2022-05-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39173: Description: How to reproduce this issue: {code:scala} sql( """ |CREATE TABLE

[jira] [Updated] (SPARK-39173) The error message is different if disable broadcast join

2022-05-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39173: Description: How to reproduce this issue: {code:scala} Seq(-1, 10L).foreach {

[jira] [Updated] (SPARK-39173) The error message is different if disable broadcast join

2022-05-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39173: Description: How to reproduce this issue: {code:scala} Seq(-1, 10L).foreach {

[jira] [Created] (SPARK-39173) The error message is different if disable broadcast join

2022-05-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39173: --- Summary: The error message is different if disable broadcast join Key: SPARK-39173 URL: https://issues.apache.org/jira/browse/SPARK-39173 Project: Spark Issue

[jira] [Updated] (SPARK-39120) Prunes the duplicate columns from child of UnaryNode

2022-05-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39120: Description: {code:scala} sql("create table t1(a int, b int, c int) using parquet") sql("select

[jira] [Created] (SPARK-39120) Prunes the duplicate columns from child of UnaryNode

2022-05-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39120: --- Summary: Prunes the duplicate columns from child of UnaryNode Key: SPARK-39120 URL: https://issues.apache.org/jira/browse/SPARK-39120 Project: Spark Issue

[jira] [Updated] (SPARK-39112) UnsupportedOperationException if spark.sql.ui.explainMode is set to cost

2022-05-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39112: Summary: UnsupportedOperationException if spark.sql.ui.explainMode is set to cost (was:

[jira] [Created] (SPARK-39112) UnsupportedOperationException if set spark.sql.ui.explainMode to cost

2022-05-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39112: --- Summary: UnsupportedOperationException if set spark.sql.ui.explainMode to cost Key: SPARK-39112 URL: https://issues.apache.org/jira/browse/SPARK-39112 Project: Spark

[jira] [Created] (SPARK-39069) Simplify another conditionals case in predicate

2022-04-29 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39069: --- Summary: Simplify another conditionals case in predicate Key: SPARK-39069 URL: https://issues.apache.org/jira/browse/SPARK-39069 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-38797) Runtime Filter support pushdown through window

2022-04-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38797. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36080

[jira] [Assigned] (SPARK-38797) Runtime Filter support pushdown through window

2022-04-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38797: --- Assignee: Yuming Wang > Runtime Filter support pushdown through window >

[jira] [Resolved] (SPARK-38565) Support Left Semi join in row level runtime filters

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38565. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36131

[jira] [Assigned] (SPARK-38565) Support Left Semi join in row level runtime filters

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38565: --- Assignee: Yuming Wang > Support Left Semi join in row level runtime filters >

[jira] [Commented] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520345#comment-17520345 ] Yuming Wang commented on SPARK-38853: - Config: {noformat} spark.master

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Attachment: enable.png > optimizeSkewsInRebalancePartitions has performance issue >

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Attachment: Disable.png > optimizeSkewsInRebalancePartitions has performance issue >

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT

[jira] [Created] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38853: --- Summary: optimizeSkewsInRebalancePartitions has performance issue Key: SPARK-38853 URL: https://issues.apache.org/jira/browse/SPARK-38853 Project: Spark Issue

[jira] [Reopened] (SPARK-34444) Pushdown scalar-subquery filter to FileSourceScan

2022-04-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-3: - > Pushdown scalar-subquery filter to FileSourceScan >

[jira] [Created] (SPARK-38843) Fix translate metadata col filters

2022-04-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38843: --- Summary: Fix translate metadata col filters Key: SPARK-38843 URL: https://issues.apache.org/jira/browse/SPARK-38843 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-38841) Enable Bloom filter join by default

2022-04-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38841. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36102

[jira] [Assigned] (SPARK-38841) Enable Bloom filter join by default

2022-04-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38841: --- Assignee: Yingyi Bu > Enable Bloom filter join by default >

[jira] [Resolved] (SPARK-38825) Add a test to cover parquet notIn filter

2022-04-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38825. - Fix Version/s: 3.3.0 Assignee: Huaxin Gao Resolution: Fixed > Add a test to

[jira] [Updated] (SPARK-38808) Windows, Spark 3.2.1: spark-shell command throwing this error: SparkContext: Error initializing SparkContext

2022-04-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38808: Target Version/s: (was: 3.2.1) > Windows, Spark 3.2.1: spark-shell command throwing this error:

[jira] [Assigned] (SPARK-38772) Formatting the log plan in AdaptiveSparkPlanExec

2022-04-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38772: --- Assignee: Yuming Wang > Formatting the log plan in AdaptiveSparkPlanExec >

[jira] [Resolved] (SPARK-38772) Formatting the log plan in AdaptiveSparkPlanExec

2022-04-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38772. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36045

[jira] [Created] (SPARK-38797) Runtime Filter support pushdown through window

2022-04-05 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38797: --- Summary: Runtime Filter support pushdown through window Key: SPARK-38797 URL: https://issues.apache.org/jira/browse/SPARK-38797 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-38778) Replace http with https for project url in pom

2022-04-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38778. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36053

[jira] [Assigned] (SPARK-38778) Replace http with https for project url in pom

2022-04-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38778: --- Assignee: Kent Yao > Replace http with https for project url in pom >

[jira] [Resolved] (SPARK-38757) Update the Oracle docker image version used for test and integration

2022-04-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38757. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36036

[jira] [Assigned] (SPARK-38757) Update the Oracle docker image version used for test and integration

2022-04-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38757: --- Assignee: Luca Canali > Update the Oracle docker image version used for test and

[jira] [Created] (SPARK-38772) Formatting the log plan in AdaptiveSparkPlanExec

2022-04-02 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38772: --- Summary: Formatting the log plan in AdaptiveSparkPlanExec Key: SPARK-38772 URL: https://issues.apache.org/jira/browse/SPARK-38772 Project: Spark Issue Type:

[jira] [Updated] (SPARK-38771) Introduce adaptive Bloom filter Join to reduce spilling to disk when doing SortMergeJoin

2022-04-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38771: Description: Insert a bloom filter on one side of the join if it may be spill when sorting and

[jira] [Updated] (SPARK-38771) Introduce adaptive Bloom filter Join to reduce spilling to disk when doing SortMergeJoin

2022-04-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38771: Attachment: Special case.png > Introduce adaptive Bloom filter Join to reduce spilling to disk

[jira] [Created] (SPARK-38771) Introduce adaptive Bloom filter Join to reduce spilling to disk when doing SortMergeJoin

2022-04-02 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38771: --- Summary: Introduce adaptive Bloom filter Join to reduce spilling to disk when doing SortMergeJoin Key: SPARK-38771 URL: https://issues.apache.org/jira/browse/SPARK-38771

[jira] [Created] (SPARK-38708) Upgrade Hive Metastore Client to the 3.1.3 for Hive 3.1

2022-03-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38708: --- Summary: Upgrade Hive Metastore Client to the 3.1.3 for Hive 3.1 Key: SPARK-38708 URL: https://issues.apache.org/jira/browse/SPARK-38708 Project: Spark Issue

[jira] [Commented] (SPARK-38703) High GC and memory footprint after switch to ZSTD

2022-03-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515003#comment-17515003 ] Yuming Wang commented on SPARK-38703: - How to reproduce this issue? > High GC and memory footprint

[jira] [Resolved] (SPARK-38702) Is there a spark (spark-sql_2.12) using log4j 2.17.X ?

2022-03-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38702. - Resolution: Invalid > Is there a spark (spark-sql_2.12) using log4j 2.17.X ? >

[jira] [Commented] (SPARK-38702) Is there a spark (spark-sql_2.12) using log4j 2.17.X ?

2022-03-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515001#comment-17515001 ] Yuming Wang commented on SPARK-38702: - Could you use snapshot:

[jira] [Commented] (SPARK-38695) ORC can not surport the dataType,such as char or varchar

2022-03-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17514547#comment-17514547 ] Yuming Wang commented on SPARK-38695: - How to reproduce this issue? > ORC can not surport the

[jira] (SPARK-38674) Remove useless deduplicate in SubqueryBroadcastExec

2022-03-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38674 ] Yuming Wang deleted comment on SPARK-38674: - was (Author: q79969786): https://github.com/apache/spark/pull/35989 > Remove useless deduplicate in SubqueryBroadcastExec >

[jira] [Commented] (SPARK-38674) Remove useless deduplicate in SubqueryBroadcastExec

2022-03-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513301#comment-17513301 ] Yuming Wang commented on SPARK-38674: - https://github.com/apache/spark/pull/35989 > Remove useless

[jira] [Updated] (SPARK-38674) Remove useless deduplicate in SubqueryBroadcastExec

2022-03-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38674: Description: Distinct performance:

[jira] [Created] (SPARK-38674) Remove useless deduplicate in SubqueryBroadcastExec

2022-03-28 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38674: --- Summary: Remove useless deduplicate in SubqueryBroadcastExec Key: SPARK-38674 URL: https://issues.apache.org/jira/browse/SPARK-38674 Project: Spark Issue

[jira] [Commented] (SPARK-38671) Publish snapshot from branch-3.3

2022-03-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513142#comment-17513142 ] Yuming Wang commented on SPARK-38671: - https://github.com/apache/spark/pull/35986 > Publish

[jira] [Created] (SPARK-38671) Publish snapshot from branch-3.3

2022-03-27 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38671: --- Summary: Publish snapshot from branch-3.3 Key: SPARK-38671 URL: https://issues.apache.org/jira/browse/SPARK-38671 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-38665) upgrade jackson due to CVE-2020-36518

2022-03-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38665. - Fix Version/s: 3.3.0 Assignee: Cheng Pan Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38570: Fix Version/s: 3.1.4 (was: 3.1.3) > Incorrect DynamicPartitionPruning

[jira] [Updated] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38570: Fix Version/s: 3.1.3 3.0.4 3.2.2 > Incorrect

[jira] [Updated] (SPARK-38663) Remove inaccessible repository: https://dl.bintray.com

2022-03-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38663: Summary: Remove inaccessible repository: https://dl.bintray.com (was: Could not connect to

[jira] [Created] (SPARK-38663) Could not connect to dl.bintray.com

2022-03-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38663: --- Summary: Could not connect to dl.bintray.com Key: SPARK-38663 URL: https://issues.apache.org/jira/browse/SPARK-38663 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38570. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35878

[jira] [Assigned] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38570: --- Assignee: mcdull_zhang > Incorrect DynamicPartitionPruning caused by Literal >

[jira] [Resolved] (SPARK-38624) Reduce UnsafeProjection.create call times when Percentile function serializes the aggregation buffer object

2022-03-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38624. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35938

[jira] [Assigned] (SPARK-38624) Reduce UnsafeProjection.create call times when Percentile function serializes the aggregation buffer object

2022-03-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38624: --- Assignee: w l > Reduce UnsafeProjection.create call times when Percentile function

[jira] [Created] (SPARK-38634) Upgrade hive-service-rpc to 4.0.0

2022-03-23 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38634: --- Summary: Upgrade hive-service-rpc to 4.0.0 Key: SPARK-38634 URL: https://issues.apache.org/jira/browse/SPARK-38634 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38624) Reduce UnsafeProjection.create call times when Percentile function serializes the aggregation buffer object

2022-03-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38624: Fix Version/s: (was: 3.4.0) > Reduce UnsafeProjection.create call times when Percentile

[jira] [Resolved] (SPARK-38579) Requesting Restful API can cause NullPointerException

2022-03-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38579. - Fix Version/s: 3.3.0 3.2.2 3.1.3 Resolution: Fixed

[jira] [Assigned] (SPARK-38579) Requesting Restful API can cause NullPointerException

2022-03-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38579: --- Assignee: Yimin Yang > Requesting Restful API can cause NullPointerException >

[jira] [Updated] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38568: Reporter: Yuming Wang (was: Dongjoon Hyun) > Upgrade ZSTD-JNI to 1.5.2-2 >

[jira] [Commented] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507942#comment-17507942 ] Yuming Wang commented on SPARK-38568: - OK > Upgrade ZSTD-JNI to 1.5.2-2 >

[jira] [Created] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38568: --- Summary: Upgrade ZSTD-JNI to 1.5.2-2 Key: SPARK-38568 URL: https://issues.apache.org/jira/browse/SPARK-38568 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38568: Fix Version/s: (was: 3.3.0) > Upgrade ZSTD-JNI to 1.5.2-2 > --- > >

[jira] [Assigned] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38568: --- Assignee: (was: Dongjoon Hyun) > Upgrade ZSTD-JNI to 1.5.2-2 >

[jira] [Updated] (SPARK-38544) Upgrade log4j2 to 2.17.2

2022-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38544: Fix Version/s: (was: 3.3.0) > Upgrade log4j2 to 2.17.2 > > >

[jira] [Reopened] (SPARK-38544) Upgrade log4j2 to 2.17.2

2022-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-38544: - > Upgrade log4j2 to 2.17.2 > > > Key: SPARK-38544 >

[jira] [Updated] (SPARK-36194) Add a logical plan visitor to propagate the distinct attributes

2022-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36194: Summary: Add a logical plan visitor to propagate the distinct attributes (was: Remove the

[jira] [Resolved] (SPARK-36194) Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

2022-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-36194. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35779

[jira] [Assigned] (SPARK-36194) Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

2022-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-36194: --- Assignee: Yuming Wang > Remove the aggregation from left semi/anti join if the same

[jira] [Resolved] (SPARK-38540) Upgrade compress-lzf from 1.0.3 to 1.1.0

2022-03-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38540. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35834

[jira] [Assigned] (SPARK-38540) Upgrade compress-lzf from 1.0.3 to 1.1.0

2022-03-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38540: --- Assignee: Yang Jie > Upgrade compress-lzf from 1.0.3 to 1.1.0 >

[jira] [Assigned] (SPARK-38538) Fix driver environment verification in BasicDriverFeatureStepSuite

2022-03-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38538: --- Assignee: Dongjoon Hyun > Fix driver environment verification in

[jira] [Resolved] (SPARK-38538) Fix driver environment verification in BasicDriverFeatureStepSuite

2022-03-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38538. - Fix Version/s: 3.3.0 3.0.4 3.2.2 3.1.3

[jira] [Updated] (SPARK-38516) Add log4j-core, log4j-api and log4j-slf4j-impl to classpath if active hadoop-provided

2022-03-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38516: Summary: Add log4j-core, log4j-api and log4j-slf4j-impl to classpath if active hadoop-provided

[jira] [Created] (SPARK-38516) Add log4j-core and log4j-api to classpath if active hadoop-provided

2022-03-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38516: --- Summary: Add log4j-core and log4j-api to classpath if active hadoop-provided Key: SPARK-38516 URL: https://issues.apache.org/jira/browse/SPARK-38516 Project: Spark

[jira] [Created] (SPARK-38506) Push partial aggregation through join

2022-03-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38506: --- Summary: Push partial aggregation through join Key: SPARK-38506 URL: https://issues.apache.org/jira/browse/SPARK-38506 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-38505) Make partial aggregation adaptive

2022-03-10 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38505: --- Summary: Make partial aggregation adaptive Key: SPARK-38505 URL: https://issues.apache.org/jira/browse/SPARK-38505 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38489) Aggregate.groupOnly support foldable expressions

2022-03-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38489: Description: {code:scala} spark.range(200L).selectExpr("id AS a").createTempView("t1")

[jira] [Updated] (SPARK-38489) Aggregate.groupOnly support foldable expressions

2022-03-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38489: Description: {code:scala} spark.range(200L).selectExpr("id AS a").createTempView("t1")

[jira] [Updated] (SPARK-38489) Aggregate.groupOnly support foldable expressions

2022-03-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38489: Description: {code:scala} spark.range(200L).selectExpr("id AS a").createTempView("t1")

[jira] [Created] (SPARK-38489) Aggregate.groupOnly support foldable expressions

2022-03-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38489: --- Summary: Aggregate.groupOnly support foldable expressions Key: SPARK-38489 URL: https://issues.apache.org/jira/browse/SPARK-38489 Project: Spark Issue Type:

[jira] [Updated] (SPARK-38489) Aggregate.groupOnly support foldable expressions

2022-03-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38489: Fix Version/s: (was: 3.3.0) > Aggregate.groupOnly support foldable expressions >

[jira] [Assigned] (SPARK-38489) Aggregate.groupOnly support foldable expressions

2022-03-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38489: --- Assignee: (was: Yuming Wang) > Aggregate.groupOnly support foldable expressions >

[jira] [Commented] (SPARK-38282) Avoid duplicating complex partitioning expressions

2022-03-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17502674#comment-17502674 ] Yuming Wang commented on SPARK-38282: - Some related PRs: https://github.com/apache/spark/pull/33522

[jira] [Commented] (SPARK-37989) Push down limit through Aggregate if it is group only

2022-01-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17480963#comment-17480963 ] Yuming Wang commented on SPARK-37989: - Benchmark 5 {code:java} import

[jira] [Commented] (SPARK-37989) Push down limit through Aggregate if it is group only

2022-01-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17480892#comment-17480892 ] Yuming Wang commented on SPARK-37989: - Benchmark 4 {code:scala} import

[jira] [Commented] (SPARK-37989) Push down limit through Aggregate if it is group only

2022-01-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17480882#comment-17480882 ] Yuming Wang commented on SPARK-37989: - Benchmark 3 {noformat} import

[jira] [Commented] (SPARK-37989) Push down limit through Aggregate if it is group only

2022-01-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17480855#comment-17480855 ] Yuming Wang commented on SPARK-37989: - Benchmark 2 {code:scala} import

[jira] [Commented] (SPARK-37989) Push down limit through Aggregate if it is group only

2022-01-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17480839#comment-17480839 ] Yuming Wang commented on SPARK-37989: - Benchmark 1 {code:scala} import

[jira] [Created] (SPARK-37989) Push down limit through Aggregate if it is group only

2022-01-22 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37989: --- Summary: Push down limit through Aggregate if it is group only Key: SPARK-37989 URL: https://issues.apache.org/jira/browse/SPARK-37989 Project: Spark Issue

[jira] [Updated] (SPARK-37922) Combine to one cast if we can safely up-cast two casts

2022-01-18 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37922: Summary: Combine to one cast if we can safely up-cast two casts (was: Remove useless cast) >

[jira] [Created] (SPARK-37922) Remove useless cast

2022-01-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37922: --- Summary: Remove useless cast Key: SPARK-37922 URL: https://issues.apache.org/jira/browse/SPARK-37922 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-37917) Push down limit 1 for right side of left semi/anti join

2022-01-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37917: Summary: Push down limit 1 for right side of left semi/anti join (was: Push down limit for right

[jira] [Created] (SPARK-37917) Push down limit for right side of left semi/anti join

2022-01-15 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37917: --- Summary: Push down limit for right side of left semi/anti join Key: SPARK-37917 URL: https://issues.apache.org/jira/browse/SPARK-37917 Project: Spark Issue

<    3   4   5   6   7   8   9   10   11   12   >