[jira] [Commented] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17545726#comment-17545726 ] Yuming Wang commented on SPARK-29260: - Please backport HIVE-8472 to your Hive metastore service if

[jira] [Resolved] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-29260. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36750

[jira] [Assigned] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-29260: --- Assignee: Chao Sun > Enable supported Hive metastore versions once it support altering

[jira] [Commented] (SPARK-39368) Move RewritePredicateSubquery into InjectRuntimeFilter

2022-06-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17545719#comment-17545719 ] Yuming Wang commented on SPARK-39368: - https://github.com/apache/spark/pull/36755 > Move

[jira] [Created] (SPARK-39368) Move RewritePredicateSubquery into InjectRuntimeFilter

2022-06-02 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39368: --- Summary: Move RewritePredicateSubquery into InjectRuntimeFilter Key: SPARK-39368 URL: https://issues.apache.org/jira/browse/SPARK-39368 Project: Spark Issue

[jira] [Commented] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2022-06-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17545264#comment-17545264 ] Yuming Wang commented on SPARK-29260: - The syntax supports it, but it doesn't change location:

[jira] [Commented] (SPARK-39354) The analysis exception is incorrect

2022-06-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17544914#comment-17544914 ] Yuming Wang commented on SPARK-39354: - Yes. It is only exception message related. > The analysis

[jira] [Commented] (SPARK-39354) The analysis exception is incorrect

2022-06-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17544894#comment-17544894 ] Yuming Wang commented on SPARK-39354: - [~maxgekk], I think this is a blocker issue for 3.3.0

[jira] [Created] (SPARK-39354) The analysis exception is incorrect

2022-06-01 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39354: --- Summary: The analysis exception is incorrect Key: SPARK-39354 URL: https://issues.apache.org/jira/browse/SPARK-39354 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39338. - Fix Version/s: 3.4.0 Assignee: Yuming Wang Resolution: Fixed > Remove dynamic

[jira] [Updated] (SPARK-39203) Fix remote table location based on database location

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39203: Affects Version/s: 3.2.0 3.1.1 3.1.0

[jira] [Resolved] (SPARK-39203) Fix remote table location based on database location

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39203. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36625

[jira] [Assigned] (SPARK-39203) Fix remote table location based on database location

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39203: --- Assignee: Yuming Wang > Fix remote table location based on database location >

[jira] [Commented] (SPARK-37609) Transient StackOverflowError on DataFrame from Catalyst QueryPlan

2022-05-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17544204#comment-17544204 ] Yuming Wang commented on SPARK-37609: - [~angerszhuuu] How to reproduce this issue? > Transient

[jira] [Updated] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39338: Issue Type: Bug (was: Improvement) > Remove dynamic pruning subquery if pruningKey's references

[jira] [Updated] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39338: Affects Version/s: 3.3.0 > Remove dynamic pruning subquery if pruningKey's references is empty >

[jira] [Created] (SPARK-39338) Remove dynamic pruning subquery if pruningKey's references is empty

2022-05-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39338: --- Summary: Remove dynamic pruning subquery if pruningKey's references is empty Key: SPARK-39338 URL: https://issues.apache.org/jira/browse/SPARK-39338 Project: Spark

[jira] [Resolved] (SPARK-39308) Upgrade parquet to 1.12.3

2022-05-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39308. - Fix Version/s: 3.4.0 Assignee: Yuming Wang Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-39308) Upgrade parquet to 1.12.3

2022-05-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39308: --- Summary: Upgrade parquet to 1.12.3 Key: SPARK-39308 URL: https://issues.apache.org/jira/browse/SPARK-39308 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-39277) Make Optimizer extends SQLConfHelper

2022-05-24 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39277: --- Summary: Make Optimizer extends SQLConfHelper Key: SPARK-39277 URL: https://issues.apache.org/jira/browse/SPARK-39277 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-38506) Push partial aggregation through join

2022-05-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541486#comment-17541486 ] Yuming Wang edited comment on SPARK-38506 at 5/24/22 2:40 PM: -- Benchmark

[jira] [Commented] (SPARK-38506) Push partial aggregation through join

2022-05-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541486#comment-17541486 ] Yuming Wang commented on SPARK-38506: - Benchmark result: |SQL|Before(ms)|With Partial Aggregation

[jira] [Created] (SPARK-39251) Simplify MultiLike if remainPatterns is empty

2022-05-22 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39251: --- Summary: Simplify MultiLike if remainPatterns is empty Key: SPARK-39251 URL: https://issues.apache.org/jira/browse/SPARK-39251 Project: Spark Issue Type:

[jira] [Commented] (SPARK-39241) Spark SQL 'Like' operator behaves wrongly while filtering on partitioned column after Spark 3.1

2022-05-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540601#comment-17540601 ] Yuming Wang commented on SPARK-39241: - I can't reproduce this issue: {code:scala} spark.sql(

[jira] [Updated] (SPARK-39248) Decimal divide much slower than multiply

2022-05-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39248: Description: How to reproduce this issue: {code:scala} import

[jira] [Created] (SPARK-39248) Decimal divide much slower than multiply

2022-05-21 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39248: --- Summary: Decimal divide much slower than multiply Key: SPARK-39248 URL: https://issues.apache.org/jira/browse/SPARK-39248 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19609) Broadcast joins should pushdown join constraints as Filter to the larger relation

2022-05-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539884#comment-17539884 ] Yuming Wang commented on SPARK-19609: - [~devict] Do you have benchmark number if pushdown join

[jira] [Created] (SPARK-39217) Makes DPP support the pruning side has Union

2022-05-17 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39217: --- Summary: Makes DPP support the pruning side has Union Key: SPARK-39217 URL: https://issues.apache.org/jira/browse/SPARK-39217 Project: Spark Issue Type:

[jira] [Created] (SPARK-39203) Fix remote table location based on database location

2022-05-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39203: --- Summary: Fix remote table location based on database location Key: SPARK-39203 URL: https://issues.apache.org/jira/browse/SPARK-39203 Project: Spark Issue

[jira] [Resolved] (SPARK-39120) Prunes the duplicate columns from child of UnaryNode

2022-05-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39120. - Resolution: Won't Do > Prunes the duplicate columns from child of UnaryNode >

[jira] [Updated] (SPARK-39173) The error message is different if disable broadcast join

2022-05-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39173: Description: How to reproduce this issue: {code:scala} sql( """ |CREATE TABLE

[jira] [Updated] (SPARK-39173) The error message is different if disable broadcast join

2022-05-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39173: Description: How to reproduce this issue: {code:scala} Seq(-1, 10L).foreach {

[jira] [Updated] (SPARK-39173) The error message is different if disable broadcast join

2022-05-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39173: Description: How to reproduce this issue: {code:scala} Seq(-1, 10L).foreach {

[jira] [Created] (SPARK-39173) The error message is different if disable broadcast join

2022-05-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39173: --- Summary: The error message is different if disable broadcast join Key: SPARK-39173 URL: https://issues.apache.org/jira/browse/SPARK-39173 Project: Spark Issue

[jira] [Updated] (SPARK-39120) Prunes the duplicate columns from child of UnaryNode

2022-05-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39120: Description: {code:scala} sql("create table t1(a int, b int, c int) using parquet") sql("select

[jira] [Created] (SPARK-39120) Prunes the duplicate columns from child of UnaryNode

2022-05-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39120: --- Summary: Prunes the duplicate columns from child of UnaryNode Key: SPARK-39120 URL: https://issues.apache.org/jira/browse/SPARK-39120 Project: Spark Issue

[jira] [Updated] (SPARK-39112) UnsupportedOperationException if spark.sql.ui.explainMode is set to cost

2022-05-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39112: Summary: UnsupportedOperationException if spark.sql.ui.explainMode is set to cost (was:

[jira] [Created] (SPARK-39112) UnsupportedOperationException if set spark.sql.ui.explainMode to cost

2022-05-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39112: --- Summary: UnsupportedOperationException if set spark.sql.ui.explainMode to cost Key: SPARK-39112 URL: https://issues.apache.org/jira/browse/SPARK-39112 Project: Spark

[jira] [Created] (SPARK-39069) Simplify another conditionals case in predicate

2022-04-29 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39069: --- Summary: Simplify another conditionals case in predicate Key: SPARK-39069 URL: https://issues.apache.org/jira/browse/SPARK-39069 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-38797) Runtime Filter support pushdown through window

2022-04-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38797. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36080

[jira] [Assigned] (SPARK-38797) Runtime Filter support pushdown through window

2022-04-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38797: --- Assignee: Yuming Wang > Runtime Filter support pushdown through window >

[jira] [Resolved] (SPARK-38565) Support Left Semi join in row level runtime filters

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38565. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36131

[jira] [Assigned] (SPARK-38565) Support Left Semi join in row level runtime filters

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38565: --- Assignee: Yuming Wang > Support Left Semi join in row level runtime filters >

[jira] [Commented] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520345#comment-17520345 ] Yuming Wang commented on SPARK-38853: - Config: {noformat} spark.master

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Attachment: enable.png > optimizeSkewsInRebalancePartitions has performance issue >

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Attachment: Disable.png > optimizeSkewsInRebalancePartitions has performance issue >

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT

[jira] [Updated] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38853: Description: How to reproduce this issue: {code:sql} CREATE TABLE t USING PARQUET AS SELECT

[jira] [Created] (SPARK-38853) optimizeSkewsInRebalancePartitions has performance issue

2022-04-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38853: --- Summary: optimizeSkewsInRebalancePartitions has performance issue Key: SPARK-38853 URL: https://issues.apache.org/jira/browse/SPARK-38853 Project: Spark Issue

[jira] [Reopened] (SPARK-34444) Pushdown scalar-subquery filter to FileSourceScan

2022-04-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-3: - > Pushdown scalar-subquery filter to FileSourceScan >

[jira] [Created] (SPARK-38843) Fix translate metadata col filters

2022-04-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38843: --- Summary: Fix translate metadata col filters Key: SPARK-38843 URL: https://issues.apache.org/jira/browse/SPARK-38843 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-38841) Enable Bloom filter join by default

2022-04-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38841. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36102

[jira] [Assigned] (SPARK-38841) Enable Bloom filter join by default

2022-04-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38841: --- Assignee: Yingyi Bu > Enable Bloom filter join by default >

[jira] [Resolved] (SPARK-38825) Add a test to cover parquet notIn filter

2022-04-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38825. - Fix Version/s: 3.3.0 Assignee: Huaxin Gao Resolution: Fixed > Add a test to

[jira] [Updated] (SPARK-38808) Windows, Spark 3.2.1: spark-shell command throwing this error: SparkContext: Error initializing SparkContext

2022-04-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38808: Target Version/s: (was: 3.2.1) > Windows, Spark 3.2.1: spark-shell command throwing this error:

[jira] [Assigned] (SPARK-38772) Formatting the log plan in AdaptiveSparkPlanExec

2022-04-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38772: --- Assignee: Yuming Wang > Formatting the log plan in AdaptiveSparkPlanExec >

[jira] [Resolved] (SPARK-38772) Formatting the log plan in AdaptiveSparkPlanExec

2022-04-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38772. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36045

[jira] [Created] (SPARK-38797) Runtime Filter support pushdown through window

2022-04-05 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38797: --- Summary: Runtime Filter support pushdown through window Key: SPARK-38797 URL: https://issues.apache.org/jira/browse/SPARK-38797 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-38778) Replace http with https for project url in pom

2022-04-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38778. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36053

[jira] [Assigned] (SPARK-38778) Replace http with https for project url in pom

2022-04-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38778: --- Assignee: Kent Yao > Replace http with https for project url in pom >

[jira] [Resolved] (SPARK-38757) Update the Oracle docker image version used for test and integration

2022-04-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38757. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36036

[jira] [Assigned] (SPARK-38757) Update the Oracle docker image version used for test and integration

2022-04-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38757: --- Assignee: Luca Canali > Update the Oracle docker image version used for test and

[jira] [Created] (SPARK-38772) Formatting the log plan in AdaptiveSparkPlanExec

2022-04-02 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38772: --- Summary: Formatting the log plan in AdaptiveSparkPlanExec Key: SPARK-38772 URL: https://issues.apache.org/jira/browse/SPARK-38772 Project: Spark Issue Type:

[jira] [Updated] (SPARK-38771) Introduce adaptive Bloom filter Join to reduce spilling to disk when doing SortMergeJoin

2022-04-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38771: Description: Insert a bloom filter on one side of the join if it may be spill when sorting and

[jira] [Updated] (SPARK-38771) Introduce adaptive Bloom filter Join to reduce spilling to disk when doing SortMergeJoin

2022-04-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38771: Attachment: Special case.png > Introduce adaptive Bloom filter Join to reduce spilling to disk

[jira] [Created] (SPARK-38771) Introduce adaptive Bloom filter Join to reduce spilling to disk when doing SortMergeJoin

2022-04-02 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38771: --- Summary: Introduce adaptive Bloom filter Join to reduce spilling to disk when doing SortMergeJoin Key: SPARK-38771 URL: https://issues.apache.org/jira/browse/SPARK-38771

[jira] [Created] (SPARK-38708) Upgrade Hive Metastore Client to the 3.1.3 for Hive 3.1

2022-03-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38708: --- Summary: Upgrade Hive Metastore Client to the 3.1.3 for Hive 3.1 Key: SPARK-38708 URL: https://issues.apache.org/jira/browse/SPARK-38708 Project: Spark Issue

[jira] [Commented] (SPARK-38703) High GC and memory footprint after switch to ZSTD

2022-03-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515003#comment-17515003 ] Yuming Wang commented on SPARK-38703: - How to reproduce this issue? > High GC and memory footprint

[jira] [Resolved] (SPARK-38702) Is there a spark (spark-sql_2.12) using log4j 2.17.X ?

2022-03-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38702. - Resolution: Invalid > Is there a spark (spark-sql_2.12) using log4j 2.17.X ? >

[jira] [Commented] (SPARK-38702) Is there a spark (spark-sql_2.12) using log4j 2.17.X ?

2022-03-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17515001#comment-17515001 ] Yuming Wang commented on SPARK-38702: - Could you use snapshot:

[jira] [Commented] (SPARK-38695) ORC can not surport the dataType,such as char or varchar

2022-03-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17514547#comment-17514547 ] Yuming Wang commented on SPARK-38695: - How to reproduce this issue? > ORC can not surport the

[jira] (SPARK-38674) Remove useless deduplicate in SubqueryBroadcastExec

2022-03-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38674 ] Yuming Wang deleted comment on SPARK-38674: - was (Author: q79969786): https://github.com/apache/spark/pull/35989 > Remove useless deduplicate in SubqueryBroadcastExec >

[jira] [Commented] (SPARK-38674) Remove useless deduplicate in SubqueryBroadcastExec

2022-03-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513301#comment-17513301 ] Yuming Wang commented on SPARK-38674: - https://github.com/apache/spark/pull/35989 > Remove useless

[jira] [Updated] (SPARK-38674) Remove useless deduplicate in SubqueryBroadcastExec

2022-03-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38674: Description: Distinct performance:

[jira] [Created] (SPARK-38674) Remove useless deduplicate in SubqueryBroadcastExec

2022-03-28 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38674: --- Summary: Remove useless deduplicate in SubqueryBroadcastExec Key: SPARK-38674 URL: https://issues.apache.org/jira/browse/SPARK-38674 Project: Spark Issue

[jira] [Commented] (SPARK-38671) Publish snapshot from branch-3.3

2022-03-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513142#comment-17513142 ] Yuming Wang commented on SPARK-38671: - https://github.com/apache/spark/pull/35986 > Publish

[jira] [Created] (SPARK-38671) Publish snapshot from branch-3.3

2022-03-27 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38671: --- Summary: Publish snapshot from branch-3.3 Key: SPARK-38671 URL: https://issues.apache.org/jira/browse/SPARK-38671 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-38665) upgrade jackson due to CVE-2020-36518

2022-03-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38665. - Fix Version/s: 3.3.0 Assignee: Cheng Pan Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38570: Fix Version/s: 3.1.4 (was: 3.1.3) > Incorrect DynamicPartitionPruning

[jira] [Updated] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38570: Fix Version/s: 3.1.3 3.0.4 3.2.2 > Incorrect

[jira] [Updated] (SPARK-38663) Remove inaccessible repository: https://dl.bintray.com

2022-03-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38663: Summary: Remove inaccessible repository: https://dl.bintray.com (was: Could not connect to

[jira] [Created] (SPARK-38663) Could not connect to dl.bintray.com

2022-03-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38663: --- Summary: Could not connect to dl.bintray.com Key: SPARK-38663 URL: https://issues.apache.org/jira/browse/SPARK-38663 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38570. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35878

[jira] [Assigned] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38570: --- Assignee: mcdull_zhang > Incorrect DynamicPartitionPruning caused by Literal >

[jira] [Resolved] (SPARK-38624) Reduce UnsafeProjection.create call times when Percentile function serializes the aggregation buffer object

2022-03-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38624. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35938

[jira] [Assigned] (SPARK-38624) Reduce UnsafeProjection.create call times when Percentile function serializes the aggregation buffer object

2022-03-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38624: --- Assignee: w l > Reduce UnsafeProjection.create call times when Percentile function

[jira] [Created] (SPARK-38634) Upgrade hive-service-rpc to 4.0.0

2022-03-23 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38634: --- Summary: Upgrade hive-service-rpc to 4.0.0 Key: SPARK-38634 URL: https://issues.apache.org/jira/browse/SPARK-38634 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38624) Reduce UnsafeProjection.create call times when Percentile function serializes the aggregation buffer object

2022-03-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38624: Fix Version/s: (was: 3.4.0) > Reduce UnsafeProjection.create call times when Percentile

[jira] [Resolved] (SPARK-38579) Requesting Restful API can cause NullPointerException

2022-03-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38579. - Fix Version/s: 3.3.0 3.2.2 3.1.3 Resolution: Fixed

[jira] [Assigned] (SPARK-38579) Requesting Restful API can cause NullPointerException

2022-03-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38579: --- Assignee: Yimin Yang > Requesting Restful API can cause NullPointerException >

[jira] [Updated] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38568: Reporter: Yuming Wang (was: Dongjoon Hyun) > Upgrade ZSTD-JNI to 1.5.2-2 >

[jira] [Commented] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507942#comment-17507942 ] Yuming Wang commented on SPARK-38568: - OK > Upgrade ZSTD-JNI to 1.5.2-2 >

[jira] [Created] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-38568: --- Summary: Upgrade ZSTD-JNI to 1.5.2-2 Key: SPARK-38568 URL: https://issues.apache.org/jira/browse/SPARK-38568 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38568: Fix Version/s: (was: 3.3.0) > Upgrade ZSTD-JNI to 1.5.2-2 > --- > >

[jira] [Assigned] (SPARK-38568) Upgrade ZSTD-JNI to 1.5.2-2

2022-03-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38568: --- Assignee: (was: Dongjoon Hyun) > Upgrade ZSTD-JNI to 1.5.2-2 >

[jira] [Updated] (SPARK-38544) Upgrade log4j2 to 2.17.2

2022-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38544: Fix Version/s: (was: 3.3.0) > Upgrade log4j2 to 2.17.2 > > >

[jira] [Reopened] (SPARK-38544) Upgrade log4j2 to 2.17.2

2022-03-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-38544: - > Upgrade log4j2 to 2.17.2 > > > Key: SPARK-38544 >

[jira] [Updated] (SPARK-36194) Add a logical plan visitor to propagate the distinct attributes

2022-03-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-36194: Summary: Add a logical plan visitor to propagate the distinct attributes (was: Remove the

<    3   4   5   6   7   8   9   10   11   12   >