[jira] [Created] (SPARK-46097) Push down limit 1 though Union and Aggregate

2023-11-24 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-46097: --- Summary: Push down limit 1 though Union and Aggregate Key: SPARK-46097 URL: https://issues.apache.org/jira/browse/SPARK-46097 Project: Spark Issue Type: Improv

[jira] [Updated] (SPARK-45954) Avoid generating redundant ShuffleExchangeExec node

2023-11-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45954: Summary: Avoid generating redundant ShuffleExchangeExec node (was: Remove redundant shuffles) >

[jira] [Created] (SPARK-45954) Remove redundant shuffles

2023-11-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-45954: --- Summary: Remove redundant shuffles Key: SPARK-45954 URL: https://issues.apache.org/jira/browse/SPARK-45954 Project: Spark Issue Type: Improvement Com

[jira] [Updated] (SPARK-45947) Set a human readable description for Dataset api

2023-11-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45947: Description: We should set the view name to sparkSession.sparkContext.setJobDescription("xxx")

[jira] [Updated] (SPARK-45947) Set a human readable description for Dataset api

2023-11-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45947: Description: Need to sparkSession.sparkContext.setJobDescription("xxx") !screenshot-1.png! > S

[jira] [Created] (SPARK-45947) Set a human readable description for Dataset api

2023-11-15 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-45947: --- Summary: Set a human readable description for Dataset api Key: SPARK-45947 URL: https://issues.apache.org/jira/browse/SPARK-45947 Project: Spark Issue Type: Im

[jira] [Updated] (SPARK-45947) Set a human readable description for Dataset api

2023-11-15 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45947: Attachment: screenshot-1.png > Set a human readable description for Dataset api >

[jira] [Updated] (SPARK-45915) Treat decimal(x, 0) the same as IntegralType in PromoteStrings

2023-11-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45915: Summary: Treat decimal(x, 0) the same as IntegralType in PromoteStrings (was: Unwrap cast in pred

[jira] [Created] (SPARK-45915) Unwrap cast in predicate

2023-11-13 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-45915: --- Summary: Unwrap cast in predicate Key: SPARK-45915 URL: https://issues.apache.org/jira/browse/SPARK-45915 Project: Spark Issue Type: Improvement Comp

[jira] [Created] (SPARK-45909) Remove the cast if it can safely up-cast in IsNotNull

2023-11-13 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-45909: --- Summary: Remove the cast if it can safely up-cast in IsNotNull Key: SPARK-45909 URL: https://issues.apache.org/jira/browse/SPARK-45909 Project: Spark Issue Typ

[jira] [Updated] (SPARK-45894) hive table level setting hadoop.mapred.max.split.size

2023-11-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45894: Target Version/s: (was: 3.5.0) > hive table level setting hadoop.mapred.max.split.size > ---

[jira] [Created] (SPARK-45895) Combine multiple like to like all

2023-11-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-45895: --- Summary: Combine multiple like to like all Key: SPARK-45895 URL: https://issues.apache.org/jira/browse/SPARK-45895 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45853) Add Iceberg and Hudi to third party projects

2023-11-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-45853: --- Summary: Add Iceberg and Hudi to third party projects Key: SPARK-45853 URL: https://issues.apache.org/jira/browse/SPARK-45853 Project: Spark Issue Type: Improv

[jira] [Updated] (SPARK-45848) spark-build-info.ps1 missing the docroot property

2023-11-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45848: Description: https://github.com/apache/spark/blob/master/build/spark-build-info.ps1#L38-L44 https:

[jira] [Created] (SPARK-45848) spark-build-info.ps1 missing the docroot property

2023-11-08 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-45848: --- Summary: spark-build-info.ps1 missing the docroot property Key: SPARK-45848 URL: https://issues.apache.org/jira/browse/SPARK-45848 Project: Spark Issue Type: B

[jira] [Updated] (SPARK-45755) Push down limit through Dataset.isEmpty()

2023-10-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45755: Description: Push down LocalLimit can not optimize the case of distinct. {code:scala} def isEmp

[jira] [Created] (SPARK-45755) Push down limit through Dataset.isEmpty()

2023-10-31 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-45755: --- Summary: Push down limit through Dataset.isEmpty() Key: SPARK-45755 URL: https://issues.apache.org/jira/browse/SPARK-45755 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-45658) Canonicalization of DynamicPruningSubquery is broken

2023-10-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45658: Target Version/s: (was: 3.5.1) > Canonicalization of DynamicPruningSubquery is broken >

[jira] [Updated] (SPARK-45658) Canonicalization of DynamicPruningSubquery is broken

2023-10-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45658: Affects Version/s: (was: 3.5.1) > Canonicalization of DynamicPruningSubquery is broken > -

[jira] [Commented] (SPARK-43851) Support LCA in grouping expressions

2023-10-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1591#comment-1591 ] Yuming Wang commented on SPARK-43851: - The resolution should be unresolved. > Suppo

[jira] [Reopened] (SPARK-43851) Support LCA in grouping expressions

2023-10-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-43851: - Assignee: (was: Jia Fan) > Support LCA in grouping expressions > -

[jira] [Updated] (SPARK-43851) Support LCA in grouping expressions

2023-10-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43851: Fix Version/s: (was: 3.5.0) > Support LCA in grouping expressions > --

[jira] [Updated] (SPARK-45454) Set the table's default owner to current_user

2023-10-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45454: Parent: (was: SPARK-30016) Issue Type: Improvement (was: Sub-task) > Set the table's

[jira] [Updated] (SPARK-45454) Set the table's default owner to current_user

2023-10-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45454: Summary: Set the table's default owner to current_user (was: Set owner of DS v2 table to CURRENT_

[jira] [Created] (SPARK-45454) Set owner of DS v2 table to CURRENT_USER if it is already set

2023-10-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-45454: --- Summary: Set owner of DS v2 table to CURRENT_USER if it is already set Key: SPARK-45454 URL: https://issues.apache.org/jira/browse/SPARK-45454 Project: Spark

[jira] [Updated] (SPARK-45387) Partition key filter cannot be pushed down when using cast

2023-09-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45387: Target Version/s: (was: 3.1.1, 3.3.0) > Partition key filter cannot be pushed down when using ca

[jira] [Created] (SPARK-45369) Push down limit through generate

2023-09-27 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-45369: --- Summary: Push down limit through generate Key: SPARK-45369 URL: https://issues.apache.org/jira/browse/SPARK-45369 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-45282) Join loses records for cached datasets

2023-09-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768399#comment-17768399 ] Yuming Wang commented on SPARK-45282: - cc [~ulysses] [~cloud_fan] > Join loses reco

[jira] [Resolved] (SPARK-43406) enable spark sql to drop multiple partitions in one call

2023-09-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43406. - Resolution: Duplicate > enable spark sql to drop multiple partitions in one call > -

[jira] [Updated] (SPARK-43406) enable spark sql to drop multiple partitions in one call

2023-09-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43406: Target Version/s: (was: 4.0.0) > enable spark sql to drop multiple partitions in one call >

[jira] [Updated] (SPARK-43406) enable spark sql to drop multiple partitions in one call

2023-09-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43406: Fix Version/s: (was: 3.5.0) > enable spark sql to drop multiple partitions in one call > -

[jira] [Updated] (SPARK-43406) enable spark sql to drop multiple partitions in one call

2023-09-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43406: Target Version/s: 4.0.0 > enable spark sql to drop multiple partitions in one call > -

[jira] [Resolved] (SPARK-45089) Remove obsolete repo of DB2 JDBC driver

2023-09-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-45089. - Fix Version/s: 4.0.0 Assignee: Cheng Pan Resolution: Fixed Issue resolved by pul

[jira] [Updated] (SPARK-45071) Optimize the processing speed of `BinaryArithmetic#dataType` when processing multi-column data

2023-09-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45071: Fix Version/s: 3.5.1 (was: 3.5.0) > Optimize the processing speed of `Binar

[jira] [Resolved] (SPARK-45071) Optimize the processing speed of `BinaryArithmetic#dataType` when processing multi-column data

2023-09-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-45071. - Fix Version/s: 3.5.0 4.0.0 3.4.2 Resolution: Fixed

[jira] [Assigned] (SPARK-45071) Optimize the processing speed of `BinaryArithmetic#dataType` when processing multi-column data

2023-09-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-45071: --- Assignee: ming95 > Optimize the processing speed of `BinaryArithmetic#dataType` when pr

[jira] [Updated] (SPARK-45020) org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'default' not found (state=08S01,code=0)

2023-09-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45020: Fix Version/s: (was: 3.1.0) > org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException:

[jira] [Assigned] (SPARK-44846) PushFoldableIntoBranches in complex grouping expressions may cause bindReference error

2023-09-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44846: --- Assignee: zhuml > PushFoldableIntoBranches in complex grouping expressions may cause > bin

[jira] [Resolved] (SPARK-44846) PushFoldableIntoBranches in complex grouping expressions may cause bindReference error

2023-09-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44846. - Fix Version/s: 3.5.0 4.0.0 3.4.2 Resolution: Fixed

[jira] [Updated] (SPARK-44892) Add official image Dockerfile for Spark 3.3.3

2023-08-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44892: Fix Version/s: (was: 4.0.0) > Add official image Dockerfile for Spark 3.3.3 >

[jira] [Resolved] (SPARK-44892) Add official image Dockerfile for Spark 3.3.3

2023-08-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44892. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 54 [https://github.

[jira] [Assigned] (SPARK-44892) Add official image Dockerfile for Spark 3.3.3

2023-08-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44892: --- Assignee: Yuming Wang > Add official image Dockerfile for Spark 3.3.3 > ---

[jira] [Created] (SPARK-44892) Add official image Dockerfile for Spark 3.3.3

2023-08-21 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-44892: --- Summary: Add official image Dockerfile for Spark 3.3.3 Key: SPARK-44892 URL: https://issues.apache.org/jira/browse/SPARK-44892 Project: Spark Issue Type: Sub-t

[jira] [Updated] (SPARK-44813) The JIRA Python misses our assignee when it searches user again

2023-08-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44813: Fix Version/s: 3.3.4 (was: 3.3.3) > The JIRA Python misses our assignee whe

[jira] [Updated] (SPARK-44857) Fix getBaseURI error in Spark Worker LogPage UI buttons

2023-08-21 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44857: Fix Version/s: 3.3.4 (was: 3.3.3) > Fix getBaseURI error in Spark Worker Lo

[jira] [Assigned] (SPARK-44880) Remove unnecessary curly braces at the end of the thread locks info

2023-08-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44880: --- Assignee: Kent Yao > Remove unnecessary curly braces at the end of the thread locks info >

[jira] [Resolved] (SPARK-44880) Remove unnecessary curly braces at the end of the thread locks info

2023-08-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44880. - Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Updated] (SPARK-44880) Remove unnecessary curly braces at the end of the thread locks info

2023-08-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44880: Fix Version/s: 3.5.1 (was: 3.5.0) > Remove unnecessary curly braces at the

[jira] [Resolved] (SPARK-44792) Upgrade curator to 5.2.0

2023-08-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44792. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42474 [https://gith

[jira] [Assigned] (SPARK-44792) Upgrade curator to 5.2.0

2023-08-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44792: --- Assignee: Yuming Wang > Upgrade curator to 5.2.0 > > >

[jira] [Updated] (SPARK-44792) Upgrade curator to 5.2.0

2023-08-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44792: Description: https://issues.apache.org/jira/browse/HADOOP-17612 https://issues.apache.org/jira/bro

[jira] [Updated] (SPARK-44792) Upgrade curator to 5.2.0

2023-08-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44792: Description: https://issues.apache.org/jira/browse/HADOOP-17612 > Upgrade curator to 5.2.0 > -

[jira] [Created] (SPARK-44792) Upgrade curator to 5.2.0

2023-08-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-44792: --- Summary: Upgrade curator to 5.2.0 Key: SPARK-44792 URL: https://issues.apache.org/jira/browse/SPARK-44792 Project: Spark Issue Type: Improvement Comp

[jira] [Updated] (SPARK-44700) Rule OptimizeCsvJsonExprs should not be applied to expression like from_json(regexp_replace)

2023-08-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44700: Fix Version/s: 3.3.0 > Rule OptimizeCsvJsonExprs should not be applied to expression like > from_

[jira] [Resolved] (SPARK-44700) Rule OptimizeCsvJsonExprs should not be applied to expression like from_json(regexp_replace)

2023-08-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44700. - Resolution: Fixed Please upgrade Spark to the latest version to fix this issue. > Rule Optimize

[jira] [Updated] (SPARK-44700) Rule OptimizeCsvJsonExprs should not be applied to expression like from_json(regexp_replace)

2023-08-10 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44700: Affects Version/s: 3.1.1 (was: 3.4.0) (was: 3

[jira] [Commented] (SPARK-24087) Avoid shuffle when join keys are a super-set of bucket keys

2023-08-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752239#comment-17752239 ] Yuming Wang commented on SPARK-24087: - Fixed by SPARK-35703. > Avoid shuffle when j

[jira] [Commented] (SPARK-44719) NoClassDefFoundError when using Hive UDF

2023-08-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752023#comment-17752023 ] Yuming Wang commented on SPARK-44719: - There are two ways to fix it: 1. Upgrade the

[jira] [Updated] (SPARK-44719) NoClassDefFoundError when using Hive UDF

2023-08-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44719: Description: How to reproduce: {noformat} spark-sql (default)> add jar /Users/yumwang/Downloads/Hi

[jira] [Updated] (SPARK-44719) NoClassDefFoundError when using Hive UDF

2023-08-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44719: Attachment: HiveUDFs-1.0-SNAPSHOT.jar > NoClassDefFoundError when using Hive UDF > ---

[jira] [Created] (SPARK-44719) NoClassDefFoundError when using Hive UDF

2023-08-08 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-44719: --- Summary: NoClassDefFoundError when using Hive UDF Key: SPARK-44719 URL: https://issues.apache.org/jira/browse/SPARK-44719 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-42500) ConstantPropagation support more cases

2023-08-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42500. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42038 [https://gith

[jira] [Assigned] (SPARK-42500) ConstantPropagation support more cases

2023-08-06 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42500: --- Assignee: Tongwei > ConstantPropagation support more cases > --

[jira] [Updated] (SPARK-44662) SPIP: Improving performance of BroadcastHashJoin queries with stream side join key on non partition columns

2023-08-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44662: Target Version/s: (was: 3.3.3) > SPIP: Improving performance of BroadcastHashJoin queries with s

[jira] [Updated] (SPARK-44662) SPIP: Improving performance of BroadcastHashJoin queries with stream side join key on non partition columns

2023-08-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44662: Fix Version/s: (was: 3.3.3) > SPIP: Improving performance of BroadcastHashJoin queries with st

[jira] [Assigned] (SPARK-44675) Increase ReservedCodeCacheSize for release build

2023-08-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44675: --- Assignee: Yuming Wang > Increase ReservedCodeCacheSize for release build >

[jira] [Resolved] (SPARK-44675) Increase ReservedCodeCacheSize for release build

2023-08-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44675. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42344 [https://gith

[jira] [Created] (SPARK-44675) Increase ReservedCodeCacheSize for release build

2023-08-04 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-44675: --- Summary: Increase ReservedCodeCacheSize for release build Key: SPARK-44675 URL: https://issues.apache.org/jira/browse/SPARK-44675 Project: Spark Issue Type: Im

[jira] [Commented] (SPARK-44654) In subquery cannot perform partition pruning

2023-08-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750739#comment-17750739 ] Yuming Wang commented on SPARK-44654: - Another way is convert join to filter if maxi

[jira] [Created] (SPARK-44651) Make do-release-docker.sh compatible with Mac m2

2023-08-02 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-44651: --- Summary: Make do-release-docker.sh compatible with Mac m2 Key: SPARK-44651 URL: https://issues.apache.org/jira/browse/SPARK-44651 Project: Spark Issue Type: Im

[jira] [Updated] (SPARK-38506) Push partial aggregation through join

2023-08-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-38506: Description: Please see https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/SQL-Request-and-

[jira] [Assigned] (SPARK-44562) Add OptimizeOneRowRelationSubquery in batch of Subquery

2023-08-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44562: --- Assignee: Yuming Wang > Add OptimizeOneRowRelationSubquery in batch of Subquery > -

[jira] [Resolved] (SPARK-44562) Add OptimizeOneRowRelationSubquery in batch of Subquery

2023-08-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44562. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42180 [https://gith

[jira] [Resolved] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0

2023-07-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44598. - Resolution: Not A Problem > spark 3.2+ can not read hive table with hbase serde when hbase Store

[jira] [Reopened] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0

2023-07-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-44598: - > spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize > is 0 > --

[jira] [Commented] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0

2023-07-31 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749418#comment-17749418 ] Yuming Wang commented on SPARK-44598: - How to reproduce this issue? > spark 3.2+ ca

[jira] [Assigned] (SPARK-44454) HiveShim getTablesByType support fallback

2023-07-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44454: --- Assignee: dzcxzl > HiveShim getTablesByType support fallback >

[jira] [Resolved] (SPARK-44454) HiveShim getTablesByType support fallback

2023-07-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44454. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42033 [https://gith

[jira] [Updated] (SPARK-44513) Upgrade snappy-java to 1.1.10.3

2023-07-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44513: Fix Version/s: 3.4.2 > Upgrade snappy-java to 1.1.10.3 > --- > >

[jira] [Created] (SPARK-44562) Add OptimizeOneRowRelationSubquery in batch of Subquery

2023-07-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-44562: --- Summary: Add OptimizeOneRowRelationSubquery in batch of Subquery Key: SPARK-44562 URL: https://issues.apache.org/jira/browse/SPARK-44562 Project: Spark Issue T

[jira] [Resolved] (SPARK-44466) Exclude configs starting with SPARK_DRIVER_PREFIX and SPARK_EXECUTOR_PREFIX from modifiedConfigs

2023-07-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44466. - Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-44466) Exclude configs starting with SPARK_DRIVER_PREFIX and SPARK_EXECUTOR_PREFIX from modifiedConfigs

2023-07-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44466: --- Assignee: Yuming Wang > Exclude configs starting with SPARK_DRIVER_PREFIX and SPARK_EXECUTO

[jira] [Commented] (SPARK-44527) Simplify BinaryComparison if its children contain ScalarSubquery with empty output

2023-07-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746488#comment-17746488 ] Yuming Wang commented on SPARK-44527: - https://github.com/apache/spark/pull/42129 >

[jira] [Created] (SPARK-44527) Simplify BinaryComparison if its children contain ScalarSubquery with empty output

2023-07-24 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-44527: --- Summary: Simplify BinaryComparison if its children contain ScalarSubquery with empty output Key: SPARK-44527 URL: https://issues.apache.org/jira/browse/SPARK-44527 Proj

[jira] [Updated] (SPARK-44523) Filter's maxRows/maxRowsPerPartition is 0 if condition is FalseLiteral

2023-07-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44523: Summary: Filter's maxRows/maxRowsPerPartition is 0 if condition is FalseLiteral (was: Filter's ma

[jira] [Commented] (SPARK-44523) Filter's maxRows should be 0 if condition is FalseLiteral

2023-07-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746290#comment-17746290 ] Yuming Wang commented on SPARK-44523: - https://github.com/apache/spark/pull/42126 >

[jira] [Created] (SPARK-44523) Filter's maxRows should be 0 if condition is FalseLiteral

2023-07-24 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-44523: --- Summary: Filter's maxRows should be 0 if condition is FalseLiteral Key: SPARK-44523 URL: https://issues.apache.org/jira/browse/SPARK-44523 Project: Spark Issue

[jira] [Updated] (SPARK-44514) Optimize join if maximum number of rows on one side is 1

2023-07-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44514: Summary: Optimize join if maximum number of rows on one side is 1 (was: Rewrite the join to filte

[jira] [Commented] (SPARK-44514) Rewrite the join to filter if one side maximum number of rows is 1

2023-07-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746143#comment-17746143 ] Yuming Wang commented on SPARK-44514: - https://github.com/apache/spark/pull/42114 >

[jira] [Created] (SPARK-44514) Rewrite the join to filter if one side maximum number of rows is 1

2023-07-23 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-44514: --- Summary: Rewrite the join to filter if one side maximum number of rows is 1 Key: SPARK-44514 URL: https://issues.apache.org/jira/browse/SPARK-44514 Project: Spark

[jira] [Updated] (SPARK-44466) Exclude configs starting with SPARK_DRIVER_PREFIX and SPARK_EXECUTOR_PREFIX from modifiedConfigs

2023-07-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44466: Summary: Exclude configs starting with SPARK_DRIVER_PREFIX and SPARK_EXECUTOR_PREFIX from modified

[jira] [Updated] (SPARK-44493) Extract pushable predicates from disjunctive predicates

2023-07-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44493: Attachment: before.png > Extract pushable predicates from disjunctive predicates > ---

[jira] [Created] (SPARK-44493) Extract pushable predicates from disjunctive predicates

2023-07-20 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-44493: --- Summary: Extract pushable predicates from disjunctive predicates Key: SPARK-44493 URL: https://issues.apache.org/jira/browse/SPARK-44493 Project: Spark Issue T

[jira] [Updated] (SPARK-44493) Extract pushable predicates from disjunctive predicates

2023-07-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44493: Description: Example: {code:sql} select count(*) from db.very_large_table where session_start_

[jira] [Updated] (SPARK-44493) Extract pushable predicates from disjunctive predicates

2023-07-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44493: Attachment: after.png > Extract pushable predicates from disjunctive predicates >

[jira] [Commented] (SPARK-44466) Update initialSessionOptions to the value after supplementation

2023-07-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744040#comment-17744040 ] Yuming Wang commented on SPARK-44466: - https://github.com/apache/spark/pull/42049 >

[jira] [Updated] (SPARK-44466) Update initialSessionOptions to the value after supplementation

2023-07-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44466: Description: Should not include this value: !screenshot-1.png! > Update initialSessionOptions t

[jira] [Updated] (SPARK-44466) Update initialSessionOptions to the value after supplementation

2023-07-17 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44466: Attachment: screenshot-1.png > Update initialSessionOptions to the value after supplementation > -

[jira] [Created] (SPARK-44466) Update initialSessionOptions to the value after supplementation

2023-07-17 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-44466: --- Summary: Update initialSessionOptions to the value after supplementation Key: SPARK-44466 URL: https://issues.apache.org/jira/browse/SPARK-44466 Project: Spark

[jira] [Commented] (SPARK-44448) Wrong results for dense_rank() <= k from InferWindowGroupLimit and DenseRankLimitIterator

2023-07-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17743592#comment-17743592 ] Yuming Wang commented on SPARK-8: - cc [~beliefer] > Wrong results for dense_ran

<    1   2   3   4   5   6   7   8   9   10   >