[jira] [Updated] (SPARK-42935) Optimze shuffle for union spark plan

[ https://issues.apache.org/jira/browse/SPARK-42935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42935: Fix Version/s: (was: 3.5.0) > Optimze shuffle for union spark plan >

[jira] [Created] (SPARK-42926) Upgrade Parquet to 1.12.4

Yuming Wang created SPARK-42926: --- Summary: Upgrade Parquet to 1.12.4 Key: SPARK-42926 URL: https://issues.apache.org/jira/browse/SPARK-42926 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-33307) Refactor GROUPING ANALYTICS

[ https://issues.apache.org/jira/browse/SPARK-33307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-33307. - Fix Version/s: 3.2.0 Assignee: angerszhu Resolution: Fixed > Refactor GROUPING

[jira] [Created] (SPARK-42868) Support eliminate sorts in AQE Optimizer

Yuming Wang created SPARK-42868: --- Summary: Support eliminate sorts in AQE Optimizer Key: SPARK-42868 URL: https://issues.apache.org/jira/browse/SPARK-42868 Project: Spark Issue Type:

[jira] (SPARK-42760) The partition of result data frame of join is always 1

[ https://issues.apache.org/jira/browse/SPARK-42760 ] Yuming Wang deleted comment on SPARK-42760: - was (Author: apachespark): User '1511351836' has created a pull request for this issue: https://github.com/apache/spark/pull/40380 > The

[jira] [Created] (SPARK-42832) Remove repartition if it is the child of LocalLimit

Yuming Wang created SPARK-42832: --- Summary: Remove repartition if it is the child of LocalLimit Key: SPARK-42832 URL: https://issues.apache.org/jira/browse/SPARK-42832 Project: Spark Issue

[jira] [Commented] (SPARK-42804) when target table format is textfile using `insert into select` will got error

[ https://issues.apache.org/jira/browse/SPARK-42804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17701006#comment-17701006 ] Yuming Wang commented on SPARK-42804: - I can't reproduce it. Did you set any configs? > when target

[jira] [Commented] (SPARK-42804) when target table format is textfile using `insert into select` will got error

[ https://issues.apache.org/jira/browse/SPARK-42804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700984#comment-17700984 ] Yuming Wang commented on SPARK-42804: - How to reproduce? > when target table format is textfile

[jira] [Comment Edited] (SPARK-42513) Push down topK through join

[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700945#comment-17700945 ] Yuming Wang edited comment on SPARK-42513 at 3/16/23 3:12 AM: -- TiDB({{mysql

[jira] [Commented] (SPARK-42513) Push down topK through join

[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700945#comment-17700945 ] Yuming Wang commented on SPARK-42513: - TiDB: {code:sql} create table t1(a int, b int) ; create

[jira] [Resolved] (SPARK-42804) when target table format is textfile using `insert into select` will got error

[ https://issues.apache.org/jira/browse/SPARK-42804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42804. - Resolution: Invalid It looks like this is a problem of your metastore. > when target table

[jira] [Updated] (SPARK-42789) Rewrite multiple GetJsonObjects to a JsonTuple if their json expression is the same

[ https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42789: Description: Benchmark result: {noformat} Running benchmark: Benchmark rewrite GetJsonObjects

[jira] [Updated] (SPARK-42789) Rewrite multiple GetJsonObjects to a JsonTuple if their json expression is the same

[ https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42789: Summary: Rewrite multiple GetJsonObjects to a JsonTuple if their json expression is the same

[jira] [Updated] (SPARK-42789) rewrites multiple GetJsonObjects to a JsonTuple if their json expression is the same

[ https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42789: Description: Benchmark result: {noformat} Running benchmark: Benchmark rewrite GetJsonObjects

[jira] [Created] (SPARK-42789) rewrites multiple GetJsonObjects to a JsonTuple if their json expression is the same

Yuming Wang created SPARK-42789: --- Summary: rewrites multiple GetJsonObjects to a JsonTuple if their json expression is the same Key: SPARK-42789 URL: https://issues.apache.org/jira/browse/SPARK-42789

[jira] [Assigned] (SPARK-42741) Do not unwrap casts in binary comparison when literal is null

[ https://issues.apache.org/jira/browse/SPARK-42741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42741: --- Assignee: Yuming Wang > Do not unwrap casts in binary comparison when literal is null >

[jira] [Resolved] (SPARK-42741) Do not unwrap casts in binary comparison when literal is null

[ https://issues.apache.org/jira/browse/SPARK-42741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42741. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40360

[jira] [Updated] (SPARK-42782) Port the tests for get_json_object from the Apache Hive project

[ https://issues.apache.org/jira/browse/SPARK-42782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42782: Summary: Port the tests for get_json_object from the Apache Hive project (was: Port TestUDFJson

[jira] [Created] (SPARK-42782) Port TestUDFJson from Hive

Yuming Wang created SPARK-42782: --- Summary: Port TestUDFJson from Hive Key: SPARK-42782 URL: https://issues.apache.org/jira/browse/SPARK-42782 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-42597) Support unwrap date type to timestamp type

[ https://issues.apache.org/jira/browse/SPARK-42597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42597: Summary: Support unwrap date type to timestamp type (was: UnwrapCastInBinaryComparison support

[jira] [Resolved] (SPARK-42597) UnwrapCastInBinaryComparison support unwrap timestamp type

[ https://issues.apache.org/jira/browse/SPARK-42597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42597. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40190

[jira] [Assigned] (SPARK-42597) UnwrapCastInBinaryComparison support unwrap timestamp type

[ https://issues.apache.org/jira/browse/SPARK-42597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42597: --- Assignee: Yuming Wang > UnwrapCastInBinaryComparison support unwrap timestamp type >

[jira] [Commented] (SPARK-42760) The partition of result data frame of join is always 1

[ https://issues.apache.org/jira/browse/SPARK-42760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699291#comment-17699291 ] Yuming Wang commented on SPARK-42760: - Could you try to disable AQE(set spark.sql.adaptive.enabled =

[jira] [Commented] (SPARK-42749) CAST(x as int) does not generate error with overflow

[ https://issues.apache.org/jira/browse/SPARK-42749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699137#comment-17699137 ] Yuming Wang commented on SPARK-42749: - Please enable ansi: {code:sql} spark-sql (default)> set

[jira] [Created] (SPARK-42741) Do not unwrap casts in binary comparison when literal is null

Yuming Wang created SPARK-42741: --- Summary: Do not unwrap casts in binary comparison when literal is null Key: SPARK-42741 URL: https://issues.apache.org/jira/browse/SPARK-42741 Project: Spark

[jira] [Assigned] (SPARK-42696) Speed up parquet reading with Java Vector API

[ https://issues.apache.org/jira/browse/SPARK-42696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42696: --- Assignee: jiangjiguang0719 (was: Yuming Wang) > Speed up parquet reading with Java Vector

[jira] [Assigned] (SPARK-42696) Speed up parquet reading with Java Vector API

[ https://issues.apache.org/jira/browse/SPARK-42696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42696: --- Assignee: Yuming Wang > Speed up parquet reading with Java Vector API >

[jira] [Commented] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697316#comment-17697316 ] Yuming Wang commented on SPARK-42694: - Could you upgrade to Spark 3.1.3 or Spark 3.3.2? > Data

[jira] [Resolved] (SPARK-42682) When I execute the spark-shell command, "WARN ui.JettyUtils: GET /jobs/ failed: java.util.NoSuchElementException: Failed to get the application information. If you are

[ https://issues.apache.org/jira/browse/SPARK-42682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42682. - Resolution: Invalid This is because the spark context is not fully started. > When I execute

[jira] [Resolved] (SPARK-34145) Combine scalar subqueries

[ https://issues.apache.org/jira/browse/SPARK-34145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-34145. - Resolution: Not A Problem Issue fixed through SPARK-34079. > Combine scalar subqueries >

[jira] [Resolved] (SPARK-34117) Disable LeftSemi/LeftAnti push down over Aggregate

[ https://issues.apache.org/jira/browse/SPARK-34117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-34117. - Resolution: Not A Problem Issue fixed through SPARK-34081. > Disable LeftSemi/LeftAnti push

[jira] [Commented] (SPARK-42650) link issue SPARK-42550

[ https://issues.apache.org/jira/browse/SPARK-42650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696665#comment-17696665 ] Yuming Wang commented on SPARK-42650: - It seems to be caused by

[jira] [Resolved] (SPARK-42675) Should clean up temp view after test

[ https://issues.apache.org/jira/browse/SPARK-42675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42675. - Fix Version/s: 3.4.0 Assignee: Yang Jie (was: Apache Spark) Resolution: Fixed

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696634#comment-17696634 ] Yuming Wang commented on SPARK-40610: - This is because of

[jira] [Resolved] (SPARK-42473) An explicit cast will be needed when INSERT OVERWRITE SELECT UNION ALL

[ https://issues.apache.org/jira/browse/SPARK-42473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42473. - Fix Version/s: 3.3.3 Assignee: Runyao.Chen Resolution: Fixed > An explicit cast

[jira] [Commented] (SPARK-42650) link issue SPARK-42550

[ https://issues.apache.org/jira/browse/SPARK-42650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695989#comment-17695989 ] Yuming Wang commented on SPARK-42650: - It seems like a Kyuubi bug? > link issue SPARK-42550 >

[jira] [Updated] (SPARK-42597) UnwrapCastInBinaryComparison support unwrap timestamp type

[ https://issues.apache.org/jira/browse/SPARK-42597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42597: Summary: UnwrapCastInBinaryComparison support unwrap timestamp type (was:

[jira] [Created] (SPARK-42597) UnwrapCastInBinaryComparison also support timestamp type

Yuming Wang created SPARK-42597: --- Summary: UnwrapCastInBinaryComparison also support timestamp type Key: SPARK-42597 URL: https://issues.apache.org/jira/browse/SPARK-42597 Project: Spark Issue

[jira] [Resolved] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42594. - Resolution: Not A Bug > spark can not read lastest view sql when run `create or replace view`

[jira] [Reopened] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-42594: - > spark can not read lastest view sql when run `create or replace view` by hive >

[jira] [Commented] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693799#comment-17693799 ] Yuming Wang commented on SPARK-42594: - Spark saves information to table properties, Hive does not

[jira] [Updated] (SPARK-42583) Remove outer join if all aggregate functions are distinct

[ https://issues.apache.org/jira/browse/SPARK-42583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42583: Description: To support more cases:

[jira] [Created] (SPARK-42583) Remove outer join if all aggregate functions are distinct

Yuming Wang created SPARK-42583: --- Summary: Remove outer join if all aggregate functions are distinct Key: SPARK-42583 URL: https://issues.apache.org/jira/browse/SPARK-42583 Project: Spark

[jira] [Resolved] (SPARK-42525) collapse two adjacent windows with the same partition/order in subquery

[ https://issues.apache.org/jira/browse/SPARK-42525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42525. - Fix Version/s: 3.5.0 Assignee: zhuml Resolution: Fixed > collapse two adjacent

[jira] [Resolved] (SPARK-42550) table directory will lost on hdfs when `INSERT OVERWRITE` faild

[ https://issues.apache.org/jira/browse/SPARK-42550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42550. - Resolution: Cannot Reproduce > table directory will lost on hdfs when `INSERT OVERWRITE` faild

[jira] [Commented] (SPARK-42550) table directory will lost on hdfs when `INSERT OVERWRITE` faild

[ https://issues.apache.org/jira/browse/SPARK-42550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693103#comment-17693103 ] Yuming Wang commented on SPARK-42550: - I can't reproduce this issue: {noformat} spark-sql> select

[jira] [Commented] (SPARK-42550) table directory will lost on hdfs when `INSERT OVERWRITE` faild

[ https://issues.apache.org/jira/browse/SPARK-42550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693016#comment-17693016 ] Yuming Wang commented on SPARK-42550: - Please refresh table: {code:sql} refresh table

[jira] [Updated] (SPARK-42550) table directory will lost on hdfs when `INSERT OVERWRITE` faild

[ https://issues.apache.org/jira/browse/SPARK-42550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42550: Description: {color:#4c9aff}when a  `{*}INSERT{*} OVERWRITE *TABLE`* statment faild during

[jira] [Updated] (SPARK-40158) Remove useless configuration & extract common code for parquet read

[ https://issues.apache.org/jira/browse/SPARK-40158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40158: Fix Version/s: (was: 3.4.0) > Remove useless configuration & extract common code for parquet

[jira] [Updated] (SPARK-40355) Improve pushdown for orc & parquet when cast scenario

[ https://issues.apache.org/jira/browse/SPARK-40355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40355: Fix Version/s: (was: 3.4.0) > Improve pushdown for orc & parquet when cast scenario >

[jira] [Updated] (SPARK-42513) Push down topK through join

[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42513: Summary: Push down topK through join (was: Push topK through join) > Push down topK through join

[jira] [Updated] (SPARK-42513) Push topK through join

[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42513: Attachment: before-UI.png > Push topK through join > -- > >

[jira] [Updated] (SPARK-42513) Push topK through join

[ https://issues.apache.org/jira/browse/SPARK-42513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42513: Attachment: after-UI.png > Push topK through join > -- > >

[jira] [Created] (SPARK-42513) Push topK through join

Yuming Wang created SPARK-42513: --- Summary: Push topK through join Key: SPARK-42513 URL: https://issues.apache.org/jira/browse/SPARK-42513 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-42506) Fix Sort's maxRowsPerPartition if maxRows does not exist

[ https://issues.apache.org/jira/browse/SPARK-42506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42506. - Resolution: Not A Problem > Fix Sort's maxRowsPerPartition if maxRows does not exist >

[jira] [Created] (SPARK-42506) Fix Sort's maxRowsPerPartition if maxRows does not exist

Yuming Wang created SPARK-42506: --- Summary: Fix Sort's maxRowsPerPartition if maxRows does not exist Key: SPARK-42506 URL: https://issues.apache.org/jira/browse/SPARK-42506 Project: Spark Issue

[jira] [Commented] (SPARK-40278) Used databricks spark-sql-pref with Spark 3.3 to run 3TB tpcds test failed

[ https://issues.apache.org/jira/browse/SPARK-40278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691388#comment-17691388 ] Yuming Wang commented on SPARK-40278: - [~LuciferYang] Is this issue still exist? > Used databricks

[jira] [Commented] (SPARK-40610) Spark fall back to use getPartitions instead of getPartitionsByFilter when date_add functions used in where clause

[ https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691387#comment-17691387 ] Yuming Wang commented on SPARK-40610: - [~icyjhl] What's your dt data type? date, string or

[jira] [Commented] (SPARK-42503) Spark SQL should do further validation on join condition fields

[ https://issues.apache.org/jira/browse/SPARK-42503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691321#comment-17691321 ] Yuming Wang commented on SPARK-42503: - Do other databases also have this validation? > Spark SQL

[jira] [Updated] (SPARK-42503) Spark SQL should do further validation on join condition fields

[ https://issues.apache.org/jira/browse/SPARK-42503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42503: Fix Version/s: (was: 3.4.0) > Spark SQL should do further validation on join condition fields

[jira] [Updated] (SPARK-42503) Spark SQL should do further validation on join condition fields

[ https://issues.apache.org/jira/browse/SPARK-42503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42503: Target Version/s: (was: 3.4.0) > Spark SQL should do further validation on join condition

[jira] [Resolved] (SPARK-41741) [SQL] ParquetFilters StringStartsWith push down matching string do not use UTF-8

[ https://issues.apache.org/jira/browse/SPARK-41741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-41741. - Resolution: Fixed > [SQL] ParquetFilters StringStartsWith push down matching string do not use

[jira] [Assigned] (SPARK-41741) [SQL] ParquetFilters StringStartsWith push down matching string do not use UTF-8

[ https://issues.apache.org/jira/browse/SPARK-41741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-41741: --- Fix Version/s: 3.4.0 3.3.3 Assignee: Yuming Wang > [SQL]

[jira] [Created] (SPARK-42500) ConstantPropagation support more cases

Yuming Wang created SPARK-42500: --- Summary: ConstantPropagation support more cases Key: SPARK-42500 URL: https://issues.apache.org/jira/browse/SPARK-42500 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-42286) Fix internal error for valid CASE WHEN expression with CAST when inserting into a table

[ https://issues.apache.org/jira/browse/SPARK-42286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42286: --- Fix Version/s: 3.4.0 Assignee: Runyao.Chen > Fix internal error for valid CASE

[jira] [Commented] (SPARK-42473) An explicit cast will be needed when INSERT OVERWRITE SELECT UNION ALL

[ https://issues.apache.org/jira/browse/SPARK-42473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691011#comment-17691011 ] Yuming Wang commented on SPARK-42473: - It seems we should backport

[jira] [Commented] (SPARK-42473) An explicit cast will be needed when INSERT OVERWRITE SELECT UNION ALL

[ https://issues.apache.org/jira/browse/SPARK-42473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690833#comment-17690833 ] Yuming Wang commented on SPARK-42473: - What is your test.spark33_decimal_orc column type? > An

[jira] [Assigned] (SPARK-39851) Improve join stats estimation if one side can keep uniqueness

[ https://issues.apache.org/jira/browse/SPARK-39851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39851: --- Assignee: Wan Kun > Improve join stats estimation if one side can keep uniqueness >

[jira] [Updated] (SPARK-42450) dataset.where() omit quotes if where IN clause has more than 10 operands

[ https://issues.apache.org/jira/browse/SPARK-42450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42450: Fix Version/s: (was: 3.4.0) (was: 3.3.2) > dataset.where() omit quotes

[jira] [Updated] (SPARK-42450) dataset.where() omit quotes if where IN clause has more than 10 operands

[ https://issues.apache.org/jira/browse/SPARK-42450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42450: Target Version/s: (was: 3.3.0) > dataset.where() omit quotes if where IN clause has more than

[jira] [Assigned] (SPARK-42451) Remove 3.1 and Java 17 check from filter condition of `testingVersions` in `HiveExternalCatalogVersionsSuite`

[ https://issues.apache.org/jira/browse/SPARK-42451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42451: --- Assignee: Yang Jie > Remove 3.1 and Java 17 check from filter condition of

[jira] [Resolved] (SPARK-42451) Remove 3.1 and Java 17 check from filter condition of `testingVersions` in `HiveExternalCatalogVersionsSuite`

[ https://issues.apache.org/jira/browse/SPARK-42451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42451. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40039

[jira] [Resolved] (SPARK-42346) distinct(count colname) with UNION ALL causes query analyzer bug

[ https://issues.apache.org/jira/browse/SPARK-42346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42346. - Fix Version/s: 3.3.2 3.4.0 3.5.0 Assignee: Peter

[jira] [Commented] (SPARK-42346) distinct(count colname) with UNION ALL causes query analyzer bug

[ https://issues.apache.org/jira/browse/SPARK-42346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684177#comment-17684177 ] Yuming Wang commented on SPARK-42346: - cc [~petertoth] > distinct(count colname) with UNION ALL

[jira] [Created] (SPARK-42333) Change log level to debug when fetching result set from SparkExecuteStatementOperation

Yuming Wang created SPARK-42333: --- Summary: Change log level to debug when fetching result set from SparkExecuteStatementOperation Key: SPARK-42333 URL: https://issues.apache.org/jira/browse/SPARK-42333

[jira] [Updated] (SPARK-14922) Alter Table Drop Partition Using Predicate-based Partition Spec

[ https://issues.apache.org/jira/browse/SPARK-14922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-14922: Target Version/s: 3.5.0 (was: 3.2.0) > Alter Table Drop Partition Using Predicate-based

[jira] [Reopened] (SPARK-40303) The performance will be worse after codegen

[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-40303: - > The performance will be worse after codegen > --- > >

[jira] [Resolved] (SPARK-42063) Register `byte[][]` to KyroSerializer

[ https://issues.apache.org/jira/browse/SPARK-42063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42063. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39569

[jira] [Assigned] (SPARK-42063) Register `byte[][]` to KyroSerializer

[ https://issues.apache.org/jira/browse/SPARK-42063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42063: --- Assignee: Dongjoon Hyun > Register `byte[][]` to KyroSerializer >

[jira] [Created] (SPARK-42064) Implement bloom filter join hint

Yuming Wang created SPARK-42064: --- Summary: Implement bloom filter join hint Key: SPARK-42064 URL: https://issues.apache.org/jira/browse/SPARK-42064 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-39217) Makes DPP support the pruning side has Union

[ https://issues.apache.org/jira/browse/SPARK-39217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-39217: --- Assignee: Wan Kun > Makes DPP support the pruning side has Union >

[jira] [Resolved] (SPARK-39217) Makes DPP support the pruning side has Union

[ https://issues.apache.org/jira/browse/SPARK-39217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-39217. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39460

[jira] [Created] (SPARK-41986) Introduce shuffle on SinglePartition

Yuming Wang created SPARK-41986: --- Summary: Introduce shuffle on SinglePartition Key: SPARK-41986 URL: https://issues.apache.org/jira/browse/SPARK-41986 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-41741) [SQL] ParquetFilters StringStartsWith push down matching string do not use UTF-8

[ https://issues.apache.org/jira/browse/SPARK-41741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655893#comment-17655893 ] Yuming Wang commented on SPARK-41741: - What is your env? you can put the env in your terminal. >

[jira] [Commented] (SPARK-41741) [SQL] ParquetFilters StringStartsWith push down matching string do not use UTF-8

[ https://issues.apache.org/jira/browse/SPARK-41741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655882#comment-17655882 ] Yuming Wang commented on SPARK-41741: - What is your {{file.encoding}}? > [SQL] ParquetFilters

[jira] [Updated] (SPARK-41498) Union does not propagate Metadata output

[ https://issues.apache.org/jira/browse/SPARK-41498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-41498: Fix Version/s: (was: 3.4.0) > Union does not propagate Metadata output >

[jira] [Commented] (SPARK-41459) Spark Thrift Server operation log output is empty

[ https://issues.apache.org/jira/browse/SPARK-41459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645111#comment-17645111 ] Yuming Wang commented on SPARK-41459: - cc [~LuciferYang] > Spark Thrift Server operation log output

[jira] [Resolved] (SPARK-34987) AQE improve: change shuffle hash join to sort merge join when skewed shuffle hash join exists

[ https://issues.apache.org/jira/browse/SPARK-34987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-34987. - Resolution: Not A Problem > AQE improve: change shuffle hash join to sort merge join when

[jira] [Resolved] (SPARK-41167) Optimize LikeSimplification rule to improve multi like performance

[ https://issues.apache.org/jira/browse/SPARK-41167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-41167. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38682

[jira] [Assigned] (SPARK-41167) Optimize LikeSimplification rule to improve multi like performance

[ https://issues.apache.org/jira/browse/SPARK-41167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-41167: --- Assignee: Wan Kun > Optimize LikeSimplification rule to improve multi like performance >

[jira] [Commented] (SPARK-41336) BroadcastExchange does not support the execute() code path. when AQE enabled

[ https://issues.apache.org/jira/browse/SPARK-41336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641191#comment-17641191 ] Yuming Wang commented on SPARK-41336: - Could you use Spark 3.3.1? > BroadcastExchange does not

[jira] [Commented] (SPARK-41324) Follow-up on JDK-8180450

[ https://issues.apache.org/jira/browse/SPARK-41324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641155#comment-17641155 ] Yuming Wang commented on SPARK-41324: - cc [~LuciferYang] > Follow-up on JDK-8180450 >

[jira] [Commented] (SPARK-41299) OOM when filter pushdown `last_day` function

[ https://issues.apache.org/jira/browse/SPARK-41299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640347#comment-17640347 ] Yuming Wang commented on SPARK-41299: - Do you have the query plan? > OOM when filter pushdown

[jira] [Commented] (SPARK-41219) Regression in IntegralDivide returning null instead of 0

[ https://issues.apache.org/jira/browse/SPARK-41219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637102#comment-17637102 ] Yuming Wang commented on SPARK-41219: - cc [~ulysses] > Regression in IntegralDivide returning null

[jira] [Commented] (SPARK-41207) Regression in IntegralDivide

[ https://issues.apache.org/jira/browse/SPARK-41207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17636383#comment-17636383 ] Yuming Wang commented on SPARK-41207: - cc [~ulysses] > Regression in IntegralDivide >

[jira] [Updated] (SPARK-41207) Regression in IntegralDivide

[ https://issues.apache.org/jira/browse/SPARK-41207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-41207: Target Version/s: (was: 3.4.0) > Regression in IntegralDivide > >

[jira] [Updated] (SPARK-41207) Regression in IntegralDivide

[ https://issues.apache.org/jira/browse/SPARK-41207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-41207: Fix Version/s: (was: 3.4.0) > Regression in IntegralDivide > > >

[jira] [Resolved] (SPARK-41017) Support column pruning with multiple nondeterministic Filters

[ https://issues.apache.org/jira/browse/SPARK-41017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-41017. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38511

[jira] [Updated] (SPARK-41141) avoid introducing a new aggregate expression in the analysis phase when subquery is referencing it

[ https://issues.apache.org/jira/browse/SPARK-41141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-41141: Target Version/s: (was: 3.3.1) > avoid introducing a new aggregate expression in the analysis

[jira] [Created] (SPARK-41088) Add PartialAggregate and FinalAggregate logic operators

Yuming Wang created SPARK-41088: --- Summary: Add PartialAggregate and FinalAggregate logic operators Key: SPARK-41088 URL: https://issues.apache.org/jira/browse/SPARK-41088 Project: Spark Issue

<    1   2   3   4   5   6   7   8   9   10   >