[jira] [Updated] (SPARK-47322) Make `withColumnsRenamed` duplicated column name handling consisten with `withColumnRenamed`

[ https://issues.apache.org/jira/browse/SPARK-47322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47322: --- Labels: pull-request-available (was: ) > Make `withColumnsRenamed` duplicated column name

[jira] [Created] (SPARK-47322) Make `withColumnsRenamed` duplicated column name handling consisten with `withColumnRenamed`

Ruifeng Zheng created SPARK-47322: - Summary: Make `withColumnsRenamed` duplicated column name handling consisten with `withColumnRenamed` Key: SPARK-47322 URL: https://issues.apache.org/jira/browse/SPARK-47322

[jira] [Resolved] (SPARK-47319) Fix missingInput calculation

[ https://issues.apache.org/jira/browse/SPARK-47319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47319. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45424

[jira] [Assigned] (SPARK-47319) Fix missingInput calculation

[ https://issues.apache.org/jira/browse/SPARK-47319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-47319: Assignee: Peter Toth > Fix missingInput calculation > > >

[jira] [Assigned] (SPARK-47314) [DOC] Correct the ExternalSorter#writePartitionedMapOutput method comment

[ https://issues.apache.org/jira/browse/SPARK-47314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-47314: Assignee: Binjie Yang > [DOC] Correct the ExternalSorter#writePartitionedMapOutput method

[jira] [Resolved] (SPARK-47314) [DOC] Correct the ExternalSorter#writePartitionedMapOutput method comment

[ https://issues.apache.org/jira/browse/SPARK-47314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47314. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45415

[jira] [Commented] (SPARK-47320) Datasets involving self joins behave in an inconsistent and unintuitive manner

[ https://issues.apache.org/jira/browse/SPARK-47320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824589#comment-17824589 ] Asif commented on SPARK-47320: -- will be linking the bug to an open PR > Datasets involving self joins

[jira] [Created] (SPARK-47320) Datasets involving self joins behave in an inconsistent and unintuitive manner

Asif created SPARK-47320: Summary: Datasets involving self joins behave in an inconsistent and unintuitive manner Key: SPARK-47320 URL: https://issues.apache.org/jira/browse/SPARK-47320 Project: Spark

[jira] [Resolved] (SPARK-47309) [XML] Add schema inference unit tests

[ https://issues.apache.org/jira/browse/SPARK-47309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47309. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45426

[jira] [Assigned] (SPARK-47309) [XML] Add schema inference unit tests

[ https://issues.apache.org/jira/browse/SPARK-47309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47309: Assignee: Shujing Yang > [XML] Add schema inference unit tests >

[jira] [Assigned] (SPARK-47078) Documentation for SparkSession-based Profilers

[ https://issues.apache.org/jira/browse/SPARK-47078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47078: Assignee: Xinrong Meng > Documentation for SparkSession-based Profilers >

[jira] [Resolved] (SPARK-47078) Documentation for SparkSession-based Profilers

[ https://issues.apache.org/jira/browse/SPARK-47078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47078. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45269

[jira] [Commented] (SPARK-47288) DataType __repr__ change breaks datatype checking (anit-)pattern

[ https://issues.apache.org/jira/browse/SPARK-47288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824577#comment-17824577 ] Hyukjin Kwon commented on SPARK-47288: -- I agree in principle but my concern is more about backward

[jira] [Resolved] (SPARK-47276) Introduce `spark.profile.clear` for SparkSession-based profiling

[ https://issues.apache.org/jira/browse/SPARK-47276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-47276. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45378

[jira] [Comment Edited] (SPARK-36113) Unify the logic to handle COUNT bug for scalar and lateral subqueries

[ https://issues.apache.org/jira/browse/SPARK-36113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824521#comment-17824521 ] Jack Chen edited comment on SPARK-36113 at 3/7/24 8:52 PM: --- The tricky aspect

[jira] [Comment Edited] (SPARK-36113) Unify the logic to handle COUNT bug for scalar and lateral subqueries

[ https://issues.apache.org/jira/browse/SPARK-36113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824521#comment-17824521 ] Jack Chen edited comment on SPARK-36113 at 3/7/24 8:47 PM: --- The tricky aspect

[jira] [Comment Edited] (SPARK-36113) Unify the logic to handle COUNT bug for scalar and lateral subqueries

[ https://issues.apache.org/jira/browse/SPARK-36113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824521#comment-17824521 ] Jack Chen edited comment on SPARK-36113 at 3/7/24 8:26 PM: --- The tricky aspect

[jira] [Comment Edited] (SPARK-36113) Unify the logic to handle COUNT bug for scalar and lateral subqueries

[ https://issues.apache.org/jira/browse/SPARK-36113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824521#comment-17824521 ] Jack Chen edited comment on SPARK-36113 at 3/7/24 8:09 PM: --- The tricky aspect

[jira] [Commented] (SPARK-36113) Unify the logic to handle COUNT bug for scalar and lateral subqueries

[ https://issues.apache.org/jira/browse/SPARK-36113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824521#comment-17824521 ] Jack Chen commented on SPARK-36113: --- The tricky aspect of this is that if we do the count bug handling

[jira] [Updated] (SPARK-47318) AuthEngine key exchange needs additional KDF round

[ https://issues.apache.org/jira/browse/SPARK-47318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47318: --- Labels: pull-request-available (was: ) > AuthEngine key exchange needs additional KDF

[jira] [Updated] (SPARK-47319) Fix missingInput calculation

[ https://issues.apache.org/jira/browse/SPARK-47319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47319: --- Labels: pull-request-available (was: ) > Fix missingInput calculation >

[jira] [Created] (SPARK-47319) Fix missingInput calculation

Peter Toth created SPARK-47319: -- Summary: Fix missingInput calculation Key: SPARK-47319 URL: https://issues.apache.org/jira/browse/SPARK-47319 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-47318) AuthEngine key exchange needs additional KDF round

[ https://issues.apache.org/jira/browse/SPARK-47318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Weis updated SPARK-47318: --- Description: AuthEngine implements a bespoke [key exchange protocol

[jira] [Created] (SPARK-47318) AuthEngine key exchange needs additional KDF round

Steve Weis created SPARK-47318: -- Summary: AuthEngine key exchange needs additional KDF round Key: SPARK-47318 URL: https://issues.apache.org/jira/browse/SPARK-47318 Project: Spark Issue Type:

[jira] [Commented] (SPARK-47296) fail all unsupported functions

[ https://issues.apache.org/jira/browse/SPARK-47296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824484#comment-17824484 ] Uroš Bojanić commented on SPARK-47296: -- Hi [~gpgp]! Welcome to the community, and thanks for your

[jira] [Comment Edited] (SPARK-47296) fail all unsupported functions

[ https://issues.apache.org/jira/browse/SPARK-47296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824454#comment-17824454 ] Gideon P edited comment on SPARK-47296 at 3/7/24 4:42 PM: -- [~uros-db] can I

[jira] [Updated] (SPARK-47296) fail all unsupported functions

[ https://issues.apache.org/jira/browse/SPARK-47296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47296: --- Labels: pull-request-available (was: ) > fail all unsupported functions >

[jira] [Commented] (SPARK-47296) fail all unsupported functions

[ https://issues.apache.org/jira/browse/SPARK-47296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824454#comment-17824454 ] Gideon P commented on SPARK-47296: -- [~uros-db] can I raise a pr to implement this one? Long time spark

[jira] [Updated] (SPARK-47295) startswith, endswith (non-binary collations)

[ https://issues.apache.org/jira/browse/SPARK-47295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47295: --- Labels: pull-request-available (was: ) > startswith, endswith (non-binary collations) >

[jira] [Assigned] (SPARK-47248) contains (non-binary collations)

[ https://issues.apache.org/jira/browse/SPARK-47248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47248: --- Assignee: Uroš Bojanić > contains (non-binary collations) >

[jira] [Resolved] (SPARK-47248) contains (non-binary collations)

[ https://issues.apache.org/jira/browse/SPARK-47248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47248. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45382

[jira] [Updated] (SPARK-46761) quoted strings in a JSON path should support ? characters

[ https://issues.apache.org/jira/browse/SPARK-46761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46761: --- Labels: pull-request-available (was: ) > quoted strings in a JSON path should support ?

[jira] [Resolved] (SPARK-47278) Upgrade rocksdbjni to 8.11.3

[ https://issues.apache.org/jira/browse/SPARK-47278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47278. -- Fix Version/s: 4.0.0 Assignee: Yang Jie Resolution: Fixed resolved by GitHub Pull

[jira] [Resolved] (SPARK-47298) Upgrade `mysql-connector-j` to `8.3.0` and `mariadb-java-client` to `2.7.12`

[ https://issues.apache.org/jira/browse/SPARK-47298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47298. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45399

[jira] [Assigned] (SPARK-47298) Upgrade `mysql-connector-j` to `8.3.0` and `mariadb-java-client` to `2.7.12`

[ https://issues.apache.org/jira/browse/SPARK-47298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-47298: Assignee: BingKun Pan > Upgrade `mysql-connector-j` to `8.3.0` and `mariadb-java-client` to

[jira] [Resolved] (SPARK-47300) quoteIfNeeded should quote identifier starts with digits

[ https://issues.apache.org/jira/browse/SPARK-47300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47300. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45401

[jira] [Assigned] (SPARK-47300) quoteIfNeeded should quote identifier starts with digits

[ https://issues.apache.org/jira/browse/SPARK-47300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-47300: Assignee: Wenchen Fan > quoteIfNeeded should quote identifier starts with digits >

[jira] [Created] (SPARK-47317) taskSummary API should include the task state as a parameter

jam xu created SPARK-47317: -- Summary: taskSummary API should include the task state as a parameter Key: SPARK-47317 URL: https://issues.apache.org/jira/browse/SPARK-47317 Project: Spark Issue

[jira] [Resolved] (SPARK-47301) Fix flaky ParquetIOSuite

[ https://issues.apache.org/jira/browse/SPARK-47301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47301. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45403

[jira] [Assigned] (SPARK-47301) Fix flaky ParquetIOSuite

[ https://issues.apache.org/jira/browse/SPARK-47301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-47301: Assignee: BingKun Pan > Fix flaky ParquetIOSuite > > >

[jira] [Resolved] (SPARK-47241) fix rule order issues for ExtractGenerator

[ https://issues.apache.org/jira/browse/SPARK-47241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47241. - Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-47241) fix rule order issues for ExtractGenerator

[ https://issues.apache.org/jira/browse/SPARK-47241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-47241: --- Assignee: Wenchen Fan > fix rule order issues for ExtractGenerator >

[jira] [Resolved] (SPARK-47315) tempView created in createTempView UT is not cleaned up

[ https://issues.apache.org/jira/browse/SPARK-47315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47315. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45417

[jira] [Assigned] (SPARK-47315) tempView created in createTempView UT is not cleaned up

[ https://issues.apache.org/jira/browse/SPARK-47315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-47315: Assignee: Zhen Wang > tempView created in createTempView UT is not cleaned up >

[jira] [Updated] (SPARK-47316) Fix TimestampNTZ in Postgres Array

[ https://issues.apache.org/jira/browse/SPARK-47316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47316: --- Labels: pull-request-available (was: ) > Fix TimestampNTZ in Postgres Array >

[jira] [Created] (SPARK-47316) Fix TimestampNTZ in Postgres Array

Kent Yao created SPARK-47316: Summary: Fix TimestampNTZ in Postgres Array Key: SPARK-47316 URL: https://issues.apache.org/jira/browse/SPARK-47316 Project: Spark Issue Type: Improvement