[jira] [Updated] (SPARK-48279) Upgrade ORC to 2.0.1

[ https://issues.apache.org/jira/browse/SPARK-48279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48279: --- Labels: pull-request-available (was: ) > Upgrade ORC to 2.0.1 > > >

[jira] [Created] (SPARK-48279) Upgrade ORC to 2.0.1

William Hyun created SPARK-48279: Summary: Upgrade ORC to 2.0.1 Key: SPARK-48279 URL: https://issues.apache.org/jira/browse/SPARK-48279 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-48105) Fix the data corruption issue when state store unload and snapshotting happens concurrently for HDFS state store

[ https://issues.apache.org/jira/browse/SPARK-48105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48105: - Fix Version/s: 3.5.2 > Fix the data corruption issue when state store unload and snapshotting

[jira] [Resolved] (SPARK-48233) Tests for non-stateful streaming with collations

[ https://issues.apache.org/jira/browse/SPARK-48233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48233. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46247

[jira] [Assigned] (SPARK-48100) [SQL][XML] Fix issues in skipping nested structure fields not selected in schema

[ https://issues.apache.org/jira/browse/SPARK-48100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48100: Assignee: Shujing Yang > [SQL][XML] Fix issues in skipping nested structure fields not

[jira] [Resolved] (SPARK-48100) [SQL][XML] Fix issues in skipping nested structure fields not selected in schema

[ https://issues.apache.org/jira/browse/SPARK-48100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48100. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46348

[jira] [Updated] (SPARK-48172) Fix escaping issues in JDBCDialects

[ https://issues.apache.org/jira/browse/SPARK-48172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-48172: - Fix Version/s: (was: 4.0.0) (was: 3.5.2) (was: 3.4.4)

[jira] [Updated] (SPARK-48276) Add the missing __repr__ method for SQLExpression

[ https://issues.apache.org/jira/browse/SPARK-48276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48276: -- Fix Version/s: 4.0.0 > Add the missing __repr__ method for SQLExpression >

[jira] [Updated] (SPARK-48220) Allow passing PyArrow Table to createDataFrame()

[ https://issues.apache.org/jira/browse/SPARK-48220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cook updated SPARK-48220: - Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Allow passing PyArrow Table to

[jira] [Updated] (SPARK-48271) Turn match error in RowEncoder into UNSUPPORTED_DATA_TYPE_FOR_ENCODER

[ https://issues.apache.org/jira/browse/SPARK-48271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-48271: Summary: Turn match error in RowEncoder into UNSUPPORTED_DATA_TYPE_FOR_ENCODER (was: support

[jira] [Updated] (SPARK-48278) Refine the string representation of `Cast`

[ https://issues.apache.org/jira/browse/SPARK-48278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48278: --- Labels: pull-request-available (was: ) > Refine the string representation of `Cast` >

[jira] [Created] (SPARK-48278) Refine the string representation of `Cast`

Ruifeng Zheng created SPARK-48278: - Summary: Refine the string representation of `Cast` Key: SPARK-48278 URL: https://issues.apache.org/jira/browse/SPARK-48278 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-48274) Upgrade GenJavadoc to 0.19

[ https://issues.apache.org/jira/browse/SPARK-48274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-48274. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46579

[jira] [Updated] (SPARK-48277) Improve error message for ErrorClassesJsonReader.getErrorMessage

[ https://issues.apache.org/jira/browse/SPARK-48277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48277: --- Labels: pull-request-available (was: ) > Improve error message for

[jira] [Created] (SPARK-48277) Improve error message for ErrorClassesJsonReader.getErrorMessage

Rui Wang created SPARK-48277: Summary: Improve error message for ErrorClassesJsonReader.getErrorMessage Key: SPARK-48277 URL: https://issues.apache.org/jira/browse/SPARK-48277 Project: Spark

[jira] [Updated] (SPARK-48276) Add the missing __repr__ method for SQLExpression

[ https://issues.apache.org/jira/browse/SPARK-48276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48276: --- Labels: pull-request-available (was: ) > Add the missing __repr__ method for SQLExpression

[jira] [Resolved] (SPARK-47599) MLLib: Migrate logWarn with variables to structured logging framework

[ https://issues.apache.org/jira/browse/SPARK-47599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-47599. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46527

[jira] [Resolved] (SPARK-48247) Use all values in a python dict when inferring MapType schema

[ https://issues.apache.org/jira/browse/SPARK-48247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48247. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46547

[jira] [Assigned] (SPARK-48247) Use all values in a python dict when inferring MapType schema

[ https://issues.apache.org/jira/browse/SPARK-48247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48247: Assignee: Hyukjin Kwon > Use all values in a python dict when inferring MapType schema >

[jira] [Updated] (SPARK-48274) Upgrade GenJavadoc to 0.19

[ https://issues.apache.org/jira/browse/SPARK-48274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48274: --- Labels: pull-request-available (was: ) > Upgrade GenJavadoc to 0.19 >

[jira] [Resolved] (SPARK-48263) Collate function support for non UTF8_BINARY strings

[ https://issues.apache.org/jira/browse/SPARK-48263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48263. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46574

[jira] [Assigned] (SPARK-48263) Collate function support for non UTF8_BINARY strings

[ https://issues.apache.org/jira/browse/SPARK-48263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48263: --- Assignee: Nebojsa Savic > Collate function support for non UTF8_BINARY strings >

[jira] [Resolved] (SPARK-48172) Fix escaping issues in JDBCDialects

[ https://issues.apache.org/jira/browse/SPARK-48172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48172. - Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution: Fixed

[jira] [Created] (SPARK-48275) array_sort and sort_array fail for structs containing any unorderable fields

Matt Braymer-Hayes created SPARK-48275: -- Summary: array_sort and sort_array fail for structs containing any unorderable fields Key: SPARK-48275 URL: https://issues.apache.org/jira/browse/SPARK-48275

[jira] [Updated] (SPARK-48273) Late rewrite of PlanWithUnresolvedIdentifier

[ https://issues.apache.org/jira/browse/SPARK-48273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48273: --- Labels: pull-request-available (was: ) > Late rewrite of PlanWithUnresolvedIdentifier >

[jira] [Created] (SPARK-48273) Late rewrite of PlanWithUnresolvedIdentifier

Nikola Mandic created SPARK-48273: - Summary: Late rewrite of PlanWithUnresolvedIdentifier Key: SPARK-48273 URL: https://issues.apache.org/jira/browse/SPARK-48273 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846293#comment-17846293 ] gaoyajun02 commented on SPARK-42694: Have you enabled push-based shuffle? > Data duplication and

[jira] [Updated] (SPARK-48272) Add function `timestamp_diff`

[ https://issues.apache.org/jira/browse/SPARK-48272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48272: --- Labels: pull-request-available (was: ) > Add function `timestamp_diff` >

[jira] [Created] (SPARK-48272) Add function `timestamp_diff`

Ruifeng Zheng created SPARK-48272: - Summary: Add function `timestamp_diff` Key: SPARK-48272 URL: https://issues.apache.org/jira/browse/SPARK-48272 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-48263) Collate function support for non UTF8_BINARY strings

[ https://issues.apache.org/jira/browse/SPARK-48263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-48263: -- Summary: Collate function support for non UTF8_BINARY strings (was: Collate

[jira] [Resolved] (SPARK-48155) PropagateEmpty relation cause LogicalQueryStage only with broadcast without join then execute failed

[ https://issues.apache.org/jira/browse/SPARK-48155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48155. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46523

[jira] [Assigned] (SPARK-48155) PropagateEmpty relation cause LogicalQueryStage only with broadcast without join then execute failed

[ https://issues.apache.org/jira/browse/SPARK-48155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48155: --- Assignee: angerszhu > PropagateEmpty relation cause LogicalQueryStage only with broadcast

[jira] [Updated] (SPARK-48271) support char/varchar in RowEncoder

[ https://issues.apache.org/jira/browse/SPARK-48271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48271: --- Labels: pull-request-available (was: ) > support char/varchar in RowEncoder >

[jira] [Created] (SPARK-48271) support char/varchar in RowEncoder

Wenchen Fan created SPARK-48271: --- Summary: support char/varchar in RowEncoder Key: SPARK-48271 URL: https://issues.apache.org/jira/browse/SPARK-48271 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-48221) Alter string search logic for UTF8_BINARY_LCASE collation

[ https://issues.apache.org/jira/browse/SPARK-48221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48221: -- Assignee: (was: Apache Spark) > Alter string search logic for UTF8_BINARY_LCASE

[jira] [Assigned] (SPARK-47415) Levenshtein (all collations)

[ https://issues.apache.org/jira/browse/SPARK-47415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47415: -- Assignee: Apache Spark > Levenshtein (all collations) >

[jira] [Assigned] (SPARK-48221) Alter string search logic for UTF8_BINARY_LCASE collation

[ https://issues.apache.org/jira/browse/SPARK-48221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48221: -- Assignee: Apache Spark > Alter string search logic for UTF8_BINARY_LCASE collation >

[jira] [Assigned] (SPARK-47415) Levenshtein (all collations)

[ https://issues.apache.org/jira/browse/SPARK-47415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47415: -- Assignee: (was: Apache Spark) > Levenshtein (all collations) >

[jira] [Updated] (SPARK-48263) Collate expression not working when default collation set

[ https://issues.apache.org/jira/browse/SPARK-48263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48263: --- Labels: pull-request-available (was: ) > Collate expression not working when default

[jira] [Updated] (SPARK-48263) Collate expression not working when default collation set

[ https://issues.apache.org/jira/browse/SPARK-48263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-48263: -- Epic Link: SPARK-46830 > Collate expression not working when default collation set >

[jira] [Updated] (SPARK-48026) Promote trunc from a datetime only function to a datetime function

[ https://issues.apache.org/jira/browse/SPARK-48026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-48026: - Issue Type: Improvement (was: Bug) > Promote trunc from a datetime only function to a datetime >

[jira] [Updated] (SPARK-48269) DB2: Document Mapping Spark SQL Data Types from DB2 and add tests

[ https://issues.apache.org/jira/browse/SPARK-48269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48269: --- Labels: pull-request-available (was: ) > DB2: Document Mapping Spark SQL Data Types from

[jira] [Created] (SPARK-48269) DB2: Document Mapping Spark SQL Data Types from DB2 and add tests

Kent Yao created SPARK-48269: Summary: DB2: Document Mapping Spark SQL Data Types from DB2 and add tests Key: SPARK-48269 URL: https://issues.apache.org/jira/browse/SPARK-48269 Project: Spark

[jira] [Updated] (SPARK-48267) Regression e2e test with SPARK-47305

[ https://issues.apache.org/jira/browse/SPARK-48267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-48267: - Fix Version/s: 3.5.2 > Regression e2e test with SPARK-47305 >

[jira] [Resolved] (SPARK-48267) Regression e2e test with SPARK-47305

[ https://issues.apache.org/jira/browse/SPARK-48267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48267. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46569

[jira] [Assigned] (SPARK-48267) Regression e2e test with SPARK-47305

[ https://issues.apache.org/jira/browse/SPARK-48267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48267: Assignee: Jungtaek Lim > Regression e2e test with SPARK-47305 >

[jira] [Resolved] (SPARK-48157) CSV expressions (all collations)

[ https://issues.apache.org/jira/browse/SPARK-48157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48157. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46504

[jira] [Assigned] (SPARK-48157) CSV expressions (all collations)

[ https://issues.apache.org/jira/browse/SPARK-48157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48157: --- Assignee: Uroš Bojanić > CSV expressions (all collations) >

[jira] [Updated] (SPARK-48157) CSV expressions (all collations)

[ https://issues.apache.org/jira/browse/SPARK-48157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48157: --- Labels: pull-request-available (was: ) > CSV expressions (all collations) >

[jira] [Resolved] (SPARK-48229) inputFile expressions (all collations)

[ https://issues.apache.org/jira/browse/SPARK-48229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48229. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46503