[jira] [Resolved] (SPARK-48846) Fix the incorrect namings and missing params in func docs in `builtin.py`

[ https://issues.apache.org/jira/browse/SPARK-48846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48846. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47345

[jira] [Assigned] (SPARK-48846) Fix the incorrect namings and missing params in func docs in `builtin.py`

[ https://issues.apache.org/jira/browse/SPARK-48846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48846: Assignee: Wei Guo > Fix the incorrect namings and missing params in func docs in

[jira] [Assigned] (SPARK-48897) make `from_xml` support StructType schema

[ https://issues.apache.org/jira/browse/SPARK-48897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48897: Assignee: Ruifeng Zheng > make `from_xml` support StructType schema >

[jira] [Resolved] (SPARK-48897) make `from_xml` support StructType schema

[ https://issues.apache.org/jira/browse/SPARK-48897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48897. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47355

[jira] [Resolved] (SPARK-45190) XML: StructType schema issue in pyspark connect

[ https://issues.apache.org/jira/browse/SPARK-45190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45190. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47355

[jira] [Assigned] (SPARK-45190) XML: StructType schema issue in pyspark connect

[ https://issues.apache.org/jira/browse/SPARK-45190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45190: Assignee: Sandip Agarwala > XML: StructType schema issue in pyspark connect >

[jira] [Updated] (SPARK-45190) XML: StructType schema issue in pyspark connect

[ https://issues.apache.org/jira/browse/SPARK-45190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45190: --- Labels: pull-request-available (was: ) > XML: StructType schema issue in pyspark connect >

[jira] [Created] (SPARK-48906) Introduce `SHOW COLLATIONS LIKE ...` syntax to show all collations

BingKun Pan created SPARK-48906: --- Summary: Introduce `SHOW COLLATIONS LIKE ...` syntax to show all collations Key: SPARK-48906 URL: https://issues.apache.org/jira/browse/SPARK-48906 Project: Spark

[jira] [Created] (SPARK-48905) Add a guideline for version updates in DOC and various API DOCs

Kent Yao created SPARK-48905: Summary: Add a guideline for version updates in DOC and various API DOCs Key: SPARK-48905 URL: https://issues.apache.org/jira/browse/SPARK-48905 Project: Spark

[jira] [Resolved] (SPARK-48884) Remove unused helper function `PythonSQLUtils.makeInterval`

[ https://issues.apache.org/jira/browse/SPARK-48884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48884. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47330

[jira] [Created] (SPARK-48904) Update doc's version field to align with fixedVersion field of JIRA ticket

Kent Yao created SPARK-48904: Summary: Update doc's version field to align with fixedVersion field of JIRA ticket Key: SPARK-48904 URL: https://issues.apache.org/jira/browse/SPARK-48904 Project: Spark

[jira] [Resolved] (SPARK-48885) Make some inheritances of RuntimeReplaceable override replacement to lazy val

[ https://issues.apache.org/jira/browse/SPARK-48885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-48885. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47333

[jira] [Assigned] (SPARK-48885) Make some inheritances of RuntimeReplaceable override replacement to lazy val

[ https://issues.apache.org/jira/browse/SPARK-48885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-48885: Assignee: Kent Yao > Make some inheritances of RuntimeReplaceable override replacement to lazy

[jira] [Updated] (SPARK-48903) Update latestSnapshot version correctly on remote load

[ https://issues.apache.org/jira/browse/SPARK-48903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48903: --- Labels: pull-request-available (was: ) > Update latestSnapshot version correctly on remote

[jira] [Assigned] (SPARK-45155) Add API Docs

[ https://issues.apache.org/jira/browse/SPARK-45155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45155: Assignee: Venkata Sai Akhil Gudesa > Add API Docs > > >

[jira] [Resolved] (SPARK-45155) Add API Docs

[ https://issues.apache.org/jira/browse/SPARK-45155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45155. -- Resolution: Fixed Issue resolved by pull request 47332

[jira] [Created] (SPARK-48903) Update latestSnapshot version correctly on remote load

Anish Shrigondekar created SPARK-48903: -- Summary: Update latestSnapshot version correctly on remote load Key: SPARK-48903 URL: https://issues.apache.org/jira/browse/SPARK-48903 Project: Spark

[jira] [Assigned] (SPARK-48350) [M0] Support for exceptions thrown from parser/interpreter

[ https://issues.apache.org/jira/browse/SPARK-48350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48350: --- Assignee: Milan Dankovic > [M0] Support for exceptions thrown from parser/interpreter >

[jira] [Resolved] (SPARK-48350) [M0] Support for exceptions thrown from parser/interpreter

[ https://issues.apache.org/jira/browse/SPARK-48350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48350. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47147

[jira] [Updated] (SPARK-48900) Add `reason` string to all job / stage / job group cancellation calls

[ https://issues.apache.org/jira/browse/SPARK-48900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48900: --- Labels: pull-request-available (was: ) > Add `reason` string to all job / stage / job

[jira] [Created] (SPARK-48901) Add clusterBy DataStreamWriter API for Scala

Jiaheng Tang created SPARK-48901: Summary: Add clusterBy DataStreamWriter API for Scala Key: SPARK-48901 URL: https://issues.apache.org/jira/browse/SPARK-48901 Project: Spark Issue Type:

[jira] [Updated] (SPARK-48761) Add clusterBy DataFrameWriter API for Scala

[ https://issues.apache.org/jira/browse/SPARK-48761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaheng Tang updated SPARK-48761: - Description: Add a new `clusterBy` DataFrameWriter API for Scala. This allows users to interact

[jira] [Updated] (SPARK-48762) Add clusterBy DataFrameWriter API for Python

[ https://issues.apache.org/jira/browse/SPARK-48762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaheng Tang updated SPARK-48762: - Summary: Add clusterBy DataFrameWriter API for Python (was: Add clusterBy DataFrame API for

[jira] [Updated] (SPARK-48762) Add clusterBy DataFrameWriter API for Python

[ https://issues.apache.org/jira/browse/SPARK-48762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaheng Tang updated SPARK-48762: - Description: Add a new `clusterBy` DataFrameWriter API for Python. This allows users to

[jira] [Updated] (SPARK-48761) Add clusterBy DataFrameWriter API for Scala

[ https://issues.apache.org/jira/browse/SPARK-48761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaheng Tang updated SPARK-48761: - Summary: Add clusterBy DataFrameWriter API for Scala (was: Add clusterBy DataFrame API for

[jira] [Updated] (SPARK-48900) Add `reason` string to all job / stage / job group cancellation calls

[ https://issues.apache.org/jira/browse/SPARK-48900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingkang Li updated SPARK-48900: Description: Today it is difficult to determine _why_ a job, stage, or job group was canceled.

[jira] [Updated] (SPARK-48900) Add `reason` string to all job / stage / job group cancellation calls

[ https://issues.apache.org/jira/browse/SPARK-48900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingkang Li updated SPARK-48900: Affects Version/s: 4.0.0 (was: 3.4.3) > Add `reason` string to all job

[jira] [Updated] (SPARK-48900) Add `reason` string to all job / stage / job group cancellation calls

[ https://issues.apache.org/jira/browse/SPARK-48900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingkang Li updated SPARK-48900: Description: Today it is difficult to determine _why_ a job, stage, or job group was canceled.

[jira] [Assigned] (SPARK-48886) Add version info for changelog format v2

[ https://issues.apache.org/jira/browse/SPARK-48886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-48886: Assignee: Anish Shrigondekar > Add version info for changelog format v2 >

[jira] [Resolved] (SPARK-48886) Add version info for changelog format v2

[ https://issues.apache.org/jira/browse/SPARK-48886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-48886. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47336

[jira] [Created] (SPARK-48900) Add `reason` string to all job / stage / job group cancellation calls

Mingkang Li created SPARK-48900: --- Summary: Add `reason` string to all job / stage / job group cancellation calls Key: SPARK-48900 URL: https://issues.apache.org/jira/browse/SPARK-48900 Project: Spark

[jira] [Updated] (SPARK-48861) Cleanup shuffle dependencies for all SQL executions

[ https://issues.apache.org/jira/browse/SPARK-48861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48861: --- Labels: pull-request-available (was: ) > Cleanup shuffle dependencies for all SQL

[jira] [Updated] (SPARK-48891) Refactor state schema compatibility checker to handle all state schema formats

[ https://issues.apache.org/jira/browse/SPARK-48891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48891: --- Labels: pull-request-available (was: ) > Refactor state schema compatibility checker to

[jira] [Resolved] (SPARK-48899) Fix `ENV` key value format in K8s Dockerfiles

[ https://issues.apache.org/jira/browse/SPARK-48899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48899. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47357

[jira] [Assigned] (SPARK-48899) Fix `ENV` key value format in K8s Dockerfiles

[ https://issues.apache.org/jira/browse/SPARK-48899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48899: - Assignee: Dongjoon Hyun > Fix `ENV` key value format in K8s Dockerfiles >

[jira] [Updated] (SPARK-48899) Fix `ENV` key value format in K8s Dockerfiles

[ https://issues.apache.org/jira/browse/SPARK-48899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48899: --- Labels: pull-request-available (was: ) > Fix `ENV` key value format in K8s Dockerfiles >

[jira] [Created] (SPARK-48899) Fix `ENV` key value format in K8s Dockerfiles

Dongjoon Hyun created SPARK-48899: - Summary: Fix `ENV` key value format in K8s Dockerfiles Key: SPARK-48899 URL: https://issues.apache.org/jira/browse/SPARK-48899 Project: Spark Issue Type:

[jira] [Updated] (SPARK-48353) [M0] Support for TRY/CATCH statement

[ https://issues.apache.org/jira/browse/SPARK-48353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48353: --- Labels: pull-request-available (was: ) > [M0] Support for TRY/CATCH statement >

[jira] [Updated] (SPARK-48897) make `from_xml` support StructType schema

[ https://issues.apache.org/jira/browse/SPARK-48897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48897: --- Labels: pull-request-available (was: ) > make `from_xml` support StructType schema >

[jira] [Created] (SPARK-48897) make `from_xml` support StructType schema

Ruifeng Zheng created SPARK-48897: - Summary: make `from_xml` support StructType schema Key: SPARK-48897 URL: https://issues.apache.org/jira/browse/SPARK-48897 Project: Spark Issue Type:

[jira] [Updated] (SPARK-48881) when writing dynamic partitions, some dynamic partitions in InsertIntoHadoopFsRelationCommand can be compensated to specific partition values.

[ https://issues.apache.org/jira/browse/SPARK-48881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48881: --- Labels: pull-request-available (was: ) > when writing dynamic partitions, some dynamic

[jira] [Resolved] (SPARK-48441) Alter string search logic for: trim (UTF8_BINARY_LCASE)

[ https://issues.apache.org/jira/browse/SPARK-48441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48441. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46762

[jira] [Assigned] (SPARK-48441) Alter string search logic for: trim (UTF8_BINARY_LCASE)

[ https://issues.apache.org/jira/browse/SPARK-48441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48441: --- Assignee: Uroš Bojanić > Alter string search logic for: trim (UTF8_BINARY_LCASE) >