[jira] [Updated] (SPARK-45053) Improve python version mismatch logging

[ https://issues.apache.org/jira/browse/SPARK-45053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Liu updated SPARK-45053: Description: Currently the syntax of the python version mismatching is a little bit confusing, it uses

[jira] [Updated] (SPARK-45059) Add try_reflect to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-45059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jia Fan updated SPARK-45059: Summary: Add try_reflect to Scala and Python (was: Add to_reflect to Scala and Python) > Add

[jira] [Commented] (SPARK-45059) Add to_reflect to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-45059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761434#comment-17761434 ] Jia Fan commented on SPARK-45059: - I'm working on it > Add to_reflect to Scala and Python >

[jira] [Created] (SPARK-45059) Add to_reflect to Scala and Python

Jia Fan created SPARK-45059: --- Summary: Add to_reflect to Scala and Python Key: SPARK-45059 URL: https://issues.apache.org/jira/browse/SPARK-45059 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-45036) SPJ: Refactor logic to handle partially clustered distribution

[ https://issues.apache.org/jira/browse/SPARK-45036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761432#comment-17761432 ] Snoot.io commented on SPARK-45036: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-45036) SPJ: Refactor logic to handle partially clustered distribution

[ https://issues.apache.org/jira/browse/SPARK-45036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761433#comment-17761433 ] Snoot.io commented on SPARK-45036: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-45042) Upgrade jetty to 9.4.52.v20230823

[ https://issues.apache.org/jira/browse/SPARK-45042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761431#comment-17761431 ] Snoot.io commented on SPARK-45042: -- User 'panbingkun' has created a pull request for this issue:

[jira] [Updated] (SPARK-44890) Miswritten remarks in pom file

[ https://issues.apache.org/jira/browse/SPARK-44890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-44890: - Priority: Trivial (was: Minor) > Miswritten remarks in pom file >

[jira] [Commented] (SPARK-45053) Improve python version mismatch logging

[ https://issues.apache.org/jira/browse/SPARK-45053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761428#comment-17761428 ] Sean R. Owen commented on SPARK-45053: -- [~WweiL] Please fill out this JIRA. It is also not "Major"

[jira] [Commented] (SPARK-45053) Improve python version mismatch logging

[ https://issues.apache.org/jira/browse/SPARK-45053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761430#comment-17761430 ] Snoot.io commented on SPARK-45053: -- User 'WweiL' has created a pull request for this issue:

[jira] [Updated] (SPARK-45053) Improve python version mismatch logging

[ https://issues.apache.org/jira/browse/SPARK-45053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-45053: - Description: (was: [~WweiL] Please fill out this JIRA. It is also not "Major") > Improve

[jira] [Updated] (SPARK-45053) Improve python version mismatch logging

[ https://issues.apache.org/jira/browse/SPARK-45053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-45053: - Affects Version/s: 3.4.1 (was: 4.0.0) Description: [~WweiL]

[jira] [Commented] (SPARK-45053) Improve python version mismatch logging

[ https://issues.apache.org/jira/browse/SPARK-45053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761429#comment-17761429 ] Snoot.io commented on SPARK-45053: -- User 'WweiL' has created a pull request for this issue:

[jira] [Updated] (SPARK-45054) HiveExternalCatalog.listPartitions should restore Spark SQL stats

[ https://issues.apache.org/jira/browse/SPARK-45054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-45054: - Affects Version/s: 3.4.1 3.3.2 3.2.4

[jira] [Updated] (SPARK-45054) HiveExternalCatalog.listPartitions should restore Spark SQL stats

[ https://issues.apache.org/jira/browse/SPARK-45054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-45054: - Fix Version/s: 3.4.2 3.5.0 > HiveExternalCatalog.listPartitions should restore Spark

[jira] [Resolved] (SPARK-45054) HiveExternalCatalog.listPartitions should restore Spark SQL stats

[ https://issues.apache.org/jira/browse/SPARK-45054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-45054. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42777

[jira] [Assigned] (SPARK-45054) HiveExternalCatalog.listPartitions should restore Spark SQL stats

[ https://issues.apache.org/jira/browse/SPARK-45054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-45054: Assignee: Chao Sun > HiveExternalCatalog.listPartitions should restore Spark SQL stats >

[jira] [Resolved] (SPARK-44901) Add API in 'analyze' method to return partitioning/ordering expressions

[ https://issues.apache.org/jira/browse/SPARK-44901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44901. --- Fix Version/s: 4.0.0 Assignee: Daniel Resolution: Fixed Issue resolved by

[jira] [Created] (SPARK-45058) Refine the docstring of `DataFrame.distinct`

Allison Wang created SPARK-45058: Summary: Refine the docstring of `DataFrame.distinct` Key: SPARK-45058 URL: https://issues.apache.org/jira/browse/SPARK-45058 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45058) Refine docstring of `DataFrame.distinct`

[ https://issues.apache.org/jira/browse/SPARK-45058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-45058: - Summary: Refine docstring of `DataFrame.distinct` (was: Refine the docstring of

[jira] [Updated] (SPARK-45057) Deadlock caused by rdd replication level of 2

[ https://issues.apache.org/jira/browse/SPARK-45057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongwei Zhu updated SPARK-45057: - Description:   When 2 tasks try to compute same rdd with replication level of 2 and running on

[jira] [Created] (SPARK-45057) Deadlock caused by rdd replication level of 2

Zhongwei Zhu created SPARK-45057: Summary: Deadlock caused by rdd replication level of 2 Key: SPARK-45057 URL: https://issues.apache.org/jira/browse/SPARK-45057 Project: Spark Issue Type:

[jira] [Commented] (SPARK-44851) Update SparkConnectClientParser usage() method to match implementation

[ https://issues.apache.org/jira/browse/SPARK-44851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761410#comment-17761410 ] Harish Gontu commented on SPARK-44851: -- Can i pick up this task ? > Update

[jira] [Created] (SPARK-45056) Add process termination tests for Python foreachBatch and StreamingQueryListener

Wei Liu created SPARK-45056: --- Summary: Add process termination tests for Python foreachBatch and StreamingQueryListener Key: SPARK-45056 URL: https://issues.apache.org/jira/browse/SPARK-45056 Project:

[jira] [Created] (SPARK-45055) Do not transpose windows if they conflict on ORDER BY / PROJECT clauses

Andrey Gubichev created SPARK-45055: --- Summary: Do not transpose windows if they conflict on ORDER BY / PROJECT clauses Key: SPARK-45055 URL: https://issues.apache.org/jira/browse/SPARK-45055

[jira] [Created] (SPARK-45054) HiveExternalCatalog.listPartitions should restore Spark SQL stats

Chao Sun created SPARK-45054: Summary: HiveExternalCatalog.listPartitions should restore Spark SQL stats Key: SPARK-45054 URL: https://issues.apache.org/jira/browse/SPARK-45054 Project: Spark

[jira] [Resolved] (SPARK-44952) Add named argument support for aggregate Pandas UDFs

[ https://issues.apache.org/jira/browse/SPARK-44952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-44952. --- Fix Version/s: 4.0.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Created] (SPARK-45053) Improve python version mismatch logging

Wei Liu created SPARK-45053: --- Summary: Improve python version mismatch logging Key: SPARK-45053 URL: https://issues.apache.org/jira/browse/SPARK-45053 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-45037) Upload unit tests log files for timeouted cancel

[ https://issues.apache.org/jira/browse/SPARK-45037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45037: - Assignee: Kent Yao > Upload unit tests log files for timeouted cancel >

[jira] [Resolved] (SPARK-45037) Upload unit tests log files for timeouted cancel

[ https://issues.apache.org/jira/browse/SPARK-45037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45037. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42756

[jira] [Assigned] (SPARK-44942) Use Jira notification options to sync with Github

[ https://issues.apache.org/jira/browse/SPARK-44942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44942: - Assignee: Kent Yao > Use Jira notification options to sync with Github >

[jira] [Resolved] (SPARK-44942) Use Jira notification options to sync with Github

[ https://issues.apache.org/jira/browse/SPARK-44942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44942. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42750

[jira] [Created] (SPARK-45052) Make functions default output column name consistent with SQL

Ruifeng Zheng created SPARK-45052: - Summary: Make functions default output column name consistent with SQL Key: SPARK-45052 URL: https://issues.apache.org/jira/browse/SPARK-45052 Project: Spark

[jira] [Resolved] (SPARK-45048) Add additional tests for Python client

[ https://issues.apache.org/jira/browse/SPARK-45048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45048. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42769

[jira] [Assigned] (SPARK-45048) Add additional tests for Python client

[ https://issues.apache.org/jira/browse/SPARK-45048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-45048: - Assignee: Martin Grund > Add additional tests for Python client >

[jira] [Updated] (SPARK-45051) Connect: Use UUIDv7 for operation IDs to make operations chronologically sortable

[ https://issues.apache.org/jira/browse/SPARK-45051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Dillitz updated SPARK-45051: --- Summary: Connect: Use UUIDv7 for operation IDs to make operations chronologically sortable

[jira] [Updated] (SPARK-44577) INSERT BY NAME returns non-sensical error message

[ https://issues.apache.org/jira/browse/SPARK-44577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-44577: Fix Version/s: (was: 4.0.0) > INSERT BY NAME returns non-sensical error message >

[jira] [Resolved] (SPARK-44577) INSERT BY NAME returns non-sensical error message

[ https://issues.apache.org/jira/browse/SPARK-44577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44577. - Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44577) INSERT BY NAME returns non-sensical error message

[ https://issues.apache.org/jira/browse/SPARK-44577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44577: --- Assignee: Jia Fan > INSERT BY NAME returns non-sensical error message >

[jira] [Created] (SPARK-45051) Connect: Use UUIDv7 for operation IDs to make operations chronological sortable

Robert Dillitz created SPARK-45051: -- Summary: Connect: Use UUIDv7 for operation IDs to make operations chronological sortable Key: SPARK-45051 URL: https://issues.apache.org/jira/browse/SPARK-45051

[jira] [Assigned] (SPARK-44743) Reflect function behavior different from Hive

[ https://issues.apache.org/jira/browse/SPARK-44743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44743: --- Assignee: Jia Fan > Reflect function behavior different from Hive >

[jira] [Assigned] (SPARK-44743) Reflect function behavior different from Hive

[ https://issues.apache.org/jira/browse/SPARK-44743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44743: --- Assignee: (was: Fan Jiang) > Reflect function behavior different from Hive >

[jira] [Assigned] (SPARK-44743) Reflect function behavior different from Hive

[ https://issues.apache.org/jira/browse/SPARK-44743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44743: --- Assignee: Fan Jiang > Reflect function behavior different from Hive >

[jira] [Resolved] (SPARK-44743) Reflect function behavior different from Hive

[ https://issues.apache.org/jira/browse/SPARK-44743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44743. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42661

[jira] [Created] (SPARK-45050) Improve error message for UNKNOWN io.grpc.StatusRuntimeException

Yihong He created SPARK-45050: - Summary: Improve error message for UNKNOWN io.grpc.StatusRuntimeException Key: SPARK-45050 URL: https://issues.apache.org/jira/browse/SPARK-45050 Project: Spark

[jira] [Commented] (SPARK-45039) Include full identifier in Storage tab

[ https://issues.apache.org/jira/browse/SPARK-45039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761295#comment-17761295 ] Pablo Langa Blanco commented on SPARK-45039: https://github.com/apache/spark/pull/42759 >

[jira] [Created] (SPARK-45049) Enable doctests for `coalesce/repartition/repartitionByRange`

Ruifeng Zheng created SPARK-45049: - Summary: Enable doctests for `coalesce/repartition/repartitionByRange` Key: SPARK-45049 URL: https://issues.apache.org/jira/browse/SPARK-45049 Project: Spark

[jira] [Created] (SPARK-45048) Add additional tests for Python client

Martin Grund created SPARK-45048: Summary: Add additional tests for Python client Key: SPARK-45048 URL: https://issues.apache.org/jira/browse/SPARK-45048 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45032) Fix compilation warnings related to `Top-level wildcard is not allowed and will error under -Xsource:3`

[ https://issues.apache.org/jira/browse/SPARK-45032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45032: - Affects Version/s: (was: 3.5.0) > Fix compilation warnings related to `Top-level wildcard is not

[jira] [Created] (SPARK-45047) DataFrame.groupBy support Ordinal input

Ruifeng Zheng created SPARK-45047: - Summary: DataFrame.groupBy support Ordinal input Key: SPARK-45047 URL: https://issues.apache.org/jira/browse/SPARK-45047 Project: Spark Issue Type: New

[jira] [Created] (SPARK-45045) SPARK-43183 broke various tests in 3rd party streaming data sources

Jungtaek Lim created SPARK-45045: Summary: SPARK-43183 broke various tests in 3rd party streaming data sources Key: SPARK-45045 URL: https://issues.apache.org/jira/browse/SPARK-45045 Project: Spark

[jira] [Created] (SPARK-45046) Set shadeTestJar of core module to false

Yang Jie created SPARK-45046: Summary: Set shadeTestJar of core module to false Key: SPARK-45046 URL: https://issues.apache.org/jira/browse/SPARK-45046 Project: Spark Issue Type: Improvement