[jira] [Updated] (SPARK-52962) BroadcastExchangeExec should not reset metrics

[ https://issues.apache.org/jira/browse/SPARK-52962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-52962: Issue Type: Improvement (was: Bug) > BroadcastExchangeExec should not reset metrics > ---

[jira] [Updated] (SPARK-52962) BroadcastExchangeExec should not reset metrics

[ https://issues.apache.org/jira/browse/SPARK-52962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-52962: Affects Version/s: 4.1.0 (was: 4.0.0) (was: 3

[jira] [Updated] (SPARK-52960) Show subtree string in LogicalQueryStage toString

[ https://issues.apache.org/jira/browse/SPARK-52960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52960: --- Labels: pull-request-available (was: ) > Show subtree string in LogicalQueryStage toString

[jira] [Commented] (SPARK-52957) Add support for Strings with Collation in Expression.Literal

[ https://issues.apache.org/jira/browse/SPARK-52957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18010079#comment-18010079 ] Deng Ziming commented on SPARK-52957: - SPARK-52444 has already included this, we can

[jira] [Created] (SPARK-52962) BroadcastExchangeExec should not reset metrics

L. C. Hsieh created SPARK-52962: --- Summary: BroadcastExchangeExec should not reset metrics Key: SPARK-52962 URL: https://issues.apache.org/jira/browse/SPARK-52962 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-52961) Fix Python UDTF with 0-arg eval on lateral join

[ https://issues.apache.org/jira/browse/SPARK-52961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52961: --- Labels: pull-request-available (was: ) > Fix Python UDTF with 0-arg eval on lateral join >

[jira] [Updated] (SPARK-52961) Fix Python UDTF with 0-arg eval on lateral join

[ https://issues.apache.org/jira/browse/SPARK-52961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-52961: -- Summary: Fix Python UDTF with 0-arg eval on lateral join (was: Fix Python UDTF on lateral joi

[jira] [Created] (SPARK-52961) Fix Python UDTF on lateral join with 0-arg eval

Takuya Ueshin created SPARK-52961: - Summary: Fix Python UDTF on lateral join with 0-arg eval Key: SPARK-52961 URL: https://issues.apache.org/jira/browse/SPARK-52961 Project: Spark Issue Type:

[jira] [Created] (SPARK-52960) Show subtree string in LogicalQueryStage toString

Ziqi Liu created SPARK-52960: Summary: Show subtree string in LogicalQueryStage toString Key: SPARK-52960 URL: https://issues.apache.org/jira/browse/SPARK-52960 Project: Spark Issue Type: Improve

[jira] [Created] (SPARK-52959) Support UDT in Python UDTF

Takuya Ueshin created SPARK-52959: - Summary: Support UDT in Python UDTF Key: SPARK-52959 URL: https://issues.apache.org/jira/browse/SPARK-52959 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-52949) Avoid roundtrip between RecordBatch and Table in Arrow-optimized UDTF

[ https://issues.apache.org/jira/browse/SPARK-52949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52949: -- Parent: SPARK-51166 Issue Type: Sub-task (was: Improvement) > Avoid roundtrip between

[jira] [Updated] (SPARK-50748) Fix a flaky test: `SparkSessionE2ESuite.interrupt all - background queries, foreground interrupt`

[ https://issues.apache.org/jira/browse/SPARK-50748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50748: --- Labels: pull-request-available (was: ) > Fix a flaky test: `SparkSessionE2ESuite.interrupt

[jira] [Resolved] (SPARK-52949) Avoid roundtrip between RecordBatch and Table in Arrow-optimized UDTF

[ https://issues.apache.org/jira/browse/SPARK-52949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-52949. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51659 [https://

[jira] [Assigned] (SPARK-52949) Avoid roundtrip between RecordBatch and Table in Arrow-optimized UDTF

[ https://issues.apache.org/jira/browse/SPARK-52949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-52949: - Assignee: Takuya Ueshin > Avoid roundtrip between RecordBatch and Table in Arrow-optimi

[jira] [Updated] (SPARK-52772) Inconsistent table attribute handling during updates

[ https://issues.apache.org/jira/browse/SPARK-52772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Beyer updated SPARK-52772: Description: TableOutputResolver is inconsistent in copying the table column metadata to the quer

[jira] [Updated] (SPARK-50748) Fix a flaky test: `SparkSessionE2ESuite.interrupt all - background queries, foreground interrupt`

[ https://issues.apache.org/jira/browse/SPARK-50748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50748: -- Fix Version/s: 4.1.0 > Fix a flaky test: `SparkSessionE2ESuite.interrupt all - background quer

[jira] [Resolved] (SPARK-50748) Fix a flaky test: `SparkSessionE2ESuite.interrupt all - background queries, foreground interrupt`

[ https://issues.apache.org/jira/browse/SPARK-50748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50748. --- Resolution: Fixed This is resolved via https://github.com/apache/spark/pull/51638 > Fix a f

[jira] [Assigned] (SPARK-50748) Fix a flaky test: `SparkSessionE2ESuite.interrupt all - background queries, foreground interrupt`

[ https://issues.apache.org/jira/browse/SPARK-50748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-50748: - Assignee: Kousuke Saruta > Fix a flaky test: `SparkSessionE2ESuite.interrupt all - back

[jira] [Commented] (SPARK-50889) Fix Flaky Test: `SparkSessionE2ESuite.interrupt operation` (Hang)

[ https://issues.apache.org/jira/browse/SPARK-50889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18010022#comment-18010022 ] Dongjoon Hyun commented on SPARK-50889: --- I reassigned this issue to [~sarutak] fro

[jira] [Assigned] (SPARK-50889) Fix Flaky Test: `SparkSessionE2ESuite.interrupt operation` (Hang)

[ https://issues.apache.org/jira/browse/SPARK-50889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-50889: - Assignee: Kousuke Saruta (was: Dongjoon Hyun) > Fix Flaky Test: `SparkSessionE2ESuite.

[jira] [Assigned] (SPARK-52141) Display constraints in DESC TABLE and DESC TABLE AS JSON commands

[ https://issues.apache.org/jira/browse/SPARK-52141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-52141: -- Assignee: Deng Ziming > Display constraints in DESC TABLE and DESC TABLE AS JSON comm

[jira] [Resolved] (SPARK-52141) Display constraints in DESC TABLE and DESC TABLE AS JSON commands

[ https://issues.apache.org/jira/browse/SPARK-52141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-52141. Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51577 [https:

[jira] [Updated] (SPARK-51162) SPIP: Add the TIME data type

[ https://issues.apache.org/jira/browse/SPARK-51162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51162: -- Labels: SPIP releasenotes (was: SPIP pull-request-available releasenotes) > SPIP: Add the TIM

[jira] [Comment Edited] (SPARK-51162) SPIP: Add the TIME data type

[ https://issues.apache.org/jira/browse/SPARK-51162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009995#comment-18009995 ] Dongjoon Hyun edited comment on SPARK-51162 at 7/25/25 4:33 PM: --

[jira] [Commented] (SPARK-51162) SPIP: Add the TIME data type

[ https://issues.apache.org/jira/browse/SPARK-51162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009995#comment-18009995 ] Dongjoon Hyun commented on SPARK-51162: --- I added the label, `releasenotes`, not to

[jira] [Updated] (SPARK-51162) SPIP: Add the TIME data type

[ https://issues.apache.org/jira/browse/SPARK-51162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51162: -- Labels: SPIP pull-request-available releasenotes (was: SPIP pull-request-available) > SPIP:

[jira] [Commented] (SPARK-52889) Implement the current_time function in Python

[ https://issues.apache.org/jira/browse/SPARK-52889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009994#comment-18009994 ] Dongjoon Hyun commented on SPARK-52889: --- Thank you for working on the community, [

[jira] [Updated] (SPARK-52945) Test failure related to CastSuiteBase#checkInvalidCastFromNumericType

[ https://issues.apache.org/jira/browse/SPARK-52945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52945: --- Labels: pull-request-available (was: ) > Test failure related to CastSuiteBase#checkInvalid

[jira] [Commented] (SPARK-52889) Implement the current_time function in Python

[ https://issues.apache.org/jira/browse/SPARK-52889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009993#comment-18009993 ] Dongjoon Hyun commented on SPARK-52889: --- Apparently, I don't see any PR so far, [~

[jira] [Commented] (SPARK-52891) Implement the try_to_time function in Python

[ https://issues.apache.org/jira/browse/SPARK-52891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009992#comment-18009992 ] Dongjoon Hyun commented on SPARK-52891: --- Apparently, I don't see any PR until now,

[jira] [Commented] (SPARK-52890) Implement the to_time function in Python

[ https://issues.apache.org/jira/browse/SPARK-52890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009991#comment-18009991 ] Dongjoon Hyun commented on SPARK-52890: --- Apparently, I don't see any PR until now,

[jira] [Commented] (SPARK-52958) Fix PEP 8 naming convention issues in PySpark/SQL session.py

[ https://issues.apache.org/jira/browse/SPARK-52958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009989#comment-18009989 ] Robert Fraker commented on SPARK-52958: --- I would like to work on correcting these,

[jira] [Created] (SPARK-52958) Fix PEP 8 naming convention issues in PySpark/SQL session.py

Robert Fraker created SPARK-52958: - Summary: Fix PEP 8 naming convention issues in PySpark/SQL session.py Key: SPARK-52958 URL: https://issues.apache.org/jira/browse/SPARK-52958 Project: Spark

[jira] [Updated] (SPARK-41166) Check errorSubClass of DataTypeMismatch in *ExpressionSuites

[ https://issues.apache.org/jira/browse/SPARK-41166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-41166: --- Labels: pull-request-available (was: ) > Check errorSubClass of DataTypeMismatch in *Expres

[jira] [Created] (SPARK-52957) Add support for Strings with Collation in Expression.Literal

Yihong He created SPARK-52957: - Summary: Add support for Strings with Collation in Expression.Literal Key: SPARK-52957 URL: https://issues.apache.org/jira/browse/SPARK-52957 Project: Spark Issue

[jira] [Assigned] (SPARK-52944) Fix invalid assertions in tests

[ https://issues.apache.org/jira/browse/SPARK-52944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-52944: Assignee: Yang Jie > Fix invalid assertions in tests > --- > >

[jira] [Resolved] (SPARK-52944) Fix invalid assertions in tests

[ https://issues.apache.org/jira/browse/SPARK-52944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-52944. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51654 [https://github.com

[jira] [Resolved] (SPARK-52955) Change return types of WindowResolution.resolveOrder and WindowResolution.resolveFrame to WindowExpression

[ https://issues.apache.org/jira/browse/SPARK-52955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-52955. - Resolution: Fixed Issue resolved by pull request 51666 [https://github.com/apache/spark/pull/516

[jira] [Assigned] (SPARK-52955) Change return types of WindowResolution.resolveOrder and WindowResolution.resolveFrame to WindowExpression

[ https://issues.apache.org/jira/browse/SPARK-52955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-52955: --- Assignee: Nikola Jovićević > Change return types of WindowResolution.resolveOrder and > Wi

[jira] [Updated] (SPARK-52955) Change return types of WindowResolution.resolveOrder and WindowResolution.resolveFrame to WindowExpression

[ https://issues.apache.org/jira/browse/SPARK-52955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52955: --- Labels: pull-request-available (was: ) > Change return types of WindowResolution.resolveOrd

[jira] [Updated] (SPARK-52944) Fix invalid assertions in tests

[ https://issues.apache.org/jira/browse/SPARK-52944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52944: --- Labels: pull-request-available (was: ) > Fix invalid assertions in tests >

[jira] [Comment Edited] (SPARK-52443) Enable MiMa for sql-api

[ https://issues.apache.org/jira/browse/SPARK-52443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009945#comment-18009945 ] Yihong He edited comment on SPARK-52443 at 7/25/25 1:23 PM:

[jira] [Comment Edited] (SPARK-52443) Enable MiMa for sql-api

[ https://issues.apache.org/jira/browse/SPARK-52443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009945#comment-18009945 ] Yihong He edited comment on SPARK-52443 at 7/25/25 1:23 PM:

[jira] [Updated] (SPARK-52943) Enable arrow_cast for all pandas UDF eval types

[ https://issues.apache.org/jira/browse/SPARK-52943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52943: --- Labels: pull-request-available (was: ) > Enable arrow_cast for all pandas UDF eval types >

[jira] [Comment Edited] (SPARK-52443) Enable MiMa for sql-api

[ https://issues.apache.org/jira/browse/SPARK-52443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009945#comment-18009945 ] Yihong He edited comment on SPARK-52443 at 7/25/25 1:08 PM:

[jira] [Comment Edited] (SPARK-52443) Enable MiMa for sql-api

[ https://issues.apache.org/jira/browse/SPARK-52443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009945#comment-18009945 ] Yihong He edited comment on SPARK-52443 at 7/25/25 1:08 PM:

[jira] [Commented] (SPARK-52443) Enable MiMa for sql-api

[ https://issues.apache.org/jira/browse/SPARK-52443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009945#comment-18009945 ] Yihong He commented on SPARK-52443: --- It seems that MiMa is already enabled for sql-api

[jira] [Comment Edited] (SPARK-52443) Enable MiMa for sql-api

[ https://issues.apache.org/jira/browse/SPARK-52443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009945#comment-18009945 ] Yihong He edited comment on SPARK-52443 at 7/25/25 1:07 PM:

[jira] [Commented] (SPARK-52585) Failed to load class "org.slf4j.impl.StaticLoggerBinder" when using hadoop-aws

[ https://issues.apache.org/jira/browse/SPARK-52585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009934#comment-18009934 ] rami commented on SPARK-52585: -- I observe the same behavior when running  batch jobs on Kub

[jira] [Updated] (SPARK-52544) Allow configuring Json datasource string length limit through SQLConf

[ https://issues.apache.org/jira/browse/SPARK-52544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52544: --- Labels: pull-request-available (was: ) > Allow configuring Json datasource string length li

[jira] [Updated] (SPARK-52956) Preserve alias metadata when collapsing projects

[ https://issues.apache.org/jira/browse/SPARK-52956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52956: --- Labels: pull-request-available (was: ) > Preserve alias metadata when collapsing projects >

[jira] [Created] (SPARK-52956) Preserve alias metadata when collapsing projects

Mihailo Timotic created SPARK-52956: --- Summary: Preserve alias metadata when collapsing projects Key: SPARK-52956 URL: https://issues.apache.org/jira/browse/SPARK-52956 Project: Spark Issue

[jira] [Updated] (SPARK-52955) Change return types of WindowResolution.resolveOrder and WindowResolution.resolveFrame to WindowExpression

[ https://issues.apache.org/jira/browse/SPARK-52955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikola Jovićević updated SPARK-52955: - Summary: Change return types of WindowResolution.resolveOrder and WindowResolution.resol

[jira] [Updated] (SPARK-52955) Change return type of WindowResolution.resolveOrder and WindowResolution.resolveFrame to WindowExpression

[ https://issues.apache.org/jira/browse/SPARK-52955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikola Jovićević updated SPARK-52955: - Summary: Change return type of WindowResolution.resolveOrder and WindowResolution.resolv

[jira] [Created] (SPARK-52955) Change return type of WindowResolution.resolveOrder to WindowExpression

Nikola Jovićević created SPARK-52955: Summary: Change return type of WindowResolution.resolveOrder to WindowExpression Key: SPARK-52955 URL: https://issues.apache.org/jira/browse/SPARK-52955 Proje

[jira] [Resolved] (SPARK-49968) The split function produces incorrect results with an empty regex and a limit

[ https://issues.apache.org/jira/browse/SPARK-49968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-49968. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 48470 [https://gith

[jira] [Assigned] (SPARK-49968) The split function produces incorrect results with an empty regex and a limit

[ https://issues.apache.org/jira/browse/SPARK-49968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-49968: --- Assignee: Dejiu Lu > The split function produces incorrect results with an empty regex and

[jira] [Commented] (SPARK-52411) Spark event log not able to write to s3

[ https://issues.apache.org/jira/browse/SPARK-52411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009888#comment-18009888 ] Jungtaek Lim commented on SPARK-52411: -- Also, Apache Spark assumes the file system

[jira] [Commented] (SPARK-52411) Spark event log not able to write to s3

[ https://issues.apache.org/jira/browse/SPARK-52411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009886#comment-18009886 ] Jungtaek Lim commented on SPARK-52411: -- Fixed versions is to present which version

[jira] [Updated] (SPARK-52411) Spark event log not able to write to s3

[ https://issues.apache.org/jira/browse/SPARK-52411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-52411: - Fix Version/s: (was: 4.0.1) (was: 3.5.7) > Spark event log not able t

[jira] [Updated] (SPARK-52954) Arrow UDF support return type coercion

[ https://issues.apache.org/jira/browse/SPARK-52954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52954: --- Labels: pull-request-available (was: ) > Arrow UDF support return type coercion > -

[jira] [Updated] (SPARK-52953) Incorrect parameter order in some ExpressionEvalHelper.checkResult() method invocations

[ https://issues.apache.org/jira/browse/SPARK-52953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52953: --- Labels: pull-request-available (was: ) > Incorrect parameter order in some ExpressionEvalHe

[jira] [Created] (SPARK-52954) Arrow UDF support return type coercion

Ruifeng Zheng created SPARK-52954: - Summary: Arrow UDF support return type coercion Key: SPARK-52954 URL: https://issues.apache.org/jira/browse/SPARK-52954 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-52953) Incorrect parameter order in some ExpressionEvalHelper.checkResult() method invocations

morvenhuang created SPARK-52953: --- Summary: Incorrect parameter order in some ExpressionEvalHelper.checkResult() method invocations Key: SPARK-52953 URL: https://issues.apache.org/jira/browse/SPARK-52953