[jira] [Resolved] (SPARK-52458) Support `spark.eventLog.excludedPatterns`

2025-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52458. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51163 [https://

[jira] [Assigned] (SPARK-52456) Lower the minimum limit of `spark.eventLog.rolling.maxFileSize`

2025-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52456: - Assignee: Dongjoon Hyun > Lower the minimum limit of `spark.eventLog.rolling.maxFileSiz

[jira] [Updated] (SPARK-52431) Use spark-submit underneath the spark-pipelines CLI

2025-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52431: --- Labels: pull-request-available (was: ) > Use spark-submit underneath the spark-pipelines CL

[jira] [Updated] (SPARK-52459) SPDI-168275.Fix a subtle thread-safety issue with SQLAppStatusListener

2025-06-11 Thread xiaoping.huang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaoping.huang updated SPARK-52459: --- Description: In our scenario, a deadlock occurred in the HashMap in SQLAppStatusListener. T

[jira] [Updated] (SPARK-52459) SPDI-168275.Fix a subtle thread-safety issue with SQLAppStatusListener

2025-06-11 Thread xiaoping.huang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaoping.huang updated SPARK-52459: --- Attachment: image-2025-06-12-12-06-59-179.png > SPDI-168275.Fix a subtle thread-safety issue

[jira] [Updated] (SPARK-52459) SPDI-168275.Fix a subtle thread-safety issue with SQLAppStatusListener

2025-06-11 Thread xiaoping.huang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaoping.huang updated SPARK-52459: --- Description: In our scenario, a deadlock occurred in the HashMap in SQLAppStatusListener. T

[jira] [Updated] (SPARK-52458) Support `spark.eventLog.excludedPatterns`

2025-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52458: --- Labels: pull-request-available (was: ) > Support `spark.eventLog.excludedPatterns` > --

[jira] [Resolved] (SPARK-52456) Lower the minimum limit of `spark.eventLog.rolling.maxFileSize`

2025-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52456. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51162 [https://

[jira] [Created] (SPARK-52458) Support `spark.eventLog.excludedPatterns`

2025-06-11 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-52458: - Summary: Support `spark.eventLog.excludedPatterns` Key: SPARK-52458 URL: https://issues.apache.org/jira/browse/SPARK-52458 Project: Spark Issue Type: Sub-t

[jira] [Resolved] (SPARK-52430) Resolve bug with nullability of Union in rCTEs

2025-06-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-52430. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51134 [https://gith

[jira] [Created] (SPARK-52457) ParseToDate/ParseToTimestamp can return incorrect value for TimestampNTZ

2025-06-11 Thread Kelvin Jiang (Jira)
Kelvin Jiang created SPARK-52457: Summary: ParseToDate/ParseToTimestamp can return incorrect value for TimestampNTZ Key: SPARK-52457 URL: https://issues.apache.org/jira/browse/SPARK-52457 Project: Spa

[jira] [Updated] (SPARK-52456) Lower the minimum limit of `spark.eventLog.rolling.maxFileSize`

2025-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52456: -- Summary: Lower the minimum limit of `spark.eventLog.rolling.maxFileSize` (was: Lower the maxi

[jira] [Updated] (SPARK-52456) Lower the maximum size limit of event log files

2025-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52456: --- Labels: pull-request-available (was: ) > Lower the maximum size limit of event log files >

[jira] [Created] (SPARK-52456) Lower the maximum size limit of event log files

2025-06-11 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-52456: - Summary: Lower the maximum size limit of event log files Key: SPARK-52456 URL: https://issues.apache.org/jira/browse/SPARK-52456 Project: Spark Issue Type:

[jira] [Updated] (SPARK-52454) Automatically remove old releases from the mirror

2025-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52454: --- Labels: pull-request-available (was: ) > Automatically remove old releases from the mirror

[jira] [Updated] (SPARK-52455) Test DSv2 expression-based defaults in UPDATE and MERGE

2025-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52455: --- Labels: pull-request-available (was: ) > Test DSv2 expression-based defaults in UPDATE and

[jira] [Created] (SPARK-52455) Test DSv2 expression-based defaults in UPDATE and MERGE

2025-06-11 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-52455: Summary: Test DSv2 expression-based defaults in UPDATE and MERGE Key: SPARK-52455 URL: https://issues.apache.org/jira/browse/SPARK-52455 Project: Spark

[jira] [Updated] (SPARK-52453) Automatically release and drop artifacts in Apache Nexus repository

2025-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52453: --- Labels: pull-request-available (was: ) > Automatically release and drop artifacts in Apache

[jira] [Created] (SPARK-52454) Automatically remove old releases from the mirror

2025-06-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-52454: Summary: Automatically remove old releases from the mirror Key: SPARK-52454 URL: https://issues.apache.org/jira/browse/SPARK-52454 Project: Spark Issue Type:

[jira] [Created] (SPARK-52453) Automatically release and drop artifacts in Apache Nexus repository

2025-06-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-52453: Summary: Automatically release and drop artifacts in Apache Nexus repository Key: SPARK-52453 URL: https://issues.apache.org/jira/browse/SPARK-52453 Project: Spark

[jira] [Resolved] (SPARK-52440) Add python install extra for declarative pipeline

2025-06-11 Thread Sandy Ryza (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved SPARK-52440. Resolution: Fixed > Add python install extra for declarative pipeline > --

[jira] [Commented] (SPARK-52442) Spark Connect improvements in Spark 4.1

2025-06-11 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-52442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17959756#comment-17959756 ] Herman van Hövell commented on SPARK-52442: --- I need to populate this with open

[jira] [Resolved] (SPARK-42636) Audit annotation usage

2025-06-11 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42636. --- Resolution: Not A Problem Not a problem anymore. > Audit annotation usage > ---

[jira] [Commented] (SPARK-42562) UnresolvedLambdaVariables in python do not need unique names

2025-06-11 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17959752#comment-17959752 ] Herman van Hövell commented on SPARK-42562: --- I was wrong here :) > Unresolved

[jira] [Resolved] (SPARK-42562) UnresolvedLambdaVariables in python do not need unique names

2025-06-11 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42562. --- Resolution: Not A Problem > UnresolvedLambdaVariables in python do not need unique n

[jira] [Updated] (SPARK-52450) Improve performance of schema deepcopy

2025-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52450: --- Labels: pull-request-available (was: ) > Improve performance of schema deepcopy > -

[jira] [Created] (SPARK-52452) Break-up the SparkConnectPlanner

2025-06-11 Thread Jira
Herman van Hövell created SPARK-52452: - Summary: Break-up the SparkConnectPlanner Key: SPARK-52452 URL: https://issues.apache.org/jira/browse/SPARK-52452 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-52451) The SparkConnectPlanner should side effect free

2025-06-11 Thread Jira
Herman van Hövell created SPARK-52451: - Summary: The SparkConnectPlanner should side effect free Key: SPARK-52451 URL: https://issues.apache.org/jira/browse/SPARK-52451 Project: Spark Iss

[jira] [Created] (SPARK-52450) Improve performance of schema deepcopy

2025-06-11 Thread Xi Lyu (Jira)
Xi Lyu created SPARK-52450: -- Summary: Improve performance of schema deepcopy Key: SPARK-52450 URL: https://issues.apache.org/jira/browse/SPARK-52450 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-52449) Make datatypes for Expression.Literal.Map/Expression.Literal.Array optional

2025-06-11 Thread Jira
Herman van Hövell created SPARK-52449: - Summary: Make datatypes for Expression.Literal.Map/Expression.Literal.Array optional Key: SPARK-52449 URL: https://issues.apache.org/jira/browse/SPARK-52449

[jira] [Created] (SPARK-52443) Enable MiMa for sql-api

2025-06-11 Thread Jira
Herman van Hövell created SPARK-52443: - Summary: Enable MiMa for sql-api Key: SPARK-52443 URL: https://issues.apache.org/jira/browse/SPARK-52443 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-52448) Add simplified Struct Expression.Literal

2025-06-11 Thread Jira
Herman van Hövell created SPARK-52448: - Summary: Add simplified Struct Expression.Literal Key: SPARK-52448 URL: https://issues.apache.org/jira/browse/SPARK-52448 Project: Spark Issue Type

[jira] [Created] (SPARK-52447) Move GetViewColumnByNameAndOrdinal normalization to SessionCatalogSuite

2025-06-11 Thread Vladimir Golubev (Jira)
Vladimir Golubev created SPARK-52447: Summary: Move GetViewColumnByNameAndOrdinal normalization to SessionCatalogSuite Key: SPARK-52447 URL: https://issues.apache.org/jira/browse/SPARK-52447 Proje

[jira] [Created] (SPARK-52446) Ramp-up documentation of the protos

2025-06-11 Thread Jira
Herman van Hövell created SPARK-52446: - Summary: Ramp-up documentation of the protos Key: SPARK-52446 URL: https://issues.apache.org/jira/browse/SPARK-52446 Project: Spark Issue Type: Tas

[jira] [Created] (SPARK-52445) Improve limit handling in Spark Connect

2025-06-11 Thread Jira
Herman van Hövell created SPARK-52445: - Summary: Improve limit handling in Spark Connect Key: SPARK-52445 URL: https://issues.apache.org/jira/browse/SPARK-52445 Project: Spark Issue Type:

[jira] [Created] (SPARK-52444) Add support for all DataTypes in Expression.Literal

2025-06-11 Thread Jira
Herman van Hövell created SPARK-52444: - Summary: Add support for all DataTypes in Expression.Literal Key: SPARK-52444 URL: https://issues.apache.org/jira/browse/SPARK-52444 Project: Spark

[jira] [Updated] (SPARK-52442) Spark Connect improvements in Spark 4.1

2025-06-11 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-52442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-52442: -- Summary: Spark Connect improvements in Spark 4.1 (was: Spark Connect improvement we w

[jira] [Created] (SPARK-52442) Spark Connect improvement we want to ship in Spark 4.1

2025-06-11 Thread Jira
Herman van Hövell created SPARK-52442: - Summary: Spark Connect improvement we want to ship in Spark 4.1 Key: SPARK-52442 URL: https://issues.apache.org/jira/browse/SPARK-52442 Project: Spark

[jira] [Updated] (SPARK-49493) ORDERY BY result is wrong when use sub query

2025-06-11 Thread chaijunjie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated SPARK-49493: --- Description: 1. select * from test.orc_table where load_time<='2024-06-14' limit 10; it work fine w

[jira] [Updated] (SPARK-49493) ORDERY BY result is wrong when use sub query

2025-06-11 Thread chaijunjie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated SPARK-49493: --- Description: 1. select * from test.orc_table where load_time<='2024-06-14' limit 10; it work fine w

[jira] [Resolved] (SPARK-52441) Skip testing `sql/pipelines` module in the Maven daily test when the input branch is branch-4.0.

2025-06-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-52441. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51151 [https://github.com

[jira] [Assigned] (SPARK-52441) Skip testing `sql/pipelines` module in the Maven daily test when the input branch is branch-4.0.

2025-06-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-52441: Assignee: Yang Jie > Skip testing `sql/pipelines` module in the Maven daily test when the input

[jira] [Resolved] (SPARK-52423) pipelines.proto misses golang package

2025-06-11 Thread Martin Grund (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Grund resolved SPARK-52423. -- Resolution: Fixed Merged as https://github.com/apache/spark/commit/46de57b55113ddf3da10d50555

[jira] [Assigned] (SPARK-52423) pipelines.proto misses golang package

2025-06-11 Thread Martin Grund (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Grund reassigned SPARK-52423: Assignee: Martin Grund > pipelines.proto misses golang package >

[jira] [Updated] (SPARK-52441) Skip testing `sql/pipelines` module in the Maven daily test when the input branch is branch-4.0.

2025-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52441: --- Labels: pull-request-available (was: ) > Skip testing `sql/pipelines` module in the Maven d

[jira] [Created] (SPARK-52441) Skip testing `sql/pipelines` module in the Maven daily test when the input branch is branch-4.0.

2025-06-11 Thread Yang Jie (Jira)
Yang Jie created SPARK-52441: Summary: Skip testing `sql/pipelines` module in the Maven daily test when the input branch is branch-4.0. Key: SPARK-52441 URL: https://issues.apache.org/jira/browse/SPARK-52441