[jira] [Updated] (SPARK-50672) Make openpyxl optional in PySpark Tests

2024-12-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50672: --- Labels: pull-request-available (was: ) > Make openpyxl optional in PySpark Tests >

[jira] [Updated] (SPARK-50671) Error compiling spark-protobuf module using user-defined protoc

2024-12-25 Thread morvenhuang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] morvenhuang updated SPARK-50671: Description: I'm using user-defined protoc since default version of protoc and protoc-gen-grpc-ja

[jira] [Updated] (SPARK-50671) Error compiling spark-protobuf module using user-defined protoc

2024-12-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50671: --- Labels: pull-request-available (was: ) > Error compiling spark-protobuf module using user-d

[jira] [Updated] (SPARK-48458) Dynamic partition override mode might be ignored in certain scenarios causing data loss

2024-12-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48458: --- Labels: corruption pull-request-available (was: corruption) > Dynamic partition override mo

[jira] [Created] (SPARK-50672) Make openpyxl optional in PySpark Tests

2024-12-25 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-50672: - Summary: Make openpyxl optional in PySpark Tests Key: SPARK-50672 URL: https://issues.apache.org/jira/browse/SPARK-50672 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-50608) Malformed configuration page caused by unclosed tags

2024-12-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-50608: Assignee: Kent Yao > Malformed configuration page caused by unclosed tags > -

[jira] [Updated] (SPARK-50666) DataFrameReader supporting SQL hint

2024-12-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-50666: Affects Version/s: 4.0.0 (was: 3.5.4) > DataFrameReader supporting SQL

[jira] [Commented] (SPARK-50671) Error compiling spark-protobuf module using user-defined protoc

2024-12-25 Thread morvenhuang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17908250#comment-17908250 ] morvenhuang commented on SPARK-50671: - Similar with https://issues.apache.org/jira/b

[jira] [Created] (SPARK-50671) Error compiling spark-protobuf module using user-defined protoc

2024-12-25 Thread morvenhuang (Jira)
morvenhuang created SPARK-50671: --- Summary: Error compiling spark-protobuf module using user-defined protoc Key: SPARK-50671 URL: https://issues.apache.org/jira/browse/SPARK-50671 Project: Spark

[jira] [Resolved] (SPARK-50608) Malformed configuration page caused by unclosed tags

2024-12-25 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-50608. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49223 [https://github.com

[jira] [Comment Edited] (SPARK-50666) DataFrameReader supporting SQL hint

2024-12-25 Thread Shaojie Wu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17908245#comment-17908245 ] Shaojie Wu edited comment on SPARK-50666 at 12/26/24 5:50 AM:

[jira] [Resolved] (SPARK-50529) Change char/varchar behavior under the `spark.sql.preserveCharVarcharTypeInfo` config

2024-12-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-50529. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49128 [https://gith

[jira] [Assigned] (SPARK-50529) Change char/varchar behavior under the `spark.sql.preserveCharVarcharTypeInfo` config

2024-12-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-50529: --- Assignee: Jovan Markovic > Change char/varchar behavior under the > `spark.sql.preserveCha

[jira] [Assigned] (SPARK-50667) Make jinja2 optional in PySpark Tests

2024-12-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-50667: Assignee: Ruifeng Zheng > Make jinja2 optional in PySpark Tests > ---

[jira] [Resolved] (SPARK-50667) Make jinja2 optional in PySpark Tests

2024-12-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50667. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49288 [https://gi

[jira] [Assigned] (SPARK-50650) Improve PlanLogger in the single-pass Analyzer

2024-12-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-50650: --- Assignee: Vladimir Golubev > Improve PlanLogger in the single-pass Analyzer > -

[jira] [Resolved] (SPARK-50665) Substitute LocalRelation with ComparableLocalRelation in NormalizePlan

2024-12-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-50665. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49287 [https://gith

[jira] [Assigned] (SPARK-50665) Substitute LocalRelation with ComparableLocalRelation in NormalizePlan

2024-12-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-50665: --- Assignee: Vladimir Golubev > Substitute LocalRelation with ComparableLocalRelation in Norma

[jira] [Resolved] (SPARK-50650) Improve PlanLogger in the single-pass Analyzer

2024-12-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-50650. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49271 [https://gith

[jira] [Assigned] (SPARK-50647) Add a daily build for PySpark with old dependencies

2024-12-25 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-50647: - Assignee: Ruifeng Zheng > Add a daily build for PySpark with old dependencies > ---

[jira] [Resolved] (SPARK-50647) Add a daily build for PySpark with old dependencies

2024-12-25 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-50647. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49267 [https://

[jira] [Updated] (SPARK-50522) Support for indeterminate collation

2024-12-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50522: --- Labels: pull-request-available (was: ) > Support for indeterminate collation >

[jira] [Resolved] (SPARK-50670) Use Spark 3.5.4 in E2E test

2024-12-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50670. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Assigned] (SPARK-50670) Use Spark 3.5.4 in E2E test

2024-12-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-50670: - Assignee: Dongjoon Hyun > Use Spark 3.5.4 in E2E test > --- > >

[jira] [Updated] (SPARK-48334) NettyServer doesn't shutdown if SparkContext initialize failed

2024-12-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48334: --- Labels: pull-request-available (was: ) > NettyServer doesn't shutdown if SparkContext initi

[jira] [Updated] (SPARK-50670) Use Spark 3.5.4 in E2E test

2024-12-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50670: --- Labels: pull-request-available (was: ) > Use Spark 3.5.4 in E2E test >

[jira] [Created] (SPARK-50670) Use Spark 3.5.4 in E2E test

2024-12-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-50670: - Summary: Use Spark 3.5.4 in E2E test Key: SPARK-50670 URL: https://issues.apache.org/jira/browse/SPARK-50670 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-50657) Upgrade the minimum version of `pyarrow` to 11.0.0

2024-12-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50657: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Upgrade the minimum ver

[jira] [Resolved] (SPARK-50657) Upgrade the minimum version of `pyarrow` to 11.0.0

2024-12-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50657. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49282 [https://

[jira] [Assigned] (SPARK-50657) Upgrade the minimum version of `pyarrow` to 11.0.0

2024-12-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-50657: - Assignee: Ruifeng Zheng > Upgrade the minimum version of `pyarrow` to 11.0.0 >

[jira] [Commented] (SPARK-48334) NettyServer doesn't shutdown if SparkContext initialize failed

2024-12-25 Thread Soumasish Goswami (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17908210#comment-17908210 ] Soumasish Goswami commented on SPARK-48334: --- I manged to reproduce this locall

[jira] [Updated] (SPARK-50510) [3.5] Sporadic ReattachableExecuteSuite failure

2024-12-25 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-50510: - Fix Version/s: 3.5.4 (was: 3.5.5) > [3.5] Sporadic ReattachableExecuteSuite failu

[jira] [Updated] (SPARK-50510) [3.5] Sporadic ReattachableExecuteSuite failure

2024-12-25 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-50510: - Fix Version/s: 3.5.5 (was: 3.5.4) > [3.5] Sporadic ReattachableExecuteSuite failu

[jira] [Updated] (SPARK-50669) Allow long values for timestamp add expression

2024-12-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50669: --- Labels: pull-request-available (was: ) > Allow long values for timestamp add expression > -

[jira] [Created] (SPARK-50669) Allow long values for timestamp add expression

2024-12-25 Thread Stevo Mitric (Jira)
Stevo Mitric created SPARK-50669: Summary: Allow long values for timestamp add expression Key: SPARK-50669 URL: https://issues.apache.org/jira/browse/SPARK-50669 Project: Spark Issue Type: Ta

[jira] [Updated] (SPARK-50668) UNSUPPORTED_INSERT.RDD_BASED when insert into a jdbc datasource

2024-12-25 Thread kevinshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevinshin updated SPARK-50668: -- Description: I have some SQL scripts for pushing the calculation results of the data warehouse to the

[jira] [Updated] (SPARK-50668) UNSUPPORTED_INSERT.RDD_BASED when insert into a jdbc datasource

2024-12-25 Thread kevinshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevinshin updated SPARK-50668: -- Description: I have some SQL scripts for pushing the calculation results of the data warehouse to the

[jira] [Updated] (SPARK-50668) UNSUPPORTED_INSERT.RDD_BASED when insert into a jdbc datasource

2024-12-25 Thread kevinshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevinshin updated SPARK-50668: -- Description: I have some SQL scripts for pushing the calculation results of the data warehouse to the

[jira] [Updated] (SPARK-50668) UNSUPPORTED_INSERT.RDD_BASED when insert into a jdbc datasource

2024-12-25 Thread kevinshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevinshin updated SPARK-50668: -- Description: I have some SQL scripts for pushing the calculation results of the data warehouse to the

[jira] [Updated] (SPARK-50668) UNSUPPORTED_INSERT.RDD_BASED when insert into a jdbc datasource

2024-12-25 Thread kevinshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevinshin updated SPARK-50668: -- Description: I have some SQL scripts for pushing the calculation results of the data warehouse to the

[jira] [Created] (SPARK-50668) UNSUPPORTED_INSERT.RDD_BASED when insert into a jdbc datasource

2024-12-25 Thread kevinshin (Jira)
kevinshin created SPARK-50668: - Summary: UNSUPPORTED_INSERT.RDD_BASED when insert into a jdbc datasource Key: SPARK-50668 URL: https://issues.apache.org/jira/browse/SPARK-50668 Project: Spark Is

[jira] [Resolved] (SPARK-50220) Support LISTAGG() function in PySpark

2024-12-25 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-50220. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49231 [https://

[jira] [Assigned] (SPARK-50220) Support LISTAGG() function in PySpark

2024-12-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50220: -- Assignee: (was: Apache Spark) > Support LISTAGG() function in PySpark > -

[jira] [Assigned] (SPARK-50220) Support LISTAGG() function in PySpark

2024-12-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50220: -- Assignee: Apache Spark > Support LISTAGG() function in PySpark >

[jira] [Assigned] (SPARK-50647) Add a daily build for PySpark with old dependencies

2024-12-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50647: -- Assignee: (was: Apache Spark) > Add a daily build for PySpark with old dependenci

[jira] [Assigned] (SPARK-50647) Add a daily build for PySpark with old dependencies

2024-12-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50647: -- Assignee: Apache Spark > Add a daily build for PySpark with old dependencies > --

[jira] [Updated] (SPARK-50360) StateStoreChangelogReader should read state store version from the changelog

2024-12-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-50360: - Component/s: Structured Streaming (was: SS) > StateStoreChangelogReader sho

[jira] [Resolved] (SPARK-50360) StateStoreChangelogReader should read state store version from the changelog

2024-12-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-50360. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49272 [https://gi

[jira] [Assigned] (SPARK-50360) StateStoreChangelogReader should read state store version from the changelog

2024-12-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-50360: Assignee: Wei Liu > StateStoreChangelogReader should read state store version from the ch