[jira] [Updated] (SPARK-43117) proto message abbreviation should support repeated fields

2024-02-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43117: --- Labels: pull-request-available (was: ) > proto message abbreviation should support

[jira] [Assigned] (SPARK-46997) Enable `spark.worker.cleanup.enabled` by default

2024-02-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46997: - Assignee: Dongjoon Hyun > Enable `spark.worker.cleanup.enabled` by default >

[jira] [Commented] (SPARK-45414) spark-xml misplaces string tag content

2024-02-06 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815123#comment-17815123 ] Ritika Maheshwari commented on SPARK-45414: --- So basically the fix is to replace  val

[jira] [Commented] (SPARK-45414) spark-xml misplaces string tag content

2024-02-06 Thread Ritika Maheshwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815121#comment-17815121 ] Ritika Maheshwari commented on SPARK-45414: --- @Sean Owen  What is causing this issue is this

[jira] [Updated] (SPARK-46997) Enable `spark.worker.cleanup.enabled` by default

2024-02-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46997: --- Labels: pull-request-available (was: ) > Enable `spark.worker.cleanup.enabled` by default

[jira] [Created] (SPARK-46997) Enable `spark.worker.cleanup.enabled` by default

2024-02-06 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-46997: - Summary: Enable `spark.worker.cleanup.enabled` by default Key: SPARK-46997 URL: https://issues.apache.org/jira/browse/SPARK-46997 Project: Spark Issue

[jira] [Commented] (SPARK-46938) Migrate jetty 10 to jetty 11

2024-02-06 Thread HiuFung Kwok (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815117#comment-17815117 ] HiuFung Kwok commented on SPARK-46938: -- [~LuciferYang]  The Hive community ended up not releasing

[jira] [Updated] (SPARK-46996) Use AnalyzeTableCommand overwrite statistics information incorrectly

2024-02-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46996: --- Labels: pull-request-available (was: ) > Use AnalyzeTableCommand overwrite statistics

[jira] [Updated] (SPARK-46995) Allow AQE coalesce final stage in SQL cached plan

2024-02-06 Thread Ziqi Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ziqi Liu updated SPARK-46995: - Component/s: SQL > Allow AQE coalesce final stage in SQL cached plan >

[jira] [Resolved] (SPARK-46526) Limit over certain correlated subqueries results in Nosuchelement exception

2024-02-06 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46526. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44514

[jira] [Created] (SPARK-46996) Use AnalyzeTableCommand overwrite statistics information incorrectly

2024-02-06 Thread Davy Xu (Jira)
Davy Xu created SPARK-46996: --- Summary: Use AnalyzeTableCommand overwrite statistics information incorrectly Key: SPARK-46996 URL: https://issues.apache.org/jira/browse/SPARK-46996 Project: Spark

[jira] [Assigned] (SPARK-46987) ProtoUtils.abbreviate avoid unnecessary setField

2024-02-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-46987: Assignee: Ruifeng Zheng > ProtoUtils.abbreviate avoid unnecessary setField >

[jira] [Resolved] (SPARK-46987) ProtoUtils.abbreviate avoid unnecessary setField

2024-02-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-46987. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45045

[jira] [Created] (SPARK-46995) Allow AQE coalesce final stage in SQL cached plan

2024-02-06 Thread Ziqi Liu (Jira)
Ziqi Liu created SPARK-46995: Summary: Allow AQE coalesce final stage in SQL cached plan Key: SPARK-46995 URL: https://issues.apache.org/jira/browse/SPARK-46995 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-46963) Verify AQE is not enabled for Structured Streaming

2024-02-06 Thread Bo Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Gao resolved SPARK-46963. Resolution: Won't Do > Verify AQE is not enabled for Structured Streaming >

[jira] [Resolved] (SPARK-46984) Remove pyspark.copy_func

2024-02-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46984. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45042

[jira] [Assigned] (SPARK-46984) Remove pyspark.copy_func

2024-02-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46984: Assignee: Hyukjin Kwon > Remove pyspark.copy_func > > >

[jira] [Updated] (SPARK-46913) Implement timer functionality for transformWithState operator

2024-02-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46913: --- Labels: pull-request-available (was: ) > Implement timer functionality for

[jira] [Created] (SPARK-46994) Refactor PythonWrite to prepare for supporting python data source streaming write

2024-02-06 Thread Chaoqin Li (Jira)
Chaoqin Li created SPARK-46994: -- Summary: Refactor PythonWrite to prepare for supporting python data source streaming write Key: SPARK-46994 URL: https://issues.apache.org/jira/browse/SPARK-46994

[jira] [Updated] (SPARK-46689) Support profiling on FlatMapGroupsInBatchExec

2024-02-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46689: --- Labels: pull-request-available (was: ) > Support profiling on FlatMapGroupsInBatchExec >

[jira] [Created] (SPARK-46993) Allow session variables in more places such as from_json for schema

2024-02-06 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-46993: Summary: Allow session variables in more places such as from_json for schema Key: SPARK-46993 URL: https://issues.apache.org/jira/browse/SPARK-46993 Project: Spark

[jira] [Updated] (SPARK-46688) Support profiling on AggregateInPandasExec

2024-02-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46688: --- Labels: pull-request-available (was: ) > Support profiling on AggregateInPandasExec >

[jira] [Updated] (SPARK-46992) Inconsistent results with 'sort', 'cache', and AQE.

2024-02-06 Thread Denis Tarima (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Tarima updated SPARK-46992: - Description:   With AQE enabled, having {color:#4c9aff}sort{color} in the plan changes

[jira] [Updated] (SPARK-46992) Inconsistent results with 'sort', 'cache', and AQE.

2024-02-06 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-46992: - Labels: correctness (was: ) > Inconsistent results with 'sort', 'cache', and AQE. >

[jira] [Commented] (SPARK-46992) Inconsistent results with 'sort', 'cache', and AQE.

2024-02-06 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814913#comment-17814913 ] Nicholas Chammas commented on SPARK-46992: -- I can confirm the behavior described above is still

[jira] [Updated] (SPARK-46992) Inconsistent results with 'sort', 'cache', and AQE.

2024-02-06 Thread Denis Tarima (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Tarima updated SPARK-46992: - Description:   With AQE enabled, having {color:#4c9aff}sort{color} in the plan changes

[jira] [Created] (SPARK-46992) Inconsistent results with 'sort', 'cache', and AQE.

2024-02-06 Thread Denis Tarima (Jira)
Denis Tarima created SPARK-46992: Summary: Inconsistent results with 'sort', 'cache', and AQE. Key: SPARK-46992 URL: https://issues.apache.org/jira/browse/SPARK-46992 Project: Spark Issue

[jira] [Updated] (SPARK-46991) Replace IllegalArgumentException by SparkIllegalArgumentException in catalyst

2024-02-06 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-46991: - Description: Replace all IllegalArgumentException by SparkIllegalArgumentException in Catalyst code

[jira] [Created] (SPARK-46991) Replace IllegalArgumentException by SparkIllegalArgumentException in catalyst

2024-02-06 Thread Max Gekk (Jira)
Max Gekk created SPARK-46991: Summary: Replace IllegalArgumentException by SparkIllegalArgumentException in catalyst Key: SPARK-46991 URL: https://issues.apache.org/jira/browse/SPARK-46991 Project: Spark

[jira] [Created] (SPARK-46990) Regression: Unable to load empty avro files emitted by event-hubs

2024-02-06 Thread Kamil Kandzia (Jira)
Kamil Kandzia created SPARK-46990: - Summary: Regression: Unable to load empty avro files emitted by event-hubs Key: SPARK-46990 URL: https://issues.apache.org/jira/browse/SPARK-46990 Project: Spark

[jira] [Comment Edited] (SPARK-46032) connect: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.rdd.MapPartitionsRDD.f

2024-02-06 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814432#comment-17814432 ] Gaétan CACACE edited comment on SPARK-46032 at 2/6/24 9:59 AM: --- Hello

[jira] [Created] (SPARK-46988) proto message abbreviation should support map fields

2024-02-06 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-46988: - Summary: proto message abbreviation should support map fields Key: SPARK-46988 URL: https://issues.apache.org/jira/browse/SPARK-46988 Project: Spark Issue

[jira] [Updated] (SPARK-43117) proto message abbreviation should support repeated fields

2024-02-06 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43117: -- Summary: proto message abbreviation should support repeated fields (was: proto message

[jira] [Resolved] (SPARK-46895) Replace Timer with single thread scheduled executor

2024-02-06 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-46895. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44718

[jira] [Updated] (SPARK-46987) ProtoUtils.abbreviate avoid unnecessary setField

2024-02-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46987: --- Labels: pull-request-available (was: ) > ProtoUtils.abbreviate avoid unnecessary setField

[jira] [Created] (SPARK-46987) ProtoUtils.abbreviate avoid unnecessary setField

2024-02-06 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-46987: - Summary: ProtoUtils.abbreviate avoid unnecessary setField Key: SPARK-46987 URL: https://issues.apache.org/jira/browse/SPARK-46987 Project: Spark Issue

[jira] [Assigned] (SPARK-46976) Implement `DataFrameGroupBy.corr`

2024-02-06 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-46976: - Assignee: Ruifeng Zheng > Implement `DataFrameGroupBy.corr` >

[jira] [Resolved] (SPARK-46976) Implement `DataFrameGroupBy.corr`

2024-02-06 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46976. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45028

[jira] [Deleted] (SPARK-46986) Move pyspark.loose_version to pyspark.sql.loose_version

2024-02-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon deleted SPARK-46986: - > Move pyspark.loose_version to pyspark.sql.loose_version >

[jira] [Deleted] (SPARK-46985) Move _NoValue from pyspark.* to pyspark.sql.*

2024-02-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon deleted SPARK-46985: - > Move _NoValue from pyspark.* to pyspark.sql.* > - >

[jira] [Updated] (SPARK-46986) Move pyspark.loose_version to pyspark.sql.loose_version

2024-02-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46986: --- Labels: pull-request-available (was: ) > Move pyspark.loose_version to

[jira] [Created] (SPARK-46986) Move pyspark.loose_version to pyspark.sql.loose_version

2024-02-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46986: Summary: Move pyspark.loose_version to pyspark.sql.loose_version Key: SPARK-46986 URL: https://issues.apache.org/jira/browse/SPARK-46986 Project: Spark