[jira] [Commented] (SPARK-41388) getReusablePVCs should ignore recently created PVCs in the previous batch

[ https://issues.apache.org/jira/browse/SPARK-41388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643150#comment-17643150 ] Apache Spark commented on SPARK-41388: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-41388) getReusablePVCs should ignore recently created PVCs in the previous batch

[ https://issues.apache.org/jira/browse/SPARK-41388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41388: Assignee: (was: Apache Spark) > getReusablePVCs should ignore recently created PVCs

[jira] [Commented] (SPARK-41388) getReusablePVCs should ignore recently created PVCs in the previous batch

[ https://issues.apache.org/jira/browse/SPARK-41388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643149#comment-17643149 ] Apache Spark commented on SPARK-41388: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-41388) getReusablePVCs should ignore recently created PVCs in the previous batch

[ https://issues.apache.org/jira/browse/SPARK-41388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41388: Assignee: Apache Spark > getReusablePVCs should ignore recently created PVCs in the

[jira] [Created] (SPARK-41388) getReusablePVCs should ignore recently created PVCs in the previous batch

Dongjoon Hyun created SPARK-41388: - Summary: getReusablePVCs should ignore recently created PVCs in the previous batch Key: SPARK-41388 URL: https://issues.apache.org/jira/browse/SPARK-41388 Project:

[jira] [Assigned] (SPARK-41387) Add assertion on end offset range for Kafka data source with Trigger.AvailableNow

[ https://issues.apache.org/jira/browse/SPARK-41387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41387: Assignee: (was: Apache Spark) > Add assertion on end offset range for Kafka data

[jira] [Commented] (SPARK-41387) Add assertion on end offset range for Kafka data source with Trigger.AvailableNow

[ https://issues.apache.org/jira/browse/SPARK-41387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643145#comment-17643145 ] Apache Spark commented on SPARK-41387: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-41387) Add assertion on end offset range for Kafka data source with Trigger.AvailableNow

[ https://issues.apache.org/jira/browse/SPARK-41387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41387: Assignee: Apache Spark > Add assertion on end offset range for Kafka data source with >

[jira] [Updated] (SPARK-41386) Resolve small files with REBALANCE Hint

[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Dong updated SPARK-41386: - Description: TODO: (was: Sorry, A wrong location of Issue) > Resolve small files with REBALANCE Hint >

[jira] [Commented] (SPARK-41241) Use Hive and Spark SQL to modify table field comment, the modified results of Hive cannot be queried using Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-41241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643141#comment-17643141 ] weiliang hao commented on SPARK-41241: -- [~xkrogen] The problem is that Spark modifies the Hive

[jira] [Updated] (SPARK-41386) Resolve small files with REBALANCE Hint

[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Dong updated SPARK-41386: - Epic Link: SPARK-39375 > Resolve small files with REBALANCE Hint >

[jira] [Updated] (SPARK-41386) Resolve small files with REBALANCE Hint

[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Dong updated SPARK-41386: - Parent: (was: SPARK-41282) Issue Type: Improvement (was: Sub-task) > Resolve small files

[jira] [Updated] (SPARK-41386) Resolve small files with REBALANCE Hint

[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Dong updated SPARK-41386: - Component/s: Spark Core (was: Connect) Issue Type: Technical task (was:

[jira] [Commented] (SPARK-41387) Add assertion on end offset range for Kafka data source with Trigger.AvailableNow

[ https://issues.apache.org/jira/browse/SPARK-41387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643140#comment-17643140 ] Jungtaek Lim commented on SPARK-41387: -- Will submit a PR sooner. > Add assertion on end offset

[jira] [Updated] (SPARK-41386) Resolve small files with REBALANCE Hint

[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Dong updated SPARK-41386: - Issue Type: Sub-task (was: Technical task) > Resolve small files with REBALANCE Hint >

[jira] [Created] (SPARK-41387) Add assertion on end offset range for Kafka data source with Trigger.AvailableNow

Jungtaek Lim created SPARK-41387: Summary: Add assertion on end offset range for Kafka data source with Trigger.AvailableNow Key: SPARK-41387 URL: https://issues.apache.org/jira/browse/SPARK-41387

[jira] [Updated] (SPARK-41386) Resolve small files with REBALANCE Hint

[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Dong updated SPARK-41386: - Description: Sorry, A wrong location of Issue (was: todo: ) > Resolve small files with REBALANCE Hint

[jira] [Updated] (SPARK-41386) Resolve small files with REBALANCE Hint

[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Dong updated SPARK-41386: - Priority: Minor (was: Blocker) > Resolve small files with REBALANCE Hint >

[jira] [Updated] (SPARK-41386) Resolve small files with REBALANCE Hint

[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Dong updated SPARK-41386: - Description: todo:  (was: For compatibility, we need support for expressing window functions. Window

[jira] [Created] (SPARK-41386) Resolve small files with REBALANCE Hint

Zhe Dong created SPARK-41386: Summary: Resolve small files with REBALANCE Hint Key: SPARK-41386 URL: https://issues.apache.org/jira/browse/SPARK-41386 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-41151) Keep built-in file _metadata column nullable value consistent

[ https://issues.apache.org/jira/browse/SPARK-41151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643134#comment-17643134 ] Apache Spark commented on SPARK-41151: -- User 'Yaohua628' has created a pull request for this issue:

[jira] [Commented] (SPARK-41151) Keep built-in file _metadata column nullable value consistent

[ https://issues.apache.org/jira/browse/SPARK-41151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643133#comment-17643133 ] Apache Spark commented on SPARK-41151: -- User 'Yaohua628' has created a pull request for this issue:

[jira] [Commented] (SPARK-41344) Reading V2 datasource masks underlying error

[ https://issues.apache.org/jira/browse/SPARK-41344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643131#comment-17643131 ] Zhen Wang commented on SPARK-41344: --- [~ganaakruti]  As far as I understand, V1 and V2 only have

[jira] [Resolved] (SPARK-41385) Replace deprecated `.newInstance()` in K8s module

[ https://issues.apache.org/jira/browse/SPARK-41385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41385. --- Fix Version/s: 3.3.2 3.4.0 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-41385) Replace deprecated `.newInstance()` in K8s module

[ https://issues.apache.org/jira/browse/SPARK-41385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41385: - Assignee: Dongjoon Hyun > Replace deprecated `.newInstance()` in K8s module >

[jira] [Assigned] (SPARK-41380) Implement aggregation functions

[ https://issues.apache.org/jira/browse/SPARK-41380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41380: - Assignee: Ruifeng Zheng > Implement aggregation functions >

[jira] [Resolved] (SPARK-41379) Inconsistency of spark session in DataFrame in user function for foreachBatch sink in PySpark

[ https://issues.apache.org/jira/browse/SPARK-41379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-41379. -- Fix Version/s: 3.3.2 3.4.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-41380) Implement aggregation functions

[ https://issues.apache.org/jira/browse/SPARK-41380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41380. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38905

[jira] [Assigned] (SPARK-41379) Inconsistency of spark session in DataFrame in user function for foreachBatch sink in PySpark

[ https://issues.apache.org/jira/browse/SPARK-41379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-41379: Assignee: Jungtaek Lim > Inconsistency of spark session in DataFrame in user function

[jira] [Assigned] (SPARK-41385) Replace deprecated `.newInstance()` in K8s module

[ https://issues.apache.org/jira/browse/SPARK-41385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41385: Assignee: Apache Spark > Replace deprecated `.newInstance()` in K8s module >

[jira] [Commented] (SPARK-41385) Replace deprecated `.newInstance()` in K8s module

[ https://issues.apache.org/jira/browse/SPARK-41385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643123#comment-17643123 ] Apache Spark commented on SPARK-41385: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-41385) Replace deprecated `.newInstance()` in K8s module

[ https://issues.apache.org/jira/browse/SPARK-41385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41385: Assignee: (was: Apache Spark) > Replace deprecated `.newInstance()` in K8s module >

[jira] [Created] (SPARK-41385) Replace deprecated `.newInstance()` in K8s module

Dongjoon Hyun created SPARK-41385: - Summary: Replace deprecated `.newInstance()` in K8s module Key: SPARK-41385 URL: https://issues.apache.org/jira/browse/SPARK-41385 Project: Spark Issue

[jira] [Assigned] (SPARK-41384) Should use SQLExpression for str arguments in Projection

[ https://issues.apache.org/jira/browse/SPARK-41384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41384: Assignee: (was: Apache Spark) > Should use SQLExpression for str arguments in

[jira] [Commented] (SPARK-41384) Should use SQLExpression for str arguments in Projection

[ https://issues.apache.org/jira/browse/SPARK-41384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643121#comment-17643121 ] Apache Spark commented on SPARK-41384: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41384) Should use SQLExpression for str arguments in Projection

[ https://issues.apache.org/jira/browse/SPARK-41384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41384: Assignee: Apache Spark > Should use SQLExpression for str arguments in Projection >

[jira] [Commented] (SPARK-41384) Should use SQLExpression for str arguments in Projection

[ https://issues.apache.org/jira/browse/SPARK-41384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643119#comment-17643119 ] Apache Spark commented on SPARK-41384: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Created] (SPARK-41384) Should use SQLExpression for str arguments in Projection

Rui Wang created SPARK-41384: Summary: Should use SQLExpression for str arguments in Projection Key: SPARK-41384 URL: https://issues.apache.org/jira/browse/SPARK-41384 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-41358) Use `PhysicalDataType` instead of DataType in ColumnVectorUtils

[ https://issues.apache.org/jira/browse/SPARK-41358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41358. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38873

[jira] [Assigned] (SPARK-41358) Use `PhysicalDataType` instead of DataType in ColumnVectorUtils

[ https://issues.apache.org/jira/browse/SPARK-41358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41358: --- Assignee: Yang Jie > Use `PhysicalDataType` instead of DataType in ColumnVectorUtils >

[jira] [Commented] (SPARK-40379) Propagate decommission executor loss reason during onDisconnect in K8s

[ https://issues.apache.org/jira/browse/SPARK-40379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643115#comment-17643115 ] Apache Spark commented on SPARK-40379: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-40379) Propagate decommission executor loss reason during onDisconnect in K8s

[ https://issues.apache.org/jira/browse/SPARK-40379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643114#comment-17643114 ] Apache Spark commented on SPARK-40379: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Resolved] (SPARK-40379) Propagate decommission executor loss reason during onDisconnect in K8s

[ https://issues.apache.org/jira/browse/SPARK-40379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40379. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37821

[jira] [Commented] (SPARK-41379) Inconsistency of spark session in DataFrame in user function for foreachBatch sink in PySpark

[ https://issues.apache.org/jira/browse/SPARK-41379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643103#comment-17643103 ] Apache Spark commented on SPARK-41379: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-41379) Inconsistency of spark session in DataFrame in user function for foreachBatch sink in PySpark

[ https://issues.apache.org/jira/browse/SPARK-41379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41379: Assignee: (was: Apache Spark) > Inconsistency of spark session in DataFrame in user

[jira] [Assigned] (SPARK-41379) Inconsistency of spark session in DataFrame in user function for foreachBatch sink in PySpark

[ https://issues.apache.org/jira/browse/SPARK-41379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41379: Assignee: Apache Spark > Inconsistency of spark session in DataFrame in user function

[jira] [Commented] (SPARK-41379) Inconsistency of spark session in DataFrame in user function for foreachBatch sink in PySpark

[ https://issues.apache.org/jira/browse/SPARK-41379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643104#comment-17643104 ] Apache Spark commented on SPARK-41379: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-41380) Implement aggregation functions

[ https://issues.apache.org/jira/browse/SPARK-41380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41380: Assignee: (was: Apache Spark) > Implement aggregation functions >

[jira] [Commented] (SPARK-41380) Implement aggregation functions

[ https://issues.apache.org/jira/browse/SPARK-41380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643101#comment-17643101 ] Apache Spark commented on SPARK-41380: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41380) Implement aggregation functions

[ https://issues.apache.org/jira/browse/SPARK-41380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41380: Assignee: Apache Spark > Implement aggregation functions >

[jira] [Created] (SPARK-41383) Implement `DataFrame.cube`

Ruifeng Zheng created SPARK-41383: - Summary: Implement `DataFrame.cube` Key: SPARK-41383 URL: https://issues.apache.org/jira/browse/SPARK-41383 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41382) implement `product` function

Ruifeng Zheng created SPARK-41382: - Summary: implement `product` function Key: SPARK-41382 URL: https://issues.apache.org/jira/browse/SPARK-41382 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41380) Implement aggregation functions

Ruifeng Zheng created SPARK-41380: - Summary: Implement aggregation functions Key: SPARK-41380 URL: https://issues.apache.org/jira/browse/SPARK-41380 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41381) Implement count_distinct and sum_distinct functions

Ruifeng Zheng created SPARK-41381: - Summary: Implement count_distinct and sum_distinct functions Key: SPARK-41381 URL: https://issues.apache.org/jira/browse/SPARK-41381 Project: Spark Issue

[jira] [Reopened] (SPARK-41305) Connect Proto Completeness

[ https://issues.apache.org/jira/browse/SPARK-41305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reopened SPARK-41305: -- > Connect Proto Completeness > -- > > Key: SPARK-41305 >

[jira] [Commented] (SPARK-41371) Improve Documentation for Command proto

[ https://issues.apache.org/jira/browse/SPARK-41371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643098#comment-17643098 ] Rui Wang commented on SPARK-41371: -- https://github.com/apache/spark/pull/38890 > Improve Documentation

[jira] [Resolved] (SPARK-41371) Improve Documentation for Command proto

[ https://issues.apache.org/jira/browse/SPARK-41371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang resolved SPARK-41371. -- Fix Version/s: 3.4.0 Resolution: Fixed > Improve Documentation for Command proto >

[jira] [Assigned] (SPARK-41350) allow simple name access of using join hidden columns after subquery alias

[ https://issues.apache.org/jira/browse/SPARK-41350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41350: Assignee: Wenchen Fan > allow simple name access of using join hidden columns after

[jira] [Resolved] (SPARK-41350) allow simple name access of using join hidden columns after subquery alias

[ https://issues.apache.org/jira/browse/SPARK-41350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41350. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38862

[jira] [Resolved] (SPARK-41351) Column does not support !=

[ https://issues.apache.org/jira/browse/SPARK-41351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41351. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38863

[jira] [Assigned] (SPARK-41351) Column does not support !=

[ https://issues.apache.org/jira/browse/SPARK-41351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41351: Assignee: Martin Grund > Column does not support != > -- > >

[jira] [Commented] (SPARK-41366) DF.groupby.agg() API should be compatible

[ https://issues.apache.org/jira/browse/SPARK-41366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643086#comment-17643086 ] Apache Spark commented on SPARK-41366: -- User 'grundprinzip' has created a pull request for this

[jira] [Assigned] (SPARK-41366) DF.groupby.agg() API should be compatible

[ https://issues.apache.org/jira/browse/SPARK-41366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41366: Assignee: (was: Apache Spark) > DF.groupby.agg() API should be compatible >

[jira] [Assigned] (SPARK-41366) DF.groupby.agg() API should be compatible

[ https://issues.apache.org/jira/browse/SPARK-41366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41366: Assignee: Apache Spark > DF.groupby.agg() API should be compatible >

[jira] [Resolved] (SPARK-41305) Connect Proto Completeness

[ https://issues.apache.org/jira/browse/SPARK-41305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41305. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38890

[jira] [Resolved] (SPARK-41363) implement normal functions

[ https://issues.apache.org/jira/browse/SPARK-41363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41363. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38884

[jira] [Commented] (SPARK-41378) Support Column Stats in DS V2

[ https://issues.apache.org/jira/browse/SPARK-41378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643081#comment-17643081 ] Apache Spark commented on SPARK-41378: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41378) Support Column Stats in DS V2

[ https://issues.apache.org/jira/browse/SPARK-41378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41378: Assignee: (was: Apache Spark) > Support Column Stats in DS V2 >

[jira] [Assigned] (SPARK-41378) Support Column Stats in DS V2

[ https://issues.apache.org/jira/browse/SPARK-41378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41378: Assignee: Apache Spark > Support Column Stats in DS V2 > - >

[jira] [Created] (SPARK-41379) Inconsistency of spark session in DataFrame in user function for foreachBatch sink in PySpark

Jungtaek Lim created SPARK-41379: Summary: Inconsistency of spark session in DataFrame in user function for foreachBatch sink in PySpark Key: SPARK-41379 URL: https://issues.apache.org/jira/browse/SPARK-41379

[jira] [Commented] (SPARK-41379) Inconsistency of spark session in DataFrame in user function for foreachBatch sink in PySpark

[ https://issues.apache.org/jira/browse/SPARK-41379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643076#comment-17643076 ] Jungtaek Lim commented on SPARK-41379: -- Going to submit a PR sooner. > Inconsistency of spark

[jira] [Created] (SPARK-41378) Support Column Stats in DS V2

Huaxin Gao created SPARK-41378: -- Summary: Support Column Stats in DS V2 Key: SPARK-41378 URL: https://issues.apache.org/jira/browse/SPARK-41378 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-41377) Fix spark-version-info.properties not found on Windows

[ https://issues.apache.org/jira/browse/SPARK-41377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gautham Banasandra updated SPARK-41377: --- Description: h1. *What is the problem?* The spark-version-info.properties files is

[jira] [Assigned] (SPARK-41377) Fix spark-version-info.properties not found on Windows

[ https://issues.apache.org/jira/browse/SPARK-41377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41377: Assignee: Apache Spark > Fix spark-version-info.properties not found on Windows >

[jira] [Commented] (SPARK-41377) Fix spark-version-info.properties not found on Windows

[ https://issues.apache.org/jira/browse/SPARK-41377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643020#comment-17643020 ] Apache Spark commented on SPARK-41377: -- User 'GauthamBanasandra' has created a pull request for

[jira] [Assigned] (SPARK-41377) Fix spark-version-info.properties not found on Windows

[ https://issues.apache.org/jira/browse/SPARK-41377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41377: Assignee: (was: Apache Spark) > Fix spark-version-info.properties not found on

[jira] [Created] (SPARK-41377) Fix spark-version-info.properties not found on Windows

Gautham Banasandra created SPARK-41377: -- Summary: Fix spark-version-info.properties not found on Windows Key: SPARK-41377 URL: https://issues.apache.org/jira/browse/SPARK-41377 Project: Spark

[jira] [Commented] (SPARK-41136) Shorten graceful shutdown time of ExecutorPodsSnapshotsStoreImpl to prevent blocking shutdown process

[ https://issues.apache.org/jira/browse/SPARK-41136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643019#comment-17643019 ] Apache Spark commented on SPARK-41136: -- User 'tedyu' has created a pull request for this issue:

[jira] [Commented] (SPARK-41136) Shorten graceful shutdown time of ExecutorPodsSnapshotsStoreImpl to prevent blocking shutdown process

[ https://issues.apache.org/jira/browse/SPARK-41136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643018#comment-17643018 ] Apache Spark commented on SPARK-41136: -- User 'tedyu' has created a pull request for this issue:

[jira] [Resolved] (SPARK-41313) AM shutdown hook fails with IllegalStateException if AM crashes on startup (recurrence of SPARK-3900)

[ https://issues.apache.org/jira/browse/SPARK-41313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-41313. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38832

[jira] [Assigned] (SPARK-41313) AM shutdown hook fails with IllegalStateException if AM crashes on startup (recurrence of SPARK-3900)

[ https://issues.apache.org/jira/browse/SPARK-41313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-41313: Assignee: Xing Lin > AM shutdown hook fails with IllegalStateException if AM crashes on

[jira] [Commented] (SPARK-41376) Executor netty direct memory check should respect spark.shuffle.io.preferDirectBufs

[ https://issues.apache.org/jira/browse/SPARK-41376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643005#comment-17643005 ] Apache Spark commented on SPARK-41376: -- User 'pan3793' has created a pull request for this issue:

[jira] [Commented] (SPARK-41376) Executor netty direct memory check should respect spark.shuffle.io.preferDirectBufs

[ https://issues.apache.org/jira/browse/SPARK-41376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643004#comment-17643004 ] Apache Spark commented on SPARK-41376: -- User 'pan3793' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41376) Executor netty direct memory check should respect spark.shuffle.io.preferDirectBufs

[ https://issues.apache.org/jira/browse/SPARK-41376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41376: Assignee: (was: Apache Spark) > Executor netty direct memory check should respect >

[jira] [Assigned] (SPARK-41376) Executor netty direct memory check should respect spark.shuffle.io.preferDirectBufs

[ https://issues.apache.org/jira/browse/SPARK-41376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41376: Assignee: Apache Spark > Executor netty direct memory check should respect >

[jira] [Created] (SPARK-41376) Executor netty direct memory check should respect spark.shuffle.io.preferDirectBufs

Cheng Pan created SPARK-41376: - Summary: Executor netty direct memory check should respect spark.shuffle.io.preferDirectBufs Key: SPARK-41376 URL: https://issues.apache.org/jira/browse/SPARK-41376