[jira] [Assigned] (SPARK-39992) Upgrade slf4j to 1.7.36

[ https://issues.apache.org/jira/browse/SPARK-39992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39992: Assignee: Bjørn Jørgensen > Upgrade slf4j to 1.7.36 > --- > >

[jira] [Resolved] (SPARK-39992) Upgrade slf4j to 1.7.36

[ https://issues.apache.org/jira/browse/SPARK-39992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39992. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37422

[jira] [Updated] (SPARK-39994) How to write (save) PySpark dataframe containing vector column?

[ https://issues.apache.org/jira/browse/SPARK-39994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Kaleem Ullah updated SPARK-39994: -- Attachment: (was: df.PNG) > How to write (save) PySpark dataframe

[jira] [Created] (SPARK-39994) How to write (save) PySpark dataframe containing vector column?

Muhammad Kaleem Ullah created SPARK-39994: - Summary: How to write (save) PySpark dataframe containing vector column? Key: SPARK-39994 URL: https://issues.apache.org/jira/browse/SPARK-39994

[jira] [Updated] (SPARK-39994) How to write (save) PySpark dataframe containing vector column?

[ https://issues.apache.org/jira/browse/SPARK-39994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Kaleem Ullah updated SPARK-39994: -- Attachment: df.PNG error.PNG > How to write (save) PySpark

[jira] [Updated] (SPARK-39994) How to write (save) PySpark dataframe containing vector column?

[ https://issues.apache.org/jira/browse/SPARK-39994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Kaleem Ullah updated SPARK-39994: -- Attachment: df.PNG > How to write (save) PySpark dataframe containing vector

[jira] [Commented] (SPARK-39991) AQE should use available column statistics from completed query stages

[ https://issues.apache.org/jira/browse/SPARK-39991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576062#comment-17576062 ] Apache Spark commented on SPARK-39991: -- User 'andygrove' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39991) AQE should use available column statistics from completed query stages

[ https://issues.apache.org/jira/browse/SPARK-39991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39991: Assignee: (was: Apache Spark) > AQE should use available column statistics from

[jira] [Assigned] (SPARK-39991) AQE should use available column statistics from completed query stages

[ https://issues.apache.org/jira/browse/SPARK-39991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39991: Assignee: Apache Spark > AQE should use available column statistics from completed query

[jira] [Updated] (SPARK-39993) Spark on Kubernetes doesn't filter data by date

[ https://issues.apache.org/jira/browse/SPARK-39993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanna Liashchuk updated SPARK-39993: Priority: Blocker (was: Major) > Spark on Kubernetes doesn't filter data by date >

[jira] [Created] (SPARK-39993) Spark on Kubernetes doesn't filter data by date

Hanna Liashchuk created SPARK-39993: --- Summary: Spark on Kubernetes doesn't filter data by date Key: SPARK-39993 URL: https://issues.apache.org/jira/browse/SPARK-39993 Project: Spark Issue

[jira] [Commented] (SPARK-39985) Test DEFAULT column values with DataFrames

[ https://issues.apache.org/jira/browse/SPARK-39985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576044#comment-17576044 ] Apache Spark commented on SPARK-39985: -- User 'dtenedor' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39985) Test DEFAULT column values with DataFrames

[ https://issues.apache.org/jira/browse/SPARK-39985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39985: Assignee: (was: Apache Spark) > Test DEFAULT column values with DataFrames >

[jira] [Assigned] (SPARK-39985) Test DEFAULT column values with DataFrames

[ https://issues.apache.org/jira/browse/SPARK-39985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39985: Assignee: Apache Spark > Test DEFAULT column values with DataFrames >

[jira] [Commented] (SPARK-39992) Upgrade slf4j to 1.7.36

[ https://issues.apache.org/jira/browse/SPARK-39992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576028#comment-17576028 ] Apache Spark commented on SPARK-39992: -- User 'bjornjorgensen' has created a pull request for this

[jira] [Assigned] (SPARK-39992) Upgrade slf4j to 1.7.36

[ https://issues.apache.org/jira/browse/SPARK-39992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39992: Assignee: (was: Apache Spark) > Upgrade slf4j to 1.7.36 > --- >

[jira] [Assigned] (SPARK-39992) Upgrade slf4j to 1.7.36

[ https://issues.apache.org/jira/browse/SPARK-39992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39992: Assignee: Apache Spark > Upgrade slf4j to 1.7.36 > --- > >

[jira] [Commented] (SPARK-39992) Upgrade slf4j to 1.7.36

[ https://issues.apache.org/jira/browse/SPARK-39992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576027#comment-17576027 ] Apache Spark commented on SPARK-39992: -- User 'bjornjorgensen' has created a pull request for this

[jira] [Created] (SPARK-39992) Upgrade slf4j to 1.7.36

Bjørn Jørgensen created SPARK-39992: --- Summary: Upgrade slf4j to 1.7.36 Key: SPARK-39992 URL: https://issues.apache.org/jira/browse/SPARK-39992 Project: Spark Issue Type: Dependency upgrade

[jira] [Updated] (SPARK-39991) AQE should use available column statistics from completed query stages

[ https://issues.apache.org/jira/browse/SPARK-39991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Grove updated SPARK-39991: --- Description: In QueryStageExec.computeStats we copy partial statistics from materlized query

[jira] [Created] (SPARK-39991) AQE should use available column statistics from completed query stages

Andy Grove created SPARK-39991: -- Summary: AQE should use available column statistics from completed query stages Key: SPARK-39991 URL: https://issues.apache.org/jira/browse/SPARK-39991 Project: Spark

[jira] [Assigned] (SPARK-39987) Support PEAK_JVM_(ON|OFF)HEAP_MEMORY executor rolling policy

[ https://issues.apache.org/jira/browse/SPARK-39987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-39987: - Assignee: Dongjoon Hyun > Support PEAK_JVM_(ON|OFF)HEAP_MEMORY executor rolling policy

[jira] [Resolved] (SPARK-39987) Support PEAK_JVM_(ON|OFF)HEAP_MEMORY executor rolling policy

[ https://issues.apache.org/jira/browse/SPARK-39987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39987. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37418

[jira] [Resolved] (SPARK-39980) Change infra image to static tag

[ https://issues.apache.org/jira/browse/SPARK-39980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39980. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37403

[jira] [Assigned] (SPARK-39980) Change infra image to static tag

[ https://issues.apache.org/jira/browse/SPARK-39980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39980: Assignee: Yikun Jiang > Change infra image to static tag >

[jira] [Updated] (SPARK-39990) Restrict special characters in field name, which can be controlled by switches

[ https://issues.apache.org/jira/browse/SPARK-39990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-39990: -- Description: The hive metastore restricts field name to only contain alphanumerics and underscores. If the

[jira] [Updated] (SPARK-39990) Restrict special characters in field name, which can be controlled by switches

[ https://issues.apache.org/jira/browse/SPARK-39990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-39990: -- Description: The hive metastore restricts field name to only contain alphanumerics and underscores. If the

[jira] [Updated] (SPARK-39990) Restrict special characters in field name, which can be controlled by switches

[ https://issues.apache.org/jira/browse/SPARK-39990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-39990: -- Description: The hive metastore restricts field name to only contain alphanumerics and underscores. If the

[jira] [Updated] (SPARK-39990) Restrict special characters in field name, which can be controlled by switches

[ https://issues.apache.org/jira/browse/SPARK-39990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-39990: -- Description: The hive metastore restricts field name to only contain alphanumerics and underscores. If the

[jira] [Updated] (SPARK-39990) Restrict special characters in field name, which can be controlled by switches

[ https://issues.apache.org/jira/browse/SPARK-39990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-39990: -- Description: The hive metastore restricts field name to only contain alphanumerics and underscores. If the

[jira] [Created] (SPARK-39990) Restrict special characters in field name, which can be controlled by switches

melin created SPARK-39990: - Summary: Restrict special characters in field name, which can be controlled by switches Key: SPARK-39990 URL: https://issues.apache.org/jira/browse/SPARK-39990 Project: Spark

[jira] [Resolved] (SPARK-38034) Optimize time complexity and extend applicable cases for TransposeWindow

[ https://issues.apache.org/jira/browse/SPARK-38034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38034. - Fix Version/s: 3.4.0 3.3.1 3.2.3 Assignee: zhou

[jira] [Assigned] (SPARK-39964) DS V2 pushdown should unify the translate path

[ https://issues.apache.org/jira/browse/SPARK-39964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39964: --- Assignee: jiaan.geng > DS V2 pushdown should unify the translate path >

[jira] [Resolved] (SPARK-39964) DS V2 pushdown should unify the translate path

[ https://issues.apache.org/jira/browse/SPARK-39964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39964. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37391

[jira] [Assigned] (SPARK-39963) Simplify the implementation of SimplifyCasts

[ https://issues.apache.org/jira/browse/SPARK-39963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39963: --- Assignee: jiaan.geng > Simplify the implementation of SimplifyCasts >

[jira] [Resolved] (SPARK-39963) Simplify the implementation of SimplifyCasts

[ https://issues.apache.org/jira/browse/SPARK-39963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39963. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37389

[jira] [Resolved] (SPARK-38901) DS V2 supports push down misc functions

[ https://issues.apache.org/jira/browse/SPARK-38901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38901. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37169

[jira] [Assigned] (SPARK-39867) Global limit should not inherit OrderPreservingUnaryNode

[ https://issues.apache.org/jira/browse/SPARK-39867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39867: --- Assignee: XiDuo You > Global limit should not inherit OrderPreservingUnaryNode >

[jira] [Assigned] (SPARK-38901) DS V2 supports push down misc functions

[ https://issues.apache.org/jira/browse/SPARK-38901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38901: --- Assignee: Zhixiong Chen > DS V2 supports push down misc functions >

[jira] [Updated] (SPARK-39867) Global limit should not inherit OrderPreservingUnaryNode

[ https://issues.apache.org/jira/browse/SPARK-39867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-39867: Fix Version/s: 3.1.4 3.2.3 > Global limit should not inherit

[jira] [Resolved] (SPARK-39981) CheckOverflowInTableInsert returns exception rather than throwing it

[ https://issues.apache.org/jira/browse/SPARK-39981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39981. -- Fix Version/s: 3.3.1 3.4.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-39981) CheckOverflowInTableInsert returns exception rather than throwing it

[ https://issues.apache.org/jira/browse/SPARK-39981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39981: Assignee: Hyukjin Kwon > CheckOverflowInTableInsert returns exception rather than

[jira] [Commented] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575658#comment-17575658 ] Dongjoon Hyun commented on SPARK-39965: --- In addition, the following claim sounds invalid. As I

[jira] [Commented] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575646#comment-17575646 ] Dongjoon Hyun commented on SPARK-39965: --- Thank you for pinging me. I'll take a look. > Spark on

[jira] [Commented] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575655#comment-17575655 ] Dongjoon Hyun commented on SPARK-39965: --- Actually, Spark doesn't throw exception. It just logs the

[jira] [Assigned] (SPARK-39989) Support estimate column statistics if it is foldable expression

[ https://issues.apache.org/jira/browse/SPARK-39989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39989: Assignee: (was: Apache Spark) > Support estimate column statistics if it is foldable

[jira] [Assigned] (SPARK-39989) Support estimate column statistics if it is foldable expression

[ https://issues.apache.org/jira/browse/SPARK-39989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39989: Assignee: Apache Spark > Support estimate column statistics if it is foldable expression

[jira] [Commented] (SPARK-39989) Support estimate column statistics if it is foldable expression

[ https://issues.apache.org/jira/browse/SPARK-39989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575647#comment-17575647 ] Apache Spark commented on SPARK-39989: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-39989) Support estimate column statistics if it is foldable expression

Yuming Wang created SPARK-39989: --- Summary: Support estimate column statistics if it is foldable expression Key: SPARK-39989 URL: https://issues.apache.org/jira/browse/SPARK-39989 Project: Spark