[jira] [Created] (SPARK-39020) [CVE-2020-9480] Transitive dependency "unused" from spark-sql_2.12 highlight as vulnerable in dependency tracker

2022-04-26 Thread Sundar (Jira)
Sundar created SPARK-39020: -- Summary: [CVE-2020-9480] Transitive dependency "unused" from spark-sql_2.12 highlight as vulnerable in dependency tracker Key: SPARK-39020 URL: https://issues.apache.org/jira/browse/SPARK-

[jira] [Updated] (SPARK-39020) [CVE-2020-9480] Transitive dependency "unused" from spark-sql_2.12 highlight as vulnerable in dependency tracker

2022-04-26 Thread Sundar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sundar updated SPARK-39020: --- Attachment: Dependency-Track.png > [CVE-2020-9480] Transitive dependency "unused" from spark-sql_2.12 >

[jira] [Updated] (SPARK-39020) [CVE-2020-9480] Transitive dependency "unused" from spark-sql_2.12 highlight as vulnerable in dependency tracker

2022-04-26 Thread Sundar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sundar updated SPARK-39020: --- Description: I am using spark-sql_2.12 dependency version 3.2.1 in my project. My dependency tracker highli

[jira] [Created] (SPARK-39021) Timestamp data is converted to machine time inside UDF.

2022-04-26 Thread Xiaoming (Jira)
Xiaoming created SPARK-39021: Summary: Timestamp data is converted to machine time inside UDF. Key: SPARK-39021 URL: https://issues.apache.org/jira/browse/SPARK-39021 Project: Spark Issue Type: B

[jira] [Assigned] (SPARK-39019) Refactor `SPARK-37463: read/write Timestamp ntz to Orc with different time zone` to use `TempPath`

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39019: Assignee: Apache Spark > Refactor `SPARK-37463: read/write Timestamp ntz to Orc with diff

[jira] [Commented] (SPARK-39019) Refactor `SPARK-37463: read/write Timestamp ntz to Orc with different time zone` to use `TempPath`

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527952#comment-17527952 ] Apache Spark commented on SPARK-39019: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-39019) Refactor `SPARK-37463: read/write Timestamp ntz to Orc with different time zone` to use `TempPath`

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39019: Assignee: (was: Apache Spark) > Refactor `SPARK-37463: read/write Timestamp ntz to Or

[jira] [Commented] (SPARK-37463) Read/Write Timestamp ntz from/to Orc uses int64

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527953#comment-17527953 ] Apache Spark commented on SPARK-37463: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-38946) test_eval failed due to pandas reference changed

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527954#comment-17527954 ] Apache Spark commented on SPARK-38946: -- User 'Yikun' has created a pull request for

[jira] [Assigned] (SPARK-38946) test_eval failed due to pandas reference changed

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38946: Assignee: (was: Apache Spark) > test_eval failed due to pandas reference changed > --

[jira] [Assigned] (SPARK-38946) test_eval failed due to pandas reference changed

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38946: Assignee: Apache Spark > test_eval failed due to pandas reference changed > -

[jira] [Commented] (SPARK-38946) test_eval failed due to pandas reference changed

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527955#comment-17527955 ] Apache Spark commented on SPARK-38946: -- User 'Yikun' has created a pull request for

[jira] [Commented] (SPARK-34079) Merge non-correlated scalar subqueries

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527957#comment-17527957 ] Apache Spark commented on SPARK-34079: -- User 'peter-toth' has created a pull reques

[jira] [Commented] (SPARK-34079) Merge non-correlated scalar subqueries

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527958#comment-17527958 ] Apache Spark commented on SPARK-34079: -- User 'peter-toth' has created a pull reques

[jira] [Commented] (SPARK-38690) Use error classes in the compilation errors of SHOW CREATE TABLE

2022-04-26 Thread panbingkun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527960#comment-17527960 ] panbingkun commented on SPARK-38690: I will work on this. Thanks [~maxgekk]  > Use

[jira] [Commented] (SPARK-38982) test_categories_setter failed due to pandas bug

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527969#comment-17527969 ] Apache Spark commented on SPARK-38982: -- User 'Yikun' has created a pull request for

[jira] [Assigned] (SPARK-38982) test_categories_setter failed due to pandas bug

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38982: Assignee: (was: Apache Spark) > test_categories_setter failed due to pandas bug > ---

[jira] [Assigned] (SPARK-38982) test_categories_setter failed due to pandas bug

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38982: Assignee: Apache Spark > test_categories_setter failed due to pandas bug > --

[jira] [Assigned] (SPARK-38821) test_nsmallest test failed tue to pandas 1.4.0-1.4.2 bug

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38821: Assignee: Apache Spark > test_nsmallest test failed tue to pandas 1.4.0-1.4.2 bug > -

[jira] [Commented] (SPARK-38821) test_nsmallest test failed tue to pandas 1.4.0-1.4.2 bug

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527983#comment-17527983 ] Apache Spark commented on SPARK-38821: -- User 'Yikun' has created a pull request for

[jira] [Assigned] (SPARK-38821) test_nsmallest test failed tue to pandas 1.4.0-1.4.2 bug

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38821: Assignee: (was: Apache Spark) > test_nsmallest test failed tue to pandas 1.4.0-1.4.2

[jira] [Updated] (SPARK-38820) Refresh

2022-04-26 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-38820: Summary: Refresh (was: Support Index can hold arbitrary ExtensionArrays) > Refresh > >

[jira] [Updated] (SPARK-38820) Refresh as_category

2022-04-26 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-38820: Summary: Refresh as_category (was: Refresh ) > Refresh as_category > --- > >

[jira] [Updated] (SPARK-38820) Refresh dtype in astype("category")

2022-04-26 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-38820: Summary: Refresh dtype in astype("category") (was: Refresh as_category) > Refresh dtype in astype

[jira] [Created] (SPARK-39022) Spark SQL - Combination of HAVING and SORT not resolved correctly

2022-04-26 Thread Lukas Grasmann (Jira)
Lukas Grasmann created SPARK-39022: -- Summary: Spark SQL - Combination of HAVING and SORT not resolved correctly Key: SPARK-39022 URL: https://issues.apache.org/jira/browse/SPARK-39022 Project: Spark

[jira] [Updated] (SPARK-39022) Spark SQL - Combination of HAVING and SORT not resolved correctly

2022-04-26 Thread Lukas Grasmann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Grasmann updated SPARK-39022: --- Attachment: explain_new.txt explain_old.txt > Spark SQL - Combination of HAV

[jira] [Updated] (SPARK-38820) Refresh dtype when astype("category")

2022-04-26 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-38820: Summary: Refresh dtype when astype("category") (was: Refresh dtype in astype("category")) > Refr

[jira] [Updated] (SPARK-39022) Spark SQL - Combination of HAVING and SORT not resolved correctly

2022-04-26 Thread Lukas Grasmann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Grasmann updated SPARK-39022: --- Description: Example: Given a simple relation {{test}} with two relevant columns {{hotel}}

[jira] [Comment Edited] (SPARK-38820) Refresh dtype when astype("category")

2022-04-26 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17525648#comment-17525648 ] Yikun Jiang edited comment on SPARK-38820 at 4/26/22 9:10 AM:

[jira] [Resolved] (SPARK-38995) Format Config in Kubernetes Module

2022-04-26 Thread binjie yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binjie yang resolved SPARK-38995. - Resolution: Won't Fix Will fix with another related PR > Format Config in Kubernetes Module > -

[jira] [Commented] (SPARK-38820) Refresh dtype when astype("category")

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528037#comment-17528037 ] Apache Spark commented on SPARK-38820: -- User 'Yikun' has created a pull request for

[jira] [Assigned] (SPARK-38820) Refresh dtype when astype("category")

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38820: Assignee: (was: Apache Spark) > Refresh dtype when astype("category") > -

[jira] [Assigned] (SPARK-38820) Refresh dtype when astype("category")

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38820: Assignee: Apache Spark > Refresh dtype when astype("category") >

[jira] [Commented] (SPARK-38820) Refresh dtype when astype("category")

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528040#comment-17528040 ] Apache Spark commented on SPARK-38820: -- User 'Yikun' has created a pull request for

[jira] [Created] (SPARK-39023) Add Executor Pod inter-pod anti-addinity

2022-04-26 Thread binjie yang (Jira)
binjie yang created SPARK-39023: --- Summary: Add Executor Pod inter-pod anti-addinity Key: SPARK-39023 URL: https://issues.apache.org/jira/browse/SPARK-39023 Project: Spark Issue Type: New Featur

[jira] [Updated] (SPARK-39023) Add Executor Pod inter-pod anti-affinity

2022-04-26 Thread binjie yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binjie yang updated SPARK-39023: Summary: Add Executor Pod inter-pod anti-affinity (was: Add Executor Pod inter-pod anti-addinity)

[jira] [Created] (SPARK-39024) Add support for YARN decommissioning when ESS is Enabled

2022-04-26 Thread Abhishek Dixit (Jira)
Abhishek Dixit created SPARK-39024: -- Summary: Add support for YARN decommissioning when ESS is Enabled Key: SPARK-39024 URL: https://issues.apache.org/jira/browse/SPARK-39024 Project: Spark

[jira] [Commented] (SPARK-39023) Add Executor Pod inter-pod anti-affinity

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528055#comment-17528055 ] Apache Spark commented on SPARK-39023: -- User 'zwangsheng' has created a pull reques

[jira] [Assigned] (SPARK-39023) Add Executor Pod inter-pod anti-affinity

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39023: Assignee: (was: Apache Spark) > Add Executor Pod inter-pod anti-affinity > --

[jira] [Assigned] (SPARK-39023) Add Executor Pod inter-pod anti-affinity

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39023: Assignee: Apache Spark > Add Executor Pod inter-pod anti-affinity > -

[jira] [Commented] (SPARK-39023) Add Executor Pod inter-pod anti-affinity

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528056#comment-17528056 ] Apache Spark commented on SPARK-39023: -- User 'zwangsheng' has created a pull reques

[jira] [Commented] (SPARK-39022) Spark SQL - Combination of HAVING and SORT not resolved correctly

2022-04-26 Thread Lukas Grasmann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528071#comment-17528071 ] Lukas Grasmann commented on SPARK-39022: I am working on a Pull Request for this

[jira] [Assigned] (SPARK-38821) test_nsmallest test failed tue to pandas 1.4.0-1.4.2 bug

2022-04-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38821: Assignee: Yikun Jiang > test_nsmallest test failed tue to pandas 1.4.0-1.4.2 bug > --

[jira] [Resolved] (SPARK-38821) test_nsmallest test failed tue to pandas 1.4.0-1.4.2 bug

2022-04-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38821. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36356 [https://gi

[jira] [Created] (SPARK-39025) Add k8s Scheduling, Preemption and Eviction feature to apache spark

2022-04-26 Thread qian (Jira)
qian created SPARK-39025: Summary: Add k8s Scheduling, Preemption and Eviction feature to apache spark Key: SPARK-39025 URL: https://issues.apache.org/jira/browse/SPARK-39025 Project: Spark Issue Ty

[jira] [Commented] (SPARK-39025) Add k8s Scheduling, Preemption and Eviction feature to apache spark

2022-04-26 Thread qian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528081#comment-17528081 ] qian commented on SPARK-39025: -- I am working on this. Please assign to me [~dongjoon]  [~hy

[jira] [Created] (SPARK-39026) Add k8s tolerations support for apache spark

2022-04-26 Thread qian (Jira)
qian created SPARK-39026: Summary: Add k8s tolerations support for apache spark Key: SPARK-39026 URL: https://issues.apache.org/jira/browse/SPARK-39026 Project: Spark Issue Type: Sub-task C

[jira] [Assigned] (SPARK-39026) Add k8s tolerations support for apache spark

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39026: Assignee: Apache Spark > Add k8s tolerations support for apache spark > -

[jira] [Assigned] (SPARK-39026) Add k8s tolerations support for apache spark

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39026: Assignee: (was: Apache Spark) > Add k8s tolerations support for apache spark > --

[jira] [Commented] (SPARK-39026) Add k8s tolerations support for apache spark

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528087#comment-17528087 ] Apache Spark commented on SPARK-39026: -- User 'dcoliversun' has created a pull reque

[jira] [Commented] (SPARK-39025) Add k8s Scheduling, Preemption and Eviction feature to apache spark

2022-04-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528088#comment-17528088 ] Hyukjin Kwon commented on SPARK-39025: -- You don't have to be assigned. Once you ope

[jira] [Created] (SPARK-39027) Output SQL statements in upper case in error messages

2022-04-26 Thread Max Gekk (Jira)
Max Gekk created SPARK-39027: Summary: Output SQL statements in upper case in error messages Key: SPARK-39027 URL: https://issues.apache.org/jira/browse/SPARK-39027 Project: Spark Issue Type: Sub

[jira] [Updated] (SPARK-39027) Output SQL statements in upper case in error messages

2022-04-26 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39027: - Description: Output SQL statements in upper case ONLY in error messages. For example, the error message

[jira] [Commented] (SPARK-39027) Output SQL statements in upper case in error messages

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528089#comment-17528089 ] Apache Spark commented on SPARK-39027: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-39027) Output SQL statements in upper case in error messages

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39027: Assignee: Max Gekk (was: Apache Spark) > Output SQL statements in upper case in error me

[jira] [Assigned] (SPARK-39027) Output SQL statements in upper case in error messages

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39027: Assignee: Apache Spark (was: Max Gekk) > Output SQL statements in upper case in error me

[jira] [Commented] (SPARK-39025) Add k8s Scheduling, Preemption and Eviction feature to apache spark

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528141#comment-17528141 ] Apache Spark commented on SPARK-39025: -- User 'dcoliversun' has created a pull reque

[jira] [Assigned] (SPARK-39025) Add k8s Scheduling, Preemption and Eviction feature to apache spark

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39025: Assignee: Apache Spark > Add k8s Scheduling, Preemption and Eviction feature to apache sp

[jira] [Commented] (SPARK-39025) Add k8s Scheduling, Preemption and Eviction feature to apache spark

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528142#comment-17528142 ] Apache Spark commented on SPARK-39025: -- User 'dcoliversun' has created a pull reque

[jira] [Assigned] (SPARK-39025) Add k8s Scheduling, Preemption and Eviction feature to apache spark

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39025: Assignee: (was: Apache Spark) > Add k8s Scheduling, Preemption and Eviction feature t

[jira] [Commented] (SPARK-34079) Merge non-correlated scalar subqueries

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528164#comment-17528164 ] Apache Spark commented on SPARK-34079: -- User 'peter-toth' has created a pull reques

[jira] [Assigned] (SPARK-38944) Close `NioBufferedFileInputStream` opened by `ExternalAppendOnlyUnsafeRowArraySuite`

2022-04-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38944: Assignee: Yang Jie > Close `NioBufferedFileInputStream` opened by > `ExternalAppendOnlyU

[jira] [Resolved] (SPARK-38944) Close `NioBufferedFileInputStream` opened by `ExternalAppendOnlyUnsafeRowArraySuite`

2022-04-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38944. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36262 [https://gi

[jira] [Updated] (SPARK-37222) Max iterations reached in Operator Optimization w/left_anti or left_semi join and nested structures

2022-04-26 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-37222: - Attachment: plan-log.log > Max iterations reached in Operator Optimization w/left_anti o

[jira] [Commented] (SPARK-37222) Max iterations reached in Operator Optimization w/left_anti or left_semi join and nested structures

2022-04-26 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528233#comment-17528233 ] Nicholas Chammas commented on SPARK-37222: -- I've found a helpful log setting th

[jira] [Created] (SPARK-39028) Use SparkDateTimeException when casting to datetime types failed

2022-04-26 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-39028: -- Summary: Use SparkDateTimeException when casting to datetime types failed Key: SPARK-39028 URL: https://issues.apache.org/jira/browse/SPARK-39028 Project: Spark

[jira] [Comment Edited] (SPARK-37222) Max iterations reached in Operator Optimization w/left_anti or left_semi join and nested structures

2022-04-26 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528233#comment-17528233 ] Nicholas Chammas edited comment on SPARK-37222 at 4/26/22 3:44 PM: ---

[jira] [Commented] (SPARK-39028) Use SparkDateTimeException when casting to datetime types failed

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528258#comment-17528258 ] Apache Spark commented on SPARK-39028: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-39028) Use SparkDateTimeException when casting to datetime types failed

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39028: Assignee: Apache Spark (was: Gengliang Wang) > Use SparkDateTimeException when casting t

[jira] [Commented] (SPARK-39028) Use SparkDateTimeException when casting to datetime types failed

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528257#comment-17528257 ] Apache Spark commented on SPARK-39028: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-39028) Use SparkDateTimeException when casting to datetime types failed

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39028: Assignee: Gengliang Wang (was: Apache Spark) > Use SparkDateTimeException when casting t

[jira] [Created] (SPARK-39029) Improve the test coverage for pyspark/broadcast.py

2022-04-26 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-39029: Summary: Improve the test coverage for pyspark/broadcast.py Key: SPARK-39029 URL: https://issues.apache.org/jira/browse/SPARK-39029 Project: Spark Issue Type

[jira] [Commented] (SPARK-39029) Improve the test coverage for pyspark/broadcast.py

2022-04-26 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528269#comment-17528269 ] pralabhkumar commented on SPARK-39029: -- I am working on this .  > Improve the test

[jira] [Assigned] (SPARK-39016) Fix compilation warnings related to "`enum` will become a keyword in Scala 3"

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39016: Assignee: (was: Apache Spark) > Fix compilation warnings related to "`enum` will beco

[jira] [Assigned] (SPARK-39016) Fix compilation warnings related to "`enum` will become a keyword in Scala 3"

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39016: Assignee: Apache Spark > Fix compilation warnings related to "`enum` will become a keywor

[jira] [Commented] (SPARK-39016) Fix compilation warnings related to "`enum` will become a keyword in Scala 3"

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528274#comment-17528274 ] Apache Spark commented on SPARK-39016: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-39016) Fix compilation warnings related to "`enum` will become a keyword in Scala 3"

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528275#comment-17528275 ] Apache Spark commented on SPARK-39016: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-39016) Fix compilation warnings related to "`enum` will become a keyword in Scala 3"

2022-04-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-39016: Assignee: Yang Jie > Fix compilation warnings related to "`enum` will become a keyword in

[jira] [Resolved] (SPARK-39016) Fix compilation warnings related to "`enum` will become a keyword in Scala 3"

2022-04-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-39016. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36349 [https://gi

[jira] [Commented] (SPARK-39027) Output SQL statements in upper case in error messages

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528350#comment-17528350 ] Apache Spark commented on SPARK-39027: -- User 'MaxGekk' has created a pull request f

[jira] [Resolved] (SPARK-39028) Use SparkDateTimeException when casting to datetime types failed

2022-04-26 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-39028. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36362 [https://github.com

[jira] [Created] (SPARK-39030) Remame sum

2022-04-26 Thread Jira
Bjørn Jørgensen created SPARK-39030: --- Summary: Remame sum Key: SPARK-39030 URL: https://issues.apache.org/jira/browse/SPARK-39030 Project: Spark Issue Type: Improvement Components

[jira] [Assigned] (SPARK-39030) Remame sum

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39030: Assignee: (was: Apache Spark) > Remame sum > -- > > Key: SPAR

[jira] [Commented] (SPARK-39030) Remame sum

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528396#comment-17528396 ] Apache Spark commented on SPARK-39030: -- User 'bjornjorgensen' has created a pull re

[jira] [Assigned] (SPARK-39030) Remame sum

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39030: Assignee: Apache Spark > Remame sum > -- > > Key: SPARK-39030 >

[jira] [Commented] (SPARK-39030) Remame sum

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528397#comment-17528397 ] Apache Spark commented on SPARK-39030: -- User 'bjornjorgensen' has created a pull re

[jira] [Created] (SPARK-39031) NaN != NaN in pivot

2022-04-26 Thread Robert Joseph Evans (Jira)
Robert Joseph Evans created SPARK-39031: --- Summary: NaN != NaN in pivot Key: SPARK-39031 URL: https://issues.apache.org/jira/browse/SPARK-39031 Project: Spark Issue Type: Bug C

[jira] [Created] (SPARK-39032) Incorrectly formatted examples in pyspark.sql.functions.when

2022-04-26 Thread Vadim Patsalo (Jira)
Vadim Patsalo created SPARK-39032: - Summary: Incorrectly formatted examples in pyspark.sql.functions.when Key: SPARK-39032 URL: https://issues.apache.org/jira/browse/SPARK-39032 Project: Spark

[jira] [Updated] (SPARK-39032) Incorrectly formatted examples in pyspark.sql.functions.when

2022-04-26 Thread Vadim Patsalo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vadim Patsalo updated SPARK-39032: -- Attachment: Screen Shot 2022-04-26 at 14.06.47.png > Incorrectly formatted examples in pyspark

[jira] [Updated] (SPARK-39032) Incorrectly formatted examples in pyspark.sql.functions.when

2022-04-26 Thread Vadim Patsalo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vadim Patsalo updated SPARK-39032: -- Description: Missing `Examples\n`.     was: Missing `Examples\n`.   !im

[jira] [Commented] (SPARK-28516) Data Type Formatting Functions: `to_char`

2022-04-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528418#comment-17528418 ] Apache Spark commented on SPARK-28516: -- User 'dtenedor' has created a pull request

[jira] [Updated] (SPARK-39025) Add k8s Scheduling, Preemption and Eviction feature to apache spark

2022-04-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39025: -- Fix Version/s: (was: 3.4.0) > Add k8s Scheduling, Preemption and Eviction feature to apach

[jira] [Commented] (SPARK-39025) Add k8s Scheduling, Preemption and Eviction feature to apache spark

2022-04-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528434#comment-17528434 ] Dongjoon Hyun commented on SPARK-39025: --- +1 for [~hyukjin.kwon]'s explanation. BTW

[jira] [Commented] (SPARK-39025) Add k8s Scheduling, Preemption and Eviction feature to apache spark

2022-04-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528436#comment-17528436 ] Dongjoon Hyun commented on SPARK-39025: --- AFAIK, this claim is wrong because we alr

[jira] [Updated] (SPARK-39019) Refactor `SPARK-37463: read/write Timestamp ntz to Orc with different time zone` to use `TempPath`

2022-04-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39019: -- Issue Type: Test (was: Improvement) > Refactor `SPARK-37463: read/write Timestamp ntz to Orc

[jira] [Resolved] (SPARK-39019) Refactor `SPARK-37463: read/write Timestamp ntz to Orc with different time zone` to use `TempPath`

2022-04-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39019. --- Fix Version/s: 3.3.0 Assignee: Yang Jie Resolution: Fixed This is resolved v

[jira] [Updated] (SPARK-39019) Use `withTempPath` to clean up temporary data directory after `SPARK-37463: read/write Timestamp ntz to Orc with different time zone`

2022-04-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39019: -- Summary: Use `withTempPath` to clean up temporary data directory after `SPARK-37463: read/writ

[jira] [Updated] (SPARK-39019) Refactor `SPARK-37463: read/write Timestamp ntz to Orc with different time zone` to use `TempPath`

2022-04-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39019: -- Affects Version/s: 3.3.0 (was: 3.4.0) > Refactor `SPARK-37463: read

[jira] [Reopened] (SPARK-38354) Add hash probes metrics for shuffled hash join

2022-04-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-38354: --- Assignee: (was: Cheng Su) This is reverted due to the regression via https://github.c

[jira] [Updated] (SPARK-38354) Add hash probes metrics for shuffled hash join

2022-04-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38354: -- Fix Version/s: (was: 3.3.0) > Add hash probes metrics for shuffled hash join > ---

  1   2   >