[jira] [Assigned] (SPARK-36850) Migrate CreateTableStatement to v2 command framework

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36850: Assignee: (was: Apache Spark) > Migrate CreateTableStatement to v2 command framework

[jira] [Assigned] (SPARK-36850) Migrate CreateTableStatement to v2 command framework

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36850: Assignee: Apache Spark > Migrate CreateTableStatement to v2 command framework >

[jira] [Commented] (SPARK-36850) Migrate CreateTableStatement to v2 command framework

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420048#comment-17420048 ] Apache Spark commented on SPARK-36850: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Created] (SPARK-36850) Migrate CreateTableStatement to v2 command framework

2021-09-24 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-36850: -- Summary: Migrate CreateTableStatement to v2 command framework Key: SPARK-36850 URL: https://issues.apache.org/jira/browse/SPARK-36850 Project: Spark Issue Type:

[jira] [Created] (SPARK-36849) Migrate UseStatement to v2 command framework

2021-09-24 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-36849: -- Summary: Migrate UseStatement to v2 command framework Key: SPARK-36849 URL: https://issues.apache.org/jira/browse/SPARK-36849 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36849) Migrate UseStatement to v2 command framework

2021-09-24 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420041#comment-17420041 ] Huaxin Gao commented on SPARK-36849: I am working on this > Migrate UseStatement to v2 command

[jira] [Commented] (SPARK-36848) Migrate ShowCurrentNamespaceStatement to v2 command framework

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420040#comment-17420040 ] Apache Spark commented on SPARK-36848: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36848) Migrate ShowCurrentNamespaceStatement to v2 command framework

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36848: Assignee: (was: Apache Spark) > Migrate ShowCurrentNamespaceStatement to v2 command

[jira] [Commented] (SPARK-36848) Migrate ShowCurrentNamespaceStatement to v2 command framework

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420039#comment-17420039 ] Apache Spark commented on SPARK-36848: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36848) Migrate ShowCurrentNamespaceStatement to v2 command framework

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36848: Assignee: Apache Spark > Migrate ShowCurrentNamespaceStatement to v2 command framework >

[jira] [Created] (SPARK-36848) Migrate ShowCurrentNamespaceStatement to v2 command framework

2021-09-24 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-36848: -- Summary: Migrate ShowCurrentNamespaceStatement to v2 command framework Key: SPARK-36848 URL: https://issues.apache.org/jira/browse/SPARK-36848 Project: Spark

[jira] [Commented] (SPARK-32712) Support writing Hive non-ORC/Parquet bucketed table

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420037#comment-17420037 ] Apache Spark commented on SPARK-32712: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32712) Support writing Hive non-ORC/Parquet bucketed table

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32712: Assignee: (was: Apache Spark) > Support writing Hive non-ORC/Parquet bucketed table

[jira] [Assigned] (SPARK-32712) Support writing Hive non-ORC/Parquet bucketed table

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32712: Assignee: Apache Spark > Support writing Hive non-ORC/Parquet bucketed table >

[jira] [Commented] (SPARK-32712) Support writing Hive non-ORC/Parquet bucketed table

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420036#comment-17420036 ] Apache Spark commented on SPARK-32712: -- User 'c21' has created a pull request for this issue:

[jira] [Commented] (SPARK-36847) Explicitly specify error codes when ignoring type hint errors

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420029#comment-17420029 ] Apache Spark commented on SPARK-36847: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-36847) Explicitly specify error codes when ignoring type hint errors

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420027#comment-17420027 ] Apache Spark commented on SPARK-36847: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36847) Explicitly specify error codes when ignoring type hint errors

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36847: Assignee: Apache Spark > Explicitly specify error codes when ignoring type hint errors >

[jira] [Assigned] (SPARK-36847) Explicitly specify error codes when ignoring type hint errors

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36847: Assignee: (was: Apache Spark) > Explicitly specify error codes when ignoring type

[jira] [Created] (SPARK-36847) Explicitly specify error codes when ignoring type hint errors

2021-09-24 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36847: - Summary: Explicitly specify error codes when ignoring type hint errors Key: SPARK-36847 URL: https://issues.apache.org/jira/browse/SPARK-36847 Project: Spark

[jira] [Assigned] (SPARK-36846) Inline most of type hint files under pyspark/sql/pandas folder

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36846: Assignee: (was: Apache Spark) > Inline most of type hint files under

[jira] [Assigned] (SPARK-36846) Inline most of type hint files under pyspark/sql/pandas folder

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36846: Assignee: (was: Apache Spark) > Inline most of type hint files under

[jira] [Commented] (SPARK-36846) Inline most of type hint files under pyspark/sql/pandas folder

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419991#comment-17419991 ] Apache Spark commented on SPARK-36846: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36846) Inline most of type hint files under pyspark/sql/pandas folder

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36846: Assignee: Apache Spark > Inline most of type hint files under pyspark/sql/pandas folder

[jira] [Updated] (SPARK-36846) Inline most of type hint files under pyspark/sql/pandas folder

2021-09-24 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36846: -- Description: Inline type hint files under {{pyspark/sql/pandas}} folder, except for

[jira] [Updated] (SPARK-36846) Inline most of type hint files under pyspark/sql/pandas folder

2021-09-24 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36846: -- Description: Inline type hint files under {{pyspark/sql/pandas}} folder, except for

[jira] [Created] (SPARK-36846) Inline most of type hint files under pyspark/sql/pandas folder

2021-09-24 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36846: - Summary: Inline most of type hint files under pyspark/sql/pandas folder Key: SPARK-36846 URL: https://issues.apache.org/jira/browse/SPARK-36846 Project: Spark

[jira] [Created] (SPARK-36845) Inline type hint files

2021-09-24 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36845: - Summary: Inline type hint files Key: SPARK-36845 URL: https://issues.apache.org/jira/browse/SPARK-36845 Project: Spark Issue Type: Umbrella

[jira] [Resolved] (SPARK-35174) Avoid opening watch when waitAppCompletion is false

2021-09-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35174. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34095

[jira] [Updated] (SPARK-36844) Window function "first" (unboundedFollowing) appears significantly slower than "last" (unboundedPreceding) in identical circumstances

2021-09-24 Thread Alain Bryden (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alain Bryden updated SPARK-36844: - Summary: Window function "first" (unboundedFollowing) appears significantly slower than "last"

[jira] [Assigned] (SPARK-36721) Simplify boolean equalities if one side is literal

2021-09-24 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-36721: --- Assignee: Kazuyuki Tanimura > Simplify boolean equalities if one side is literal >

[jira] [Resolved] (SPARK-36721) Simplify boolean equalities if one side is literal

2021-09-24 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36721. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34055

[jira] [Commented] (SPARK-36835) Spark 3.2.0 POMs are no longer "dependency reduced"

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419884#comment-17419884 ] Apache Spark commented on SPARK-36835: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-36844) "first" Window function is significantly slower than "last" in identical circumstances

2021-09-24 Thread Alain Bryden (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419833#comment-17419833 ] Alain Bryden commented on SPARK-36844: -- I've attached the physical plan from the initial

[jira] [Updated] (SPARK-36844) "first" Window function is significantly slower than "last" in identical circumstances

2021-09-24 Thread Alain Bryden (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alain Bryden updated SPARK-36844: - Attachment: Physical Plan 2 - workaround.png > "first" Window function is significantly slower

[jira] [Updated] (SPARK-36844) "first" Window function is significantly slower than "last" in identical circumstances

2021-09-24 Thread Alain Bryden (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alain Bryden updated SPARK-36844: - Attachment: Pysical Plan.png > "first" Window function is significantly slower than "last" in

[jira] [Created] (SPARK-36844) "first" Window function is significantly slower than "last" in identical circumstances

2021-09-24 Thread Alain Bryden (Jira)
Alain Bryden created SPARK-36844: Summary: "first" Window function is significantly slower than "last" in identical circumstances Key: SPARK-36844 URL: https://issues.apache.org/jira/browse/SPARK-36844

[jira] [Commented] (SPARK-35672) Spark fails to launch executors with very large user classpath lists on YARN

2021-09-24 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419830#comment-17419830 ] Erik Krogen commented on SPARK-35672: - Thanks [~petertoth] [~hyukjin.kwon] [~Gengliang.Wang] for

[jira] [Assigned] (SPARK-36821) Create a test to extend ColumnarBatch

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36821: Assignee: (was: Apache Spark) > Create a test to extend ColumnarBatch >

[jira] [Commented] (SPARK-36821) Create a test to extend ColumnarBatch

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419815#comment-17419815 ] Apache Spark commented on SPARK-36821: -- User 'flyrain' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36821) Create a test to extend ColumnarBatch

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36821: Assignee: Apache Spark > Create a test to extend ColumnarBatch >

[jira] [Created] (SPARK-36843) Add an iterator method to Dataset

2021-09-24 Thread Li Xian (Jira)
Li Xian created SPARK-36843: --- Summary: Add an iterator method to Dataset Key: SPARK-36843 URL: https://issues.apache.org/jira/browse/SPARK-36843 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-36842) Stop task result getter properly on spark context stopping

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419728#comment-17419728 ] Apache Spark commented on SPARK-36842: -- User 'lxian' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36842) Stop task result getter properly on spark context stopping

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36842: Assignee: (was: Apache Spark) > Stop task result getter properly on spark context

[jira] [Assigned] (SPARK-36842) Stop task result getter properly on spark context stopping

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36842: Assignee: Apache Spark > Stop task result getter properly on spark context stopping >

[jira] [Created] (SPARK-36842) Stop task result getter properly on spark context stopping

2021-09-24 Thread Li Xian (Jira)
Li Xian created SPARK-36842: --- Summary: Stop task result getter properly on spark context stopping Key: SPARK-36842 URL: https://issues.apache.org/jira/browse/SPARK-36842 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36792) Inset should handle Double.NaN and Float.NaN

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419711#comment-17419711 ] Apache Spark commented on SPARK-36792: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-36792) Inset should handle Double.NaN and Float.NaN

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419710#comment-17419710 ] Apache Spark commented on SPARK-36792: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-36831) Read/write dataframes with ANSI intervals from/to CSV files

2021-09-24 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-36831: Assignee: (was: Max Gekk) > Read/write dataframes with ANSI intervals from/to CSV files >

[jira] [Commented] (SPARK-36841) Provide ansi syntax `set catalog xxx` to change the current catalog

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419705#comment-17419705 ] Apache Spark commented on SPARK-36841: -- User 'Peng-Lei' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36841) Provide ansi syntax `set catalog xxx` to change the current catalog

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36841: Assignee: (was: Apache Spark) > Provide ansi syntax `set catalog xxx` to change the

[jira] [Commented] (SPARK-36841) Provide ansi syntax `set catalog xxx` to change the current catalog

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419703#comment-17419703 ] Apache Spark commented on SPARK-36841: -- User 'Peng-Lei' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36841) Provide ansi syntax `set catalog xxx` to change the current catalog

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36841: Assignee: Apache Spark > Provide ansi syntax `set catalog xxx` to change the current

[jira] [Updated] (SPARK-36841) Provide ansi syntax `set catalog xxx` to change the current catalog

2021-09-24 Thread PengLei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PengLei updated SPARK-36841: Description: !SET-CATALOG.PNG! (was: !截图.PNG!) > Provide ansi syntax `set catalog xxx` to change the

[jira] [Updated] (SPARK-36841) Provide ansi syntax `set catalog xxx` to change the current catalog

2021-09-24 Thread PengLei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PengLei updated SPARK-36841: Description: !截图.PNG! > Provide ansi syntax `set catalog xxx` to change the current catalog >

[jira] [Created] (SPARK-36841) Provide ansi syntax `set catalog xxx` to change the current catalog

2021-09-24 Thread PengLei (Jira)
PengLei created SPARK-36841: --- Summary: Provide ansi syntax `set catalog xxx` to change the current catalog Key: SPARK-36841 URL: https://issues.apache.org/jira/browse/SPARK-36841 Project: Spark

[jira] [Assigned] (SPARK-35174) Avoid opening watch when waitAppCompletion is false

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35174: Assignee: (was: Apache Spark) > Avoid opening watch when waitAppCompletion is false

[jira] [Resolved] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-24 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-36827. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 34092

[jira] [Assigned] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-24 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-36827: -- Assignee: Gengliang Wang > Task/Stage/Job data remain in memory leads memory leak >

[jira] [Assigned] (SPARK-35174) Avoid opening watch when waitAppCompletion is false

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35174: Assignee: Apache Spark > Avoid opening watch when waitAppCompletion is false >

[jira] [Commented] (SPARK-35174) Avoid opening watch when waitAppCompletion is false

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419681#comment-17419681 ] Apache Spark commented on SPARK-35174: -- User 'slothspot' has created a pull request for this issue:

[jira] [Commented] (SPARK-36742) Fix ps.to_datetime with plurals of keys like years, months, days

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419678#comment-17419678 ] Apache Spark commented on SPARK-36742: -- User 'dgd-contributor' has created a pull request for this

[jira] [Updated] (SPARK-36838) Keep In/InSet use same nullSafeEval and refactor

2021-09-24 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36838: -- Summary: Keep In/InSet use same nullSafeEval and refactor (was: Keep In/InSet use same nullSafeEval)

[jira] [Resolved] (SPARK-36656) CollapseProject should not collapse correlated scalar subqueries

2021-09-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36656. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33903

[jira] [Assigned] (SPARK-36656) CollapseProject should not collapse correlated scalar subqueries

2021-09-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36656: --- Assignee: Allison Wang > CollapseProject should not collapse correlated scalar subqueries

[jira] [Assigned] (SPARK-36792) Inset should handle Double.NaN and Float.NaN

2021-09-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36792: --- Assignee: angerszhu > Inset should handle Double.NaN and Float.NaN >

[jira] [Resolved] (SPARK-36792) Inset should handle Double.NaN and Float.NaN

2021-09-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36792. - Fix Version/s: 3.1.3 3.2.0 3.0.4 Resolution: Fixed

[jira] [Assigned] (SPARK-36747) Do not collapse Project with Aggregate when correlated subqueries are present in the project list

2021-09-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36747: --- Assignee: Allison Wang > Do not collapse Project with Aggregate when correlated subqueries

[jira] [Resolved] (SPARK-36747) Do not collapse Project with Aggregate when correlated subqueries are present in the project list

2021-09-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36747. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 34081

[jira] [Commented] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-24 Thread tenglei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419631#comment-17419631 ] tenglei commented on SPARK-36827: - I met the same problem in spark 2.3.x, and find a pull request from 

[jira] [Issue Comment Deleted] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-24 Thread tenglei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tenglei updated SPARK-36827: Comment: was deleted (was: I met the same problem in spark 2.3.x, and find a pull request from

[jira] [Comment Edited] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-24 Thread tenglei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419629#comment-17419629 ] tenglei edited comment on SPARK-36827 at 9/24/21, 7:50 AM: --- I met the same

[jira] [Commented] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

2021-09-24 Thread tenglei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419629#comment-17419629 ] tenglei commented on SPARK-36827: - I met the same problem in spark 2.3.x, and find a pull request from

[jira] [Commented] (SPARK-36294) Refactor fifth set of 20 query execution errors to use error classes

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419623#comment-17419623 ] Apache Spark commented on SPARK-36294: -- User 'Peng-Lei' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36294) Refactor fifth set of 20 query execution errors to use error classes

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36294: Assignee: Apache Spark > Refactor fifth set of 20 query execution errors to use error

[jira] [Commented] (SPARK-36294) Refactor fifth set of 20 query execution errors to use error classes

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419622#comment-17419622 ] Apache Spark commented on SPARK-36294: -- User 'Peng-Lei' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36294) Refactor fifth set of 20 query execution errors to use error classes

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36294: Assignee: (was: Apache Spark) > Refactor fifth set of 20 query execution errors to

[jira] [Resolved] (SPARK-36825) Read/write dataframes with ANSI intervals from/to parquet files

2021-09-24 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36825. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34057

[jira] [Commented] (SPARK-36840) Support DPP if there is no selective predicate on the filtering side

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419604#comment-17419604 ] Apache Spark commented on SPARK-36840: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36840) Support DPP if there is no selective predicate on the filtering side

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36840: Assignee: Apache Spark > Support DPP if there is no selective predicate on the filtering

[jira] [Assigned] (SPARK-36840) Support DPP if there is no selective predicate on the filtering side

2021-09-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36840: Assignee: (was: Apache Spark) > Support DPP if there is no selective predicate on

[jira] [Updated] (SPARK-36838) Keep In/InSet use same nullSafeEval

2021-09-24 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-36838: -- Summary: Keep In/InSet use same nullSafeEval (was: In/InSet should handle NULL correctly ) > Keep

[jira] [Reopened] (SPARK-36838) Keep In/InSet use same nullSafeEval

2021-09-24 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu reopened SPARK-36838: --- > Keep In/InSet use same nullSafeEval > --- > > Key:

[jira] [Created] (SPARK-36840) Support DPP if there is no selective predicate on the filtering side

2021-09-24 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-36840: --- Summary: Support DPP if there is no selective predicate on the filtering side Key: SPARK-36840 URL: https://issues.apache.org/jira/browse/SPARK-36840 Project: Spark