[jira] [Updated] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-02 Thread Yuto Akutsu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Akutsu updated SPARK-36660: Priority: Minor (was: Major) > Cotangent is not supported by Dataframe >

[jira] [Created] (SPARK-36661) Support TimestampNTZ in PyArrow

2021-09-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-36661: Summary: Support TimestampNTZ in PyArrow Key: SPARK-36661 URL: https://issues.apache.org/jira/browse/SPARK-36661 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36661) Support TimestampNTZ in Py4J

2021-09-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-36661: - Summary: Support TimestampNTZ in Py4J (was: Support TimestampNTZ in PyArrow) > Support

[jira] [Updated] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-02 Thread Yuto Akutsu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Akutsu updated SPARK-36660: Description: Cotangent is implemented in mathExpressions but cannot be called by dataframe

[jira] [Updated] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-02 Thread Yuto Akutsu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Akutsu updated SPARK-36660: Description: Cotangent is implemented in mathExpressions but cannot be called by dataframe

[jira] [Updated] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-02 Thread Yuto Akutsu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Akutsu updated SPARK-36660: Description: Cotangent is implemented in mathExpressions but cannot be called by dataframe

[jira] [Updated] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-02 Thread Yuto Akutsu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Akutsu updated SPARK-36660: Description: Cotangent is implemented in mathExpressions but cannot be called by dataframe

[jira] [Assigned] (SPARK-36658) Expose executionId to QueryExecutionListener

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36658: Assignee: Apache Spark > Expose executionId to QueryExecutionListener >

[jira] [Assigned] (SPARK-36658) Expose executionId to QueryExecutionListener

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36658: Assignee: (was: Apache Spark) > Expose executionId to QueryExecutionListener >

[jira] [Commented] (SPARK-36658) Expose executionId to QueryExecutionListener

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409254#comment-17409254 ] Apache Spark commented on SPARK-36658: -- User 'ivoson' has created a pull request for this issue:

[jira] [Updated] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-02 Thread Yuto Akutsu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Akutsu updated SPARK-36660: Description: Cotangent is implemented in ^mathExpressions^ but cannot be called by dataframe

[jira] [Updated] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-02 Thread Yuto Akutsu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Akutsu updated SPARK-36660: Description: Cotangent is implemented in mathExpressions but cannot be called by dataframe

[jira] [Created] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-02 Thread Yuto Akutsu (Jira)
Yuto Akutsu created SPARK-36660: --- Summary: Cotangent is not supported by Dataframe Key: SPARK-36660 URL: https://issues.apache.org/jira/browse/SPARK-36660 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409240#comment-17409240 ] Apache Spark commented on SPARK-36659: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36659: Assignee: Apache Spark > Promote spark.sql.execution.topKSortFallbackThreshold to

[jira] [Assigned] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36659: Assignee: (was: Apache Spark) > Promote

[jira] [Commented] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409239#comment-17409239 ] Apache Spark commented on SPARK-36659: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Resolved] (SPARK-36652) AQE dynamic join selection should not apply to non-equi join

2021-09-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36652. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33899

[jira] [Assigned] (SPARK-36652) AQE dynamic join selection should not apply to non-equi join

2021-09-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36652: - Assignee: Cheng Su > AQE dynamic join selection should not apply to non-equi join >

[jira] [Updated] (SPARK-36633) DivideDTInterval should throw the same exception when divide by zero.

2021-09-02 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-36633: --- Summary: DivideDTInterval should throw the same exception when divide by zero. (was:

[jira] [Created] (SPARK-36659) Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config

2021-09-02 Thread Kent Yao (Jira)
Kent Yao created SPARK-36659: Summary: Promote spark.sql.execution.topKSortFallbackThreshold to user-faced config Key: SPARK-36659 URL: https://issues.apache.org/jira/browse/SPARK-36659 Project: Spark

[jira] [Resolved] (SPARK-36650) ApplicationMaster shutdown hook should catch timeout exception

2021-09-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36650. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33897

[jira] [Assigned] (SPARK-36650) ApplicationMaster shutdown hook should catch timeout exception

2021-09-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36650: Assignee: angerszhu > ApplicationMaster shutdown hook should catch timeout exception >

[jira] [Comment Edited] (SPARK-36658) Expose executionId to QueryExecutionListener

2021-09-02 Thread huangtengfei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409210#comment-17409210 ] huangtengfei edited comment on SPARK-36658 at 9/3/21, 2:36 AM: --- cc

[jira] [Commented] (SPARK-36658) Expose executionId to QueryExecutionListener

2021-09-02 Thread huangtengfei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409228#comment-17409228 ] huangtengfei commented on SPARK-36658: -- Will create a RP for this. > Expose executionId to

[jira] [Commented] (SPARK-36635) spark-sql do NOT support that select name expression as string type now

2021-09-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409225#comment-17409225 ] Hyukjin Kwon commented on SPARK-36635: -- can you justify why it should work? is it ANSI standard or

[jira] [Commented] (SPARK-36635) spark-sql do NOT support that select name expression as string type now

2021-09-02 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409222#comment-17409222 ] weixiuli commented on SPARK-36635: -- It didn't work before either. > spark-sql do NOT support that

[jira] [Resolved] (SPARK-36351) Separate partition filters and data filters in PushDownUtils

2021-09-02 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36351. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33650

[jira] [Assigned] (SPARK-36351) Separate partition filters and data filters in PushDownUtils

2021-09-02 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-36351: --- Assignee: Huaxin Gao > Separate partition filters and data filters in PushDownUtils >

[jira] [Updated] (SPARK-36658) Expose executionId to QueryExecutionListener

2021-09-02 Thread huangtengfei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangtengfei updated SPARK-36658: - Description: Now in

[jira] [Commented] (SPARK-36658) Expose executionId to QueryExecutionListener

2021-09-02 Thread huangtengfei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409210#comment-17409210 ] huangtengfei commented on SPARK-36658: -- cc [~cloud_fan] could you share any thoughts about this? >

[jira] [Created] (SPARK-36658) Expose executionId to QueryExecutionListener

2021-09-02 Thread huangtengfei (Jira)
huangtengfei created SPARK-36658: Summary: Expose executionId to QueryExecutionListener Key: SPARK-36658 URL: https://issues.apache.org/jira/browse/SPARK-36658 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36657) Update comment in `gen-sql-config-docs.py`

2021-09-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36657: -- Affects Version/s: 3.2.0 > Update comment in `gen-sql-config-docs.py` >

[jira] [Assigned] (SPARK-36657) Update comment in `gen-sql-config-docs.py`

2021-09-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36657: - Assignee: William Hyun > Update comment in `gen-sql-config-docs.py` >

[jira] [Resolved] (SPARK-36657) Update comment in `gen-sql-config-docs.py`

2021-09-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36657. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33902

[jira] [Commented] (SPARK-36656) CollapseProject should not collapse correlated scalar subqueries

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409174#comment-17409174 ] Apache Spark commented on SPARK-36656: -- User 'allisonwang-db' has created a pull request for this

[jira] [Commented] (SPARK-36657) Update comment in `gen-sql-config-docs.py`

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409171#comment-17409171 ] Apache Spark commented on SPARK-36657: -- User 'williamhyun' has created a pull request for this

[jira] [Commented] (SPARK-36656) CollapseProject should not collapse correlated scalar subqueries

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409173#comment-17409173 ] Apache Spark commented on SPARK-36656: -- User 'allisonwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-36656) CollapseProject should not collapse correlated scalar subqueries

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36656: Assignee: Apache Spark > CollapseProject should not collapse correlated scalar

[jira] [Assigned] (SPARK-36656) CollapseProject should not collapse correlated scalar subqueries

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36656: Assignee: (was: Apache Spark) > CollapseProject should not collapse correlated

[jira] [Assigned] (SPARK-36657) Update comment in `gen-sql-config-docs.py`

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36657: Assignee: (was: Apache Spark) > Update comment in `gen-sql-config-docs.py` >

[jira] [Commented] (SPARK-36657) Update comment in `gen-sql-config-docs.py`

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409170#comment-17409170 ] Apache Spark commented on SPARK-36657: -- User 'williamhyun' has created a pull request for this

[jira] [Assigned] (SPARK-36657) Update comment in `gen-sql-config-docs.py`

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36657: Assignee: Apache Spark > Update comment in `gen-sql-config-docs.py` >

[jira] [Created] (SPARK-36657) Update comment in `gen-sql-config-docs.py`

2021-09-02 Thread William Hyun (Jira)
William Hyun created SPARK-36657: Summary: Update comment in `gen-sql-config-docs.py` Key: SPARK-36657 URL: https://issues.apache.org/jira/browse/SPARK-36657 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-36656) CollapseProject should not collapse correlated scalar subqueries

2021-09-02 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-36656: - Description: Currently, the optimizer rule `CollapseProject` inlines expressions generated

[jira] [Created] (SPARK-36656) CollapseProject should not collapse correlated scalar subqueries

2021-09-02 Thread Allison Wang (Jira)
Allison Wang created SPARK-36656: Summary: CollapseProject should not collapse correlated scalar subqueries Key: SPARK-36656 URL: https://issues.apache.org/jira/browse/SPARK-36656 Project: Spark

[jira] [Assigned] (SPARK-36655) Add `versionadded` for API added in Spark 3.3.0

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36655: Assignee: Apache Spark > Add `versionadded` for API added in Spark 3.3.0 >

[jira] [Assigned] (SPARK-36655) Add `versionadded` for API added in Spark 3.3.0

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36655: Assignee: (was: Apache Spark) > Add `versionadded` for API added in Spark 3.3.0 >

[jira] [Commented] (SPARK-36655) Add `versionadded` for API added in Spark 3.3.0

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409134#comment-17409134 ] Apache Spark commented on SPARK-36655: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-36654) Drop type ignores from numpy imports

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36654: Assignee: (was: Apache Spark) > Drop type ignores from numpy imports >

[jira] [Assigned] (SPARK-36654) Drop type ignores from numpy imports

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36654: Assignee: Apache Spark > Drop type ignores from numpy imports >

[jira] [Commented] (SPARK-36654) Drop type ignores from numpy imports

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409133#comment-17409133 ] Apache Spark commented on SPARK-36654: -- User 'zero323' has created a pull request for this issue:

[jira] [Created] (SPARK-36655) Add `versionadded` for API added in Spark 3.3.0

2021-09-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36655: Summary: Add `versionadded` for API added in Spark 3.3.0 Key: SPARK-36655 URL: https://issues.apache.org/jira/browse/SPARK-36655 Project: Spark Issue Type:

[jira] [Created] (SPARK-36654) Drop type ignores from numpy imports

2021-09-02 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-36654: -- Summary: Drop type ignores from numpy imports Key: SPARK-36654 URL: https://issues.apache.org/jira/browse/SPARK-36654 Project: Spark Issue Type:

[jira] [Created] (SPARK-36653) Implement Series.__xor__

2021-09-02 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36653: - Summary: Implement Series.__xor__ Key: SPARK-36653 URL: https://issues.apache.org/jira/browse/SPARK-36653 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36652) AQE dynamic join selection should not apply to non-equi join

2021-09-02 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-36652: - Affects Version/s: (was: 3.2.0) > AQE dynamic join selection should not apply to non-equi join >

[jira] [Commented] (SPARK-36443) Demote BroadcastJoin causes performance regression and increases OOM risks

2021-09-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408958#comment-17408958 ] Dongjoon Hyun commented on SPARK-36443: --- Thank you for the details. > Demote BroadcastJoin causes

[jira] [Updated] (SPARK-36652) AQE dynamic join selection should not apply to non-equi join

2021-09-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36652: -- Parent: SPARK-33828 Issue Type: Sub-task (was: Bug) > AQE dynamic join selection

[jira] [Assigned] (SPARK-36617) Inconsistencies in approxQuantile annotations

2021-09-02 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-36617: -- Assignee: Cary Lee > Inconsistencies in approxQuantile annotations >

[jira] [Assigned] (SPARK-36652) AQE dynamic join selection should not apply to non-equi join

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36652: Assignee: (was: Apache Spark) > AQE dynamic join selection should not apply to

[jira] [Assigned] (SPARK-36652) AQE dynamic join selection should not apply to non-equi join

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36652: Assignee: Apache Spark > AQE dynamic join selection should not apply to non-equi join >

[jira] [Commented] (SPARK-36652) AQE dynamic join selection should not apply to non-equi join

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408914#comment-17408914 ] Apache Spark commented on SPARK-36652: -- User 'c21' has created a pull request for this issue:

[jira] [Updated] (SPARK-36630) Add the option to use physical statistics to avoid large tables being broadcast

2021-09-02 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-36630: --- Parent: (was: SPARK-33828) Issue Type: Question (was: Sub-task) > Add the option to

[jira] [Issue Comment Deleted] (SPARK-36630) Add the option to use physical statistics to avoid large tables being broadcast

2021-09-02 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-36630: --- Comment: was deleted (was: close it) > Add the option to use physical statistics to avoid large

[jira] [Closed] (SPARK-36630) Add the option to use physical statistics to avoid large tables being broadcast

2021-09-02 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 closed SPARK-36630. -- close it > Add the option to use physical statistics to avoid large tables being > broadcast >

[jira] [Resolved] (SPARK-36630) Add the option to use physical statistics to avoid large tables being broadcast

2021-09-02 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 resolved SPARK-36630. Resolution: Fixed > Add the option to use physical statistics to avoid large tables being >

[jira] [Commented] (SPARK-36630) Add the option to use physical statistics to avoid large tables being broadcast

2021-09-02 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408901#comment-17408901 ] gaoyajun02 commented on SPARK-36630: Found my issue is similar to

[jira] [Created] (SPARK-36652) AQE dynamic join selection should not apply to non-equi join

2021-09-02 Thread Cheng Su (Jira)
Cheng Su created SPARK-36652: Summary: AQE dynamic join selection should not apply to non-equi join Key: SPARK-36652 URL: https://issues.apache.org/jira/browse/SPARK-36652 Project: Spark Issue

[jira] [Assigned] (SPARK-36637) Bad error message when using non-existing named window

2021-09-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36637: --- Assignee: angerszhu > Bad error message when using non-existing named window >

[jira] [Resolved] (SPARK-36637) Bad error message when using non-existing named window

2021-09-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36637. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33892

[jira] [Resolved] (SPARK-36617) Inconsistencies in approxQuantile annotations

2021-09-02 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-36617. Fix Version/s: 3.1.3 3.2.0 Resolution: Resolved Issue

[jira] [Commented] (SPARK-36622) spark.history.kerberos.principal doesn't take value _HOST

2021-09-02 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408813#comment-17408813 ] Thomas Graves commented on SPARK-36622: --- Supported _HOST for SHS likely makes sense since its a

[jira] [Comment Edited] (SPARK-35623) Volcano resource manager for Spark on Kubernetes

2021-09-02 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408750#comment-17408750 ] Senthil Kumar edited comment on SPARK-35623 at 9/2/21, 11:56 AM: -

[jira] [Commented] (SPARK-35623) Volcano resource manager for Spark on Kubernetes

2021-09-02 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408750#comment-17408750 ] Senthil Kumar commented on SPARK-35623: --- [~dipanjanK] Include me too pls > Volcano resource

[jira] [Updated] (SPARK-36632) DivideYMInterval should throw the same exception when divide by zero.

2021-09-02 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-36632: --- Summary: DivideYMInterval should throw the same exception when divide by zero. (was:

[jira] [Updated] (SPARK-36627) Tasks with Java proxy objects fail to deserialize

2021-09-02 Thread Samuel Souza (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Souza updated SPARK-36627: - Description: In JavaSerializer.JavaDeserializationStream we override resolveClass of

[jira] [Updated] (SPARK-36627) Tasks with Java proxy objects fail to deserialize

2021-09-02 Thread Samuel Souza (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Souza updated SPARK-36627: - Summary: Tasks with Java proxy objects fail to deserialize (was: Tasks with Java proxy objects

[jira] [Assigned] (SPARK-36644) Push down boolean column filter

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36644: Assignee: Apache Spark > Push down boolean column filter >

[jira] [Commented] (SPARK-36644) Push down boolean column filter

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408658#comment-17408658 ] Apache Spark commented on SPARK-36644: -- User 'kazuyukitanimura' has created a pull request for this

[jira] [Assigned] (SPARK-36644) Push down boolean column filter

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36644: Assignee: (was: Apache Spark) > Push down boolean column filter >

[jira] [Updated] (SPARK-36630) Add the option to use physical statistics to avoid large tables being broadcast

2021-09-02 Thread gaoyajun02 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaoyajun02 updated SPARK-36630: --- Parent: SPARK-33828 Issue Type: Sub-task (was: Improvement) > Add the option to use

[jira] [Updated] (SPARK-36651) NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110)

2021-09-02 Thread Serge Shikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Shikov updated SPARK-36651: - Description: This is still reproducible in Spark 2.4.5. Original issue SPARK-25080 conains

[jira] [Updated] (SPARK-36651) NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110)

2021-09-02 Thread Serge Shikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Shikov updated SPARK-36651: - Issue Type: Bug (was: Improvement) > NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110) >

[jira] [Updated] (SPARK-36651) NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110)

2021-09-02 Thread Serge Shikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Shikov updated SPARK-36651: - Priority: Major (was: Minor) > NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110) >

[jira] [Updated] (SPARK-36651) NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110)

2021-09-02 Thread Serge Shikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Shikov updated SPARK-36651: - Description: This is still reproducible in Spark 2.4.5. Original issue conains steps to

[jira] [Updated] (SPARK-36651) NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110)

2021-09-02 Thread Serge Shikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Shikov updated SPARK-36651: - Affects Version/s: (was: 2.3.1) 2.4.5 > NPE in

[jira] [Created] (SPARK-36651) NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110)

2021-09-02 Thread Serge Shikov (Jira)
Serge Shikov created SPARK-36651: Summary: NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110) Key: SPARK-36651 URL: https://issues.apache.org/jira/browse/SPARK-36651 Project: Spark Issue

[jira] [Commented] (SPARK-36650) ApplicationMaster shutdown hook should catch timeout exception

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408601#comment-17408601 ] Apache Spark commented on SPARK-36650: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-36650) ApplicationMaster shutdown hook should catch timeout exception

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36650: Assignee: (was: Apache Spark) > ApplicationMaster shutdown hook should catch timeout

[jira] [Assigned] (SPARK-36650) ApplicationMaster shutdown hook should catch timeout exception

2021-09-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36650: Assignee: Apache Spark > ApplicationMaster shutdown hook should catch timeout exception

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-09-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description:  When driver broadcast object, it will send the

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-09-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description:  When driver broadcast object, it will send the

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-09-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description:  When driver broadcast object, it will send the

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-09-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description:  When driver broadcast object, it will send the

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-09-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description:  When driver broadcast object, it will send the [SparkListenerBlockUpdated| #L228] event. 

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-09-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description:  When driver broadcast object, it will send the [SparkListenerBlockUpdated|#L228] event. 

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-09-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description:  When driver broadcast object, it will send the [SparkListenerBlockUpdated|#L228] event. 

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-09-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description:  When driver broadcast object, it will send the [SparkListenerBlockUpdated|#L228] event. 

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-09-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description:  When driver broadcast object, it will send the [SparkListenerBlockUpdated|#L228]] event. 

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-09-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description:  When driver broadcast object, it will send the

  1   2   >