[jira] [Commented] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314202#comment-17314202 ] Apache Spark commented on SPARK-34949: -- User 'sumeetgajjar' has created a pull requ

[jira] [Assigned] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34949: Assignee: (was: Apache Spark) > Executor.reportHeartBeat reregisters blockManager eve

[jira] [Assigned] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34949: Assignee: Apache Spark > Executor.reportHeartBeat reregisters blockManager even when Exec

[jira] [Commented] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314200#comment-17314200 ] Apache Spark commented on SPARK-34949: -- User 'sumeetgajjar' has created a pull requ

[jira] [Updated] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-02 Thread Sumeet (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumeet updated SPARK-34949: --- Priority: Major (was: Minor) > Executor.reportHeartBeat reregisters blockManager even when Executor is > s

[jira] [Resolved] (SPARK-34945) Fix Javadoc for catalyst module

2021-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34945. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32038 [https://

[jira] [Assigned] (SPARK-34945) Fix Javadoc for catalyst module

2021-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34945: - Assignee: Chao Sun > Fix Javadoc for catalyst module > ---

[jira] [Created] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-02 Thread Sumeet (Jira)
Sumeet created SPARK-34949: -- Summary: Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down Key: SPARK-34949 URL: https://issues.apache.org/jira/browse/SPARK-34949 Project: Sp

[jira] [Commented] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-02 Thread Sumeet (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314194#comment-17314194 ] Sumeet commented on SPARK-34949: I am working on this. > Executor.reportHeartBeat rereg

[jira] [Commented] (SPARK-34946) Block unsupported correlated scalar subquery in Aggregate

2021-04-02 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314182#comment-17314182 ] Allison Wang commented on SPARK-34946: -- Hi [~maropu] I've updated the description.

[jira] [Updated] (SPARK-34946) Block unsupported correlated scalar subquery in Aggregate

2021-04-02 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-34946: - Description: Currently, Spark supports Aggregate to host correlated scalar subqueries, but in s

[jira] [Updated] (SPARK-34946) Block unsupported correlated scalar subquery in Aggregate

2021-04-02 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-34946: - Description: Currently, Spark supports Aggregate to host correlated scalar subqueries, but in s

[jira] [Updated] (SPARK-34946) Block unsupported correlated scalar subquery in Aggregate

2021-04-02 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-34946: - Description: Currently, Spark supports Aggregate to host correlated scalar subqueries, but in s

[jira] [Updated] (SPARK-34946) Block unsupported correlated scalar subquery in Aggregate

2021-04-02 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-34946: - Issue Type: Improvement (was: Bug) > Block unsupported correlated scalar subquery in Aggregate

[jira] [Updated] (SPARK-34946) Block unsupported correlated scalar subquery in Aggregate

2021-04-02 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-34946: - Summary: Block unsupported correlated scalar subquery in Aggregate (was: Fix correlated scalar

[jira] [Commented] (SPARK-34948) Add ownerReference to executor configmap to fix leakages

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314178#comment-17314178 ] Apache Spark commented on SPARK-34948: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-34948) Add ownerReference to executor configmap to fix leakages

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34948: Assignee: Apache Spark > Add ownerReference to executor configmap to fix leakages > -

[jira] [Commented] (SPARK-34948) Add ownerReference to executor configmap to fix leakages

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314177#comment-17314177 ] Apache Spark commented on SPARK-34948: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-34948) Add ownerReference to executor configmap to fix leakages

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34948: Assignee: (was: Apache Spark) > Add ownerReference to executor configmap to fix leaka

[jira] [Created] (SPARK-34948) Add ownerReference to executor configmap to fix leakages

2021-04-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-34948: - Summary: Add ownerReference to executor configmap to fix leakages Key: SPARK-34948 URL: https://issues.apache.org/jira/browse/SPARK-34948 Project: Spark Is

[jira] [Commented] (SPARK-34701) Remove analyzing temp view again in CreateViewCommand

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314143#comment-17314143 ] Apache Spark commented on SPARK-34701: -- User 'imback82' has created a pull request

[jira] [Commented] (SPARK-34701) Remove analyzing temp view again in CreateViewCommand

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314144#comment-17314144 ] Apache Spark commented on SPARK-34701: -- User 'imback82' has created a pull request

[jira] [Resolved] (SPARK-34946) Fix correlated scalar subquery behavior in Aggregate

2021-04-02 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-34946. -- Resolution: Invalid > Fix correlated scalar subquery behavior in Aggregate > -

[jira] [Commented] (SPARK-34946) Fix correlated scalar subquery behavior in Aggregate

2021-04-02 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314119#comment-17314119 ] Takeshi Yamamuro commented on SPARK-34946: -- Please fill the description first.

[jira] [Updated] (SPARK-34946) Fix correlated scalar subquery behavior in Aggregate

2021-04-02 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-34946: - Summary: Fix correlated scalar subquery behavior in Aggregate (was: Block correlated scalar sub

[jira] [Commented] (SPARK-27450) Timestamp cast fails when the ISO8601 string omits minutes, seconds or milliseconds

2021-04-02 Thread Will Zimmerman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314093#comment-17314093 ] Will Zimmerman commented on SPARK-27450: [~maxgekk] the proposed solution allows

[jira] [Assigned] (SPARK-34947) Streaming write to a V2 table should invalidate its associated cache

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34947: Assignee: (was: Apache Spark) > Streaming write to a V2 table should invalidate its a

[jira] [Commented] (SPARK-34947) Streaming write to a V2 table should invalidate its associated cache

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314092#comment-17314092 ] Apache Spark commented on SPARK-34947: -- User 'sunchao' has created a pull request f

[jira] [Assigned] (SPARK-34947) Streaming write to a V2 table should invalidate its associated cache

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34947: Assignee: Apache Spark > Streaming write to a V2 table should invalidate its associated c

[jira] [Assigned] (SPARK-34945) Fix Javadoc for catalyst module

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34945: Assignee: (was: Apache Spark) > Fix Javadoc for catalyst module > ---

[jira] [Assigned] (SPARK-34945) Fix Javadoc for catalyst module

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34945: Assignee: Apache Spark > Fix Javadoc for catalyst module > --

[jira] [Commented] (SPARK-34945) Fix Javadoc for catalyst module

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314089#comment-17314089 ] Apache Spark commented on SPARK-34945: -- User 'sunchao' has created a pull request f

[jira] [Created] (SPARK-34947) Streaming write to a V2 table should invalidate its associated cache

2021-04-02 Thread Chao Sun (Jira)
Chao Sun created SPARK-34947: Summary: Streaming write to a V2 table should invalidate its associated cache Key: SPARK-34947 URL: https://issues.apache.org/jira/browse/SPARK-34947 Project: Spark

[jira] [Created] (SPARK-34946) Block correlated scalar subqueries in grouping or aggregate expressions alone

2021-04-02 Thread Allison Wang (Jira)
Allison Wang created SPARK-34946: Summary: Block correlated scalar subqueries in grouping or aggregate expressions alone Key: SPARK-34946 URL: https://issues.apache.org/jira/browse/SPARK-34946 Project

[jira] [Created] (SPARK-34945) Fix Javadoc for catalyst module

2021-04-02 Thread Chao Sun (Jira)
Chao Sun created SPARK-34945: Summary: Fix Javadoc for catalyst module Key: SPARK-34945 URL: https://issues.apache.org/jira/browse/SPARK-34945 Project: Spark Issue Type: Improvement Com

[jira] [Commented] (SPARK-34944) Employ correct data type for web_returns and store_returns in TPCDS tests

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313968#comment-17313968 ] Apache Spark commented on SPARK-34944: -- User 'yaooqinn' has created a pull request

[jira] [Assigned] (SPARK-34944) Employ correct data type for web_returns and store_returns in TPCDS tests

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34944: Assignee: Apache Spark > Employ correct data type for web_returns and store_returns in TP

[jira] [Assigned] (SPARK-34944) Employ correct data type for web_returns and store_returns in TPCDS tests

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34944: Assignee: (was: Apache Spark) > Employ correct data type for web_returns and store_re

[jira] [Created] (SPARK-34944) Employ correct data type for web_returns and store_returns in TPCDS tests

2021-04-02 Thread Kent Yao (Jira)
Kent Yao created SPARK-34944: Summary: Employ correct data type for web_returns and store_returns in TPCDS tests Key: SPARK-34944 URL: https://issues.apache.org/jira/browse/SPARK-34944 Project: Spark

[jira] [Updated] (SPARK-33979) Filter predicate reorder

2021-04-02 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33979: Description: Reorder filter predicate to improve query performance: {noformat} others < In < Like

[jira] [Updated] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-02 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34943: Description: We should upgrade the flake8 installed in Jenkins because the flake8 test for pandas

[jira] [Created] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-02 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-34943: --- Summary: Upgrade flake8 to 3.8.0 or above in Jenkins Key: SPARK-34943 URL: https://issues.apache.org/jira/browse/SPARK-34943 Project: Spark Issue Type: Sub-tas

[jira] [Assigned] (SPARK-34942) Abstract Location in MapStatus to enable support for custom storage

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34942: Assignee: Apache Spark > Abstract Location in MapStatus to enable support for custom stor

[jira] [Commented] (SPARK-34942) Abstract Location in MapStatus to enable support for custom storage

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313873#comment-17313873 ] Apache Spark commented on SPARK-34942: -- User 'Ngone51' has created a pull request f

[jira] [Assigned] (SPARK-34942) Abstract Location in MapStatus to enable support for custom storage

2021-04-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34942: Assignee: (was: Apache Spark) > Abstract Location in MapStatus to enable support for

[jira] [Created] (SPARK-34942) Abstract Location in MapStatus to enable support for custom storage

2021-04-02 Thread wuyi (Jira)
wuyi created SPARK-34942: Summary: Abstract Location in MapStatus to enable support for custom storage Key: SPARK-34942 URL: https://issues.apache.org/jira/browse/SPARK-34942 Project: Spark Issue Ty

[jira] [Updated] (SPARK-34941) Enable mypy for pandas-on-Spark

2021-04-02 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34941: Summary: Enable mypy for pandas-on-Spark (was: Enable mypy test for pyspark.pandas) > Enable myp

[jira] [Created] (SPARK-34941) Enable mypy test for pyspark.pandas

2021-04-02 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-34941: --- Summary: Enable mypy test for pyspark.pandas Key: SPARK-34941 URL: https://issues.apache.org/jira/browse/SPARK-34941 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-34926) PartitionUtils.getPathFragment should handle null value

2021-04-02 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-34926: Assignee: angerszhu > PartitionUtils.getPathFragment should handle null value > -

[jira] [Resolved] (SPARK-34926) PartitionUtils.getPathFragment should handle null value

2021-04-02 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-34926. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32018 [https://github.com

[jira] [Updated] (SPARK-34934) Race condition while registering source in MetricsSystem

2021-04-02 Thread Harsh Panchal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh Panchal updated SPARK-34934: -- Description: {{MetricsSystem}} manages {{mutable.ArrayBuffer}} of metric sources. {{registerS