[jira] [Commented] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314405#comment-17314405 ] Haejoon Lee commented on SPARK-34943: - [~hyukjin.kwon] Updated. Thanks! > Upgrade flake8 to 3.8.0

[jira] [Updated] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34943: Description: In flake8 < 3.8.0, F401 error occurs for imports in if statements when

[jira] [Updated] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34943: Description: In flake8 < 3.8.0, F401 error occurs for imports in "if" statements when

[jira] [Updated] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34943: Description: In flake8 < 3.8.0, F401 error occurs for imports in *if* statements when

[jira] [Commented] (SPARK-34925) Spark shell failed to access external file to read content

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314391#comment-17314391 ] Hyukjin Kwon commented on SPARK-34925: -- As the exception indicates, it fails during S3

[jira] [Resolved] (SPARK-34925) Spark shell failed to access external file to read content

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34925. -- Resolution: Not A Problem > Spark shell failed to access external file to read content >

[jira] [Commented] (SPARK-34928) CTE Execution fails for Sql Server

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314389#comment-17314389 ] Hyukjin Kwon commented on SPARK-34928: -- [~supun.t.desilva], how does this relate to

[jira] [Updated] (SPARK-34941) Enable mypy for pandas-on-Spark

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34941: - Description: This JIRA aims to enable mypy test for {{pyspark.pandas}} package. (was: This

[jira] [Updated] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34943: - Description: We should upgrade the flake8 installed in Jenkins because the flake8 test for

[jira] [Commented] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314387#comment-17314387 ] Hyukjin Kwon commented on SPARK-34943: -- [~itholic] can you describe why we should upgrade? >

[jira] [Assigned] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34943: Assignee: Shane Knapp > Upgrade flake8 to 3.8.0 or above in Jenkins >

[jira] [Updated] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34943: - Parent: (was: SPARK-34849) Issue Type: Bug (was: Sub-task) > Upgrade flake8 to

[jira] [Assigned] (SPARK-34951) Recover Python linter (Sphinx build) in GitHub Actions

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34951: Assignee: (was: Apache Spark) > Recover Python linter (Sphinx build) in GitHub

[jira] [Commented] (SPARK-34951) Recover Python linter (Sphinx build) in GitHub Actions

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314382#comment-17314382 ] Apache Spark commented on SPARK-34951: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-34951) Recover Python linter (Sphinx build) in GitHub Actions

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34951: Assignee: Apache Spark > Recover Python linter (Sphinx build) in GitHub Actions >

[jira] [Updated] (SPARK-34951) Recover Python linter (Sphinx build) in GitHub Actions

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34951: - Affects Version/s: 3.0.2 3.1.1 > Recover Python linter (Sphinx build) in

[jira] [Created] (SPARK-34951) Recover Python linter (Sphinx build) in GitHub Actions

2021-04-03 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-34951: Summary: Recover Python linter (Sphinx build) in GitHub Actions Key: SPARK-34951 URL: https://issues.apache.org/jira/browse/SPARK-34951 Project: Spark Issue

[jira] [Assigned] (SPARK-34939) Throw fetch failure exception when unable to deserialize broadcasted map statuses

2021-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34939: - Assignee: L. C. Hsieh > Throw fetch failure exception when unable to deserialize

[jira] [Resolved] (SPARK-34939) Throw fetch failure exception when unable to deserialize broadcasted map statuses

2021-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34939. --- Fix Version/s: 3.0.3 3.1.2 2.4.8

[jira] [Assigned] (SPARK-34950) Update benchmark results to the ones created by GitHub Actions machines

2021-04-03 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-34950: Assignee: Hyukjin Kwon > Update benchmark results to the ones created by GitHub Actions machines

[jira] [Resolved] (SPARK-34950) Update benchmark results to the ones created by GitHub Actions machines

2021-04-03 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-34950. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32044

[jira] [Commented] (SPARK-34939) Throw fetch failure exception when unable to deserialize broadcasted map statuses

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314334#comment-17314334 ] Apache Spark commented on SPARK-34939: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-34939) Throw fetch failure exception when unable to deserialize broadcasted map statuses

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314333#comment-17314333 ] Apache Spark commented on SPARK-34939: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-23977) Add commit protocol binding to Hadoop 3.1 PathOutputCommitter mechanism

2021-04-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314296#comment-17314296 ] Steve Loughran commented on SPARK-23977: the spark settings don't make it down from sql; you can

[jira] [Commented] (SPARK-34950) Update benchmark results to the ones created by GitHub Actions machines

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314253#comment-17314253 ] Apache Spark commented on SPARK-34950: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-34950) Update benchmark results to the ones created by GitHub Actions machines

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34950: Assignee: Apache Spark > Update benchmark results to the ones created by GitHub Actions

[jira] [Assigned] (SPARK-34950) Update benchmark results to the ones created by GitHub Actions machines

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34950: Assignee: (was: Apache Spark) > Update benchmark results to the ones created by

[jira] [Updated] (SPARK-34950) Update benchmark results to the ones created by GitHub Actions machines

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34950: - Issue Type: Improvement (was: Bug) > Update benchmark results to the ones created by GitHub

[jira] [Created] (SPARK-34950) Update benchmark results to the ones created by GitHub Actions machines

2021-04-03 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-34950: Summary: Update benchmark results to the ones created by GitHub Actions machines Key: SPARK-34950 URL: https://issues.apache.org/jira/browse/SPARK-34950 Project:

[jira] [Assigned] (SPARK-34821) Set up a workflow for developers to run benchmark in their fork

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34821: Assignee: Hyukjin Kwon > Set up a workflow for developers to run benchmark in their fork

[jira] [Resolved] (SPARK-34821) Set up a workflow for developers to run benchmark in their fork

2021-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34821. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32015

[jira] [Updated] (SPARK-34934) Race condition while registering source in MetricsSystem

2021-04-03 Thread Harsh Panchal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh Panchal updated SPARK-34934: -- Description: {{MetricsSystem}} manages {{mutable.ArrayBuffer}} of metric sources.

[jira] [Resolved] (SPARK-34948) Add ownerReference to executor configmap to fix leakages

2021-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34948. --- Fix Version/s: 3.1.2 3.2.0 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-34948) Add ownerReference to executor configmap to fix leakages

2021-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34948: - Assignee: Dongjoon Hyun > Add ownerReference to executor configmap to fix leakages >

[jira] [Commented] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314202#comment-17314202 ] Apache Spark commented on SPARK-34949: -- User 'sumeetgajjar' has created a pull request for this

[jira] [Assigned] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34949: Assignee: (was: Apache Spark) > Executor.reportHeartBeat reregisters blockManager

[jira] [Assigned] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34949: Assignee: Apache Spark > Executor.reportHeartBeat reregisters blockManager even when

[jira] [Commented] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314200#comment-17314200 ] Apache Spark commented on SPARK-34949: -- User 'sumeetgajjar' has created a pull request for this

[jira] [Updated] (SPARK-34949) Executor.reportHeartBeat reregisters blockManager even when Executor is shutting down

2021-04-03 Thread Sumeet (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumeet updated SPARK-34949: --- Priority: Major (was: Minor) > Executor.reportHeartBeat reregisters blockManager even when Executor is >

[jira] [Resolved] (SPARK-34945) Fix Javadoc for catalyst module

2021-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34945. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32038

[jira] [Assigned] (SPARK-34945) Fix Javadoc for catalyst module

2021-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34945: - Assignee: Chao Sun > Fix Javadoc for catalyst module > ---