[GitHub] [spark] AmplabJenkins commented on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

AmplabJenkins commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697983126 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

SparkQA commented on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-697983153 **[Test build #129047 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129047/testReport)** for PR 29591 at commit [`fab00f1`](https://github.com

[GitHub] [spark] zero323 removed a comment on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

zero323 removed a comment on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-697980912 Jenkins, retest this please. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

AmplabJenkins removed a comment on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697983126 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

AmplabJenkins removed a comment on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697983134 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

[GitHub] [spark] SparkQA commented on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

SparkQA commented on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-697983565 **[Test build #129038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129038/testReport)** for PR 29828 at commit [`08cd0a7`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

SparkQA removed a comment on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-697669398 **[Test build #129038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129038/testReport)** for PR 29828 at commit [`08cd0a7`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

AmplabJenkins commented on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-697984381 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

AmplabJenkins removed a comment on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-697984381 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

SparkQA commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697985907 **[Test build #129048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129048/testReport)** for PR 29854 at commit [`d5b1095`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

SparkQA commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697989432 **[Test build #129042 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129042/testReport)** for PR 29854 at commit [`64d6dd8`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

dongjoon-hyun commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697989257 Thank you, @holdenk ! Let's see K8s IT result. This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA removed a comment on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

SparkQA removed a comment on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697883527 **[Test build #129042 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129042/testReport)** for PR 29854 at commit [`64d6dd8`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

AmplabJenkins removed a comment on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697990499 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

AmplabJenkins commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697990499 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29824: [SPARK-32954][YARN][TEST][test-hadoop2.7][test-maven] Add jakarta.servlet-api test dependency to yarn module to avoid UTs badcase

SparkQA commented on pull request #29824: URL: https://github.com/apache/spark/pull/29824#issuecomment-697992447 **[Test build #129032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129032/testReport)** for PR 29824 at commit [`250e397`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #29824: [SPARK-32954][YARN][TEST][test-hadoop2.7][test-maven] Add jakarta.servlet-api test dependency to yarn module to avoid UTs badcase

AmplabJenkins commented on pull request #29824: URL: https://github.com/apache/spark/pull/29824#issuecomment-697993651 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

SparkQA commented on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-697994187 **[Test build #129039 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129039/testReport)** for PR 29828 at commit [`078dc84`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29824: [SPARK-32954][YARN][TEST][test-hadoop2.7][test-maven] Add jakarta.servlet-api test dependency to yarn module to avoid UTs badcase

SparkQA removed a comment on pull request #29824: URL: https://github.com/apache/spark/pull/29824#issuecomment-697376701 **[Test build #129032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129032/testReport)** for PR 29824 at commit [`250e397`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29824: [SPARK-32954][YARN][TEST][test-hadoop2.7][test-maven] Add jakarta.servlet-api test dependency to yarn module to avoid UTs badca

AmplabJenkins removed a comment on pull request #29824: URL: https://github.com/apache/spark/pull/29824#issuecomment-697993651 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

SparkQA removed a comment on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-697707135 **[Test build #129039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129039/testReport)** for PR 29828 at commit [`078dc84`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

AmplabJenkins removed a comment on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-697995262 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

AmplabJenkins commented on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-697995262 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

SparkQA commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697998683 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33669/ -

[GitHub] [spark] dongjoon-hyun opened a new pull request #29856: [SPARK-32981][BUILD] Remove hive-1.2/hadoop-2.7 from Apache Spark 3.1 distribution

dongjoon-hyun opened a new pull request #29856: URL: https://github.com/apache/spark/pull/29856 ### What changes were proposed in this pull request? Apache Spark 3.0 switches its Hive execution version from 1.2 to 2.3, but it still provides the unofficial forked Hive 1.2 version from

[GitHub] [spark] dongjoon-hyun commented on pull request #29856: [SPARK-32981][BUILD] Remove hive-1.2/hadoop-2.7 from Apache Spark 3.1 distribution

dongjoon-hyun commented on pull request #29856: URL: https://github.com/apache/spark/pull/29856#issuecomment-698003841 cc @srowen , @HyukjinKwon ,@gatorsmile , @liancheng This is an automated message from the Apache Git Serv

[GitHub] [spark] SparkQA commented on pull request #29856: [SPARK-32981][BUILD] Remove hive-1.2/hadoop-2.7 from Apache Spark 3.1 distribution

SparkQA commented on pull request #29856: URL: https://github.com/apache/spark/pull/29856#issuecomment-698004283 **[Test build #129049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129049/testReport)** for PR 29856 at commit [`996cbe2`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29856: [SPARK-32981][BUILD] Remove hive-1.2/hadoop-2.7 from Apache Spark 3.1 distribution

AmplabJenkins removed a comment on pull request #29856: URL: https://github.com/apache/spark/pull/29856#issuecomment-698004577 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29856: [SPARK-32981][BUILD] Remove hive-1.2/hadoop-2.7 from Apache Spark 3.1 distribution

AmplabJenkins commented on pull request #29856: URL: https://github.com/apache/spark/pull/29856#issuecomment-698004577 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] dongjoon-hyun commented on pull request #29856: [SPARK-32981][BUILD] Remove hive-1.2/hadoop-2.7 from Apache Spark 3.1 distribution

dongjoon-hyun commented on pull request #29856: URL: https://github.com/apache/spark/pull/29856#issuecomment-698005804 Thank you, @srowen ! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun commented on pull request #29856: [SPARK-32981][BUILD] Remove hive-1.2/hadoop-2.7 from Apache Spark 3.1 distribution

dongjoon-hyun commented on pull request #29856: URL: https://github.com/apache/spark/pull/29856#issuecomment-698006440 Merged to master for Apache Spark 3.1 on December 2020. This is an automated message from the Apache Git S

[GitHub] [spark] dongjoon-hyun closed pull request #29856: [SPARK-32981][BUILD] Remove hive-1.2/hadoop-2.7 from Apache Spark 3.1 distribution

dongjoon-hyun closed pull request #29856: URL: https://github.com/apache/spark/pull/29856 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins commented on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

AmplabJenkins commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698006632 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

SparkQA commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698006612 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33669/ ---

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

AmplabJenkins removed a comment on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698006632 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] dongjoon-hyun commented on pull request #29854: [WIP][SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

dongjoon-hyun commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698007198 Yes! Finally, it passed. This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [spark] dongjoon-hyun commented on pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

dongjoon-hyun commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698008172 Merged to master because it recovers K8s IT and unblocks K8s module. This is an automated message from the

[GitHub] [spark] dongjoon-hyun closed pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

dongjoon-hyun closed pull request #29854: URL: https://github.com/apache/spark/pull/29854 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

dongjoon-hyun commented on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698008622 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] holdenk commented on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

holdenk commented on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698009613 Thanks for the clarity @dongjoon-hyun LGTM pending jenkins This is an automated message from the Apache Git Serv

[GitHub] [spark] dongjoon-hyun commented on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

dongjoon-hyun commented on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698009764 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

SparkQA commented on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698010893 **[Test build #129050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129050/testReport)** for PR 29846 at commit [`97d80fc`](https://github.com

[GitHub] [spark] holdenk commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

holdenk commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698013767 Jenkins retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA removed a comment on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

SparkQA removed a comment on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698010893 **[Test build #129050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129050/testReport)** for PR 29846 at commit [`97d80fc`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

SparkQA commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698015366 **[Test build #129051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129051/testReport)** for PR 29533 at commit [`6449efa`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

SparkQA commented on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698015143 **[Test build #129050 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129050/testReport)** for PR 29846 at commit [`97d80fc`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

AmplabJenkins removed a comment on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698015255 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

AmplabJenkins commented on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698015255 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] zero323 commented on a change in pull request #29806: [SPARK-32187][PYTHON][DOCS] Doc on Python packaging

zero323 commented on a change in pull request #29806: URL: https://github.com/apache/spark/pull/29806#discussion_r493945020 ## File path: python/docs/source/user_guide/python_packaging.rst ## @@ -0,0 +1,220 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [spark] viirya commented on a change in pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

viirya commented on a change in pull request #29846: URL: https://github.com/apache/spark/pull/29846#discussion_r493947730 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesVolumeSpec.scala ## @@ -21,7 +21,10 @@ private[spark]

[GitHub] [spark] holdenk commented on a change in pull request #29817: [SPARK-32850][CORE][K8S] Simplify the RPC message flow of decommission

holdenk commented on a change in pull request #29817: URL: https://github.com/apache/spark/pull/29817#discussion_r493940745 ## File path: core/src/main/scala/org/apache/spark/deploy/DeployMessage.scala ## @@ -61,13 +61,34 @@ private[deploy] object DeployMessages { } /**

[GitHub] [spark] SparkQA commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

SparkQA commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698020565 **[Test build #129051 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129051/testReport)** for PR 29533 at commit [`6449efa`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

SparkQA removed a comment on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698015366 **[Test build #129051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129051/testReport)** for PR 29533 at commit [`6449efa`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

AmplabJenkins removed a comment on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698020667 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

AmplabJenkins commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698020667 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] holdenk commented on a change in pull request #29788: [SPARK-32913][CORE][K8S] Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases

holdenk commented on a change in pull request #29788: URL: https://github.com/apache/spark/pull/29788#discussion_r493950073 ## File path: core/src/main/scala/org/apache/spark/scheduler/ExecutorLossReason.scala ## @@ -71,7 +71,8 @@ case class ExecutorProcessLost( * This is us

[GitHub] [spark] holdenk commented on pull request #29788: [SPARK-32913][CORE][K8S] Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases

holdenk commented on pull request #29788: URL: https://github.com/apache/spark/pull/29788#issuecomment-698021508 I think it would be good to see your proposal in code @Ngone51 because I'm not 100% sure what you mean. I would really like to see both this and the precursor tested more tho

[GitHub] [spark] SparkQA commented on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

SparkQA commented on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698021654 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33671/ -

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

dongjoon-hyun commented on a change in pull request #29846: URL: https://github.com/apache/spark/pull/29846#discussion_r493952674 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesVolumeSpec.scala ## @@ -21,7 +21,10 @@ private

[GitHub] [spark] holdenk commented on pull request #29817: [SPARK-32850][CORE][K8S] Simplify the RPC message flow of decommission

holdenk commented on pull request #29817: URL: https://github.com/apache/spark/pull/29817#issuecomment-698024268 I really like the idea of simplifying the RPC message flow, thanks for taking this on @Ngone51 and I'm sorry the code here is so brittle to these types of changes (the K8s integ

[GitHub] [spark] SparkQA commented on pull request #29855: SPARK-32915 Network-layer and shuffle RPC layer changes to support push shuffle blocks

SparkQA commented on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-698024864 **[Test build #129045 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129045/testReport)** for PR 29855 at commit [`2bdf800`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29855: SPARK-32915 Network-layer and shuffle RPC layer changes to support push shuffle blocks

SparkQA removed a comment on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-697963914 **[Test build #129045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129045/testReport)** for PR 29855 at commit [`2bdf800`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29855: SPARK-32915 Network-layer and shuffle RPC layer changes to support push shuffle blocks

AmplabJenkins removed a comment on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-698025357 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #29855: SPARK-32915 Network-layer and shuffle RPC layer changes to support push shuffle blocks

AmplabJenkins commented on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-698025357 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29855: SPARK-32915 Network-layer and shuffle RPC layer changes to support push shuffle blocks

AmplabJenkins removed a comment on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-698025365 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129

[GitHub] [spark] SparkQA commented on pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

SparkQA commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698026315 **[Test build #129043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129043/testReport)** for PR 29854 at commit [`d5b1095`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

SparkQA removed a comment on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697953562 **[Test build #129043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129043/testReport)** for PR 29854 at commit [`d5b1095`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

AmplabJenkins commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698027137 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

AmplabJenkins removed a comment on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698027137 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

AmplabJenkins commented on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698028783 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

SparkQA commented on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698028773 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33671/ ---

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

AmplabJenkins removed a comment on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698028783 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] dongjoon-hyun closed pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

dongjoon-hyun closed pull request #29846: URL: https://github.com/apache/spark/pull/29846 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

dongjoon-hyun commented on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698028969 All tests passed. Merged to master for Apache Spark 3.1.0 on December 2020. This is an automated message f

[GitHub] [spark] SparkQA commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

SparkQA commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698030149 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33672/ -

[GitHub] [spark] SparkQA commented on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

SparkQA commented on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-698032745 **[Test build #129047 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129047/testReport)** for PR 29591 at commit [`fab00f1`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

SparkQA removed a comment on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-697983153 **[Test build #129047 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129047/testReport)** for PR 29591 at commit [`fab00f1`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

AmplabJenkins commented on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-698033089 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

AmplabJenkins removed a comment on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-698033089 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] dongjoon-hyun commented on pull request #29846: [SPARK-32971][K8S] Support dynamic PVC creation/deletion for K8s executors

dongjoon-hyun commented on pull request #29846: URL: https://github.com/apache/spark/pull/29846#issuecomment-698033498 Thank you, @holdenk and @viirya . This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29591: [SPARK-32714][PYTHON] Initial pyspark-stubs port.

AmplabJenkins removed a comment on pull request #29591: URL: https://github.com/apache/spark/pull/29591#issuecomment-698033096 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129

[GitHub] [spark] huaxingao commented on a change in pull request #29837: [SPARK-32463][DOCS] SQL data type compatibility

huaxingao commented on a change in pull request #29837: URL: https://github.com/apache/spark/pull/29837#discussion_r493965798 ## File path: docs/sql-ref-datatypes.md ## @@ -314,3 +314,33 @@ SELECT COUNT(*), c2 FROM test GROUP BY c2; |3| Infinity| +-+-+

[GitHub] [spark] SparkQA commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

SparkQA commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698036245 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33672/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

AmplabJenkins commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698036258 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

AmplabJenkins removed a comment on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698036258 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

AmplabJenkins removed a comment on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698036269 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

[GitHub] [spark] SparkQA commented on pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

SparkQA commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698037064 **[Test build #129048 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129048/testReport)** for PR 29854 at commit [`d5b1095`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

SparkQA removed a comment on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-697985907 **[Test build #129048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129048/testReport)** for PR 29854 at commit [`d5b1095`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

AmplabJenkins removed a comment on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698037657 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

AmplabJenkins commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698037657 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] HyukjinKwon commented on pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

HyukjinKwon commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698038747 Thanks @holdenk and @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA removed a comment on pull request #29855: SPARK-32915 Network-layer and shuffle RPC layer changes to support push shuffle blocks

SparkQA removed a comment on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-697980048 **[Test build #129046 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129046/testReport)** for PR 29855 at commit [`3e9e9e1`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29855: SPARK-32915 Network-layer and shuffle RPC layer changes to support push shuffle blocks

SparkQA commented on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-698042528 **[Test build #129046 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129046/testReport)** for PR 29855 at commit [`3e9e9e1`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29855: SPARK-32915 Network-layer and shuffle RPC layer changes to support push shuffle blocks

AmplabJenkins removed a comment on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-698043224 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29855: SPARK-32915 Network-layer and shuffle RPC layer changes to support push shuffle blocks

AmplabJenkins commented on pull request #29855: URL: https://github.com/apache/spark/pull/29855#issuecomment-698043224 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] maropu commented on a change in pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

maropu commented on a change in pull request #29585: URL: https://github.com/apache/spark/pull/29585#discussion_r493976424 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala ## @@ -458,7 +463,10 @@ object RewriteCorrelatedScalarSub

[GitHub] [spark] github-actions[bot] closed pull request #27096: [SPARK-28148][SQL] Repartition after join is not optimized away

github-actions[bot] closed pull request #27096: URL: https://github.com/apache/spark/pull/27096 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] SparkQA commented on pull request #29585: [SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

SparkQA commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698046398 **[Test build #129052 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129052/testReport)** for PR 29585 at commit [`9ccb40e`](https://github.com

[GitHub] [spark] github-actions[bot] commented on pull request #27604: [SPARK-30849][CORE][SHUFFLE]Fix application failed due to failed to get MapStatuses broadcast block

github-actions[bot] commented on pull request #27604: URL: https://github.com/apache/spark/pull/27604#issuecomment-698046347 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] github-actions[bot] commented on pull request #28642: [SPARK-31809][SQL] Infer IsNotNull for non null intolerant child of null intolerant in join condition

github-actions[bot] commented on pull request #28642: URL: https://github.com/apache/spark/pull/28642#issuecomment-698046330 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

<    1   2   3   4   5   6   7   >