[GitHub] [spark] SparkQA commented on pull request #30751: [SPARK-32617][K8S][TESTS] Configure kubernetes client based on kubeconfig settings in kubernetes integration tests

2020-12-12 Thread GitBox
SparkQA commented on pull request #30751: URL: https://github.com/apache/spark/pull/30751#issuecomment-743970560 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37320/ -

[GitHub] [spark] SparkQA commented on pull request #30751: [SPARK-32617][K8S][TESTS] Configure kubernetes client based on kubeconfig settings in kubernetes integration tests

2020-12-12 Thread GitBox
SparkQA commented on pull request #30751: URL: https://github.com/apache/spark/pull/30751#issuecomment-743970554 **[Test build #132717 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132717/testReport)** for PR 30751 at commit [`8256b5d`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #30749: [MINOR][UI] Correct JobPage's skipped/pending tableHeaderId

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30749: URL: https://github.com/apache/spark/pull/30749#issuecomment-743970297 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132715/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30749: [MINOR][UI] Correct JobPage's skipped/pending tableHeaderId

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30749: URL: https://github.com/apache/spark/pull/30749#issuecomment-743970297 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132715/ -

[GitHub] [spark] SparkQA removed a comment on pull request #30749: [MINOR][UI] Correct JobPage's skipped/pending tableHeaderId

2020-12-12 Thread GitBox
SparkQA removed a comment on pull request #30749: URL: https://github.com/apache/spark/pull/30749#issuecomment-743957885 **[Test build #132715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132715/testReport)** for PR 30749 at commit [`88d9b3c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30749: [MINOR][UI] Correct JobPage's skipped/pending tableHeaderId

2020-12-12 Thread GitBox
SparkQA commented on pull request #30749: URL: https://github.com/apache/spark/pull/30749#issuecomment-743969429 **[Test build #132715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132715/testReport)** for PR 30749 at commit [`88d9b3c`](https://github.co

[GitHub] [spark] wangyum commented on pull request #30517: [DO-NOT-MERGE] Test compatibility for Parquet 1.11.1, Avro 1.10.1 and Hive 2.3.8

2020-12-12 Thread GitBox
wangyum commented on pull request #30517: URL: https://github.com/apache/spark/pull/30517#issuecomment-743969298 Parquet column index benchmark code and result: ```scala /* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. S

[GitHub] [spark] SparkQA commented on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
SparkQA commented on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743968525 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37319/ -

[GitHub] [spark] attilapiros commented on a change in pull request #30751: [SPARK-32617][K8S][TESTS] Configure kubernetes client based on kubeconfig settings in kubernetes integration tests

2020-12-12 Thread GitBox
attilapiros commented on a change in pull request #30751: URL: https://github.com/apache/spark/pull/30751#discussion_r541865925 ## File path: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/backend/minikube/Minikube.scala #

[GitHub] [spark] cxzl25 commented on pull request #30725: [SPARK-33753][CORE] Reduce the memory footprint and gc of the cache (hadoopJobMetadata)

2020-12-12 Thread GitBox
cxzl25 commented on pull request #30725: URL: https://github.com/apache/spark/pull/30725#issuecomment-743967591 > Do we really need a separate copy of the JobConf cached for each partition ID? Needs. HadoopRDD#getJobConf has such a comment. https://github.com/apache/spark/blo

[GitHub] [spark] attilapiros commented on a change in pull request #30751: [SPARK-32617][K8S][TESTS] Configure kubernetes client based on kubeconfig settings in kubernetes integration tests

2020-12-12 Thread GitBox
attilapiros commented on a change in pull request #30751: URL: https://github.com/apache/spark/pull/30751#discussion_r541865925 ## File path: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/backend/minikube/Minikube.scala #

[GitHub] [spark] attilapiros opened a new pull request #30751: [SPARK-32617][K8S][TESTS] Configure kubernetes client based on kubeconfig settings in kubernetes integration tests

2020-12-12 Thread GitBox
attilapiros opened a new pull request #30751: URL: https://github.com/apache/spark/pull/30751 ### What changes were proposed in this pull request? From [minikube version v1.1.0](https://github.com/kubernetes/minikube/blob/v1.1.0/CHANGELOG.md) kubectl is available as a command. So th

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30749: [MINOR][UI] Correct JobPage's skipped/pending tableHeaderId

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30749: URL: https://github.com/apache/spark/pull/30749#issuecomment-743963804 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37318/

[GitHub] [spark] SparkQA commented on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
SparkQA commented on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743963855 **[Test build #132716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132716/testReport)** for PR 30643 at commit [`afbfdba`](https://github.com

[GitHub] [spark] rwpenney commented on a change in pull request #30745: [SPARK-33678][SQL] Product aggregation function

2020-12-12 Thread GitBox
rwpenney commented on a change in pull request #30745: URL: https://github.com/apache/spark/pull/30745#discussion_r541862096 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -706,6 +706,28 @@ object functions { } } + /** + * Aggregat

[GitHub] [spark] SparkQA commented on pull request #30749: [MINOR][UI] Correct JobPage's skipped/pending tableHeaderId

2020-12-12 Thread GitBox
SparkQA commented on pull request #30749: URL: https://github.com/apache/spark/pull/30749#issuecomment-743963800 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37318/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #30749: [MINOR][UI] Correct JobPage's skipped/pending tableHeaderId

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30749: URL: https://github.com/apache/spark/pull/30749#issuecomment-743963804 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37318/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743963528 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132714/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743963528 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132714/ -

[GitHub] [spark] yaooqinn commented on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
yaooqinn commented on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743963522 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA removed a comment on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
SparkQA removed a comment on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743950164 **[Test build #132714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132714/testReport)** for PR 30643 at commit [`afbfdba`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
SparkQA commented on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743963329 **[Test build #132714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132714/testReport)** for PR 30643 at commit [`afbfdba`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30749: [MINOR][UI] Correct JobPage's skipped/pending tableHeaderId

2020-12-12 Thread GitBox
SparkQA commented on pull request #30749: URL: https://github.com/apache/spark/pull/30749#issuecomment-743961442 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37318/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743958337 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37317/

[GitHub] [spark] AmplabJenkins commented on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743958337 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37317/ -

[GitHub] [spark] SparkQA commented on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
SparkQA commented on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743958333 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37317/ ---

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30749: [MINOR][UI] Correct JobPage's skipped/pending tableHeaderId

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30749: URL: https://github.com/apache/spark/pull/30749#issuecomment-743922136 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #30749: [MINOR][UI] Correct JobPage's skipped/pending tableHeaderId

2020-12-12 Thread GitBox
SparkQA commented on pull request #30749: URL: https://github.com/apache/spark/pull/30749#issuecomment-743957885 **[Test build #132715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132715/testReport)** for PR 30749 at commit [`88d9b3c`](https://github.com

[GitHub] [spark] weixiuli commented on pull request #30716: [SPARK-33747][CORE] Avoid calling unregisterMapOutput when the map stage is being rerunning.

2020-12-12 Thread GitBox
weixiuli commented on pull request #30716: URL: https://github.com/apache/spark/pull/30716#issuecomment-743956322 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] sarutak commented on pull request #30749: [MINOR][UI] Correct JobPage's skipped/pending tableHeaderId

2020-12-12 Thread GitBox
sarutak commented on pull request #30749: URL: https://github.com/apache/spark/pull/30749#issuecomment-743956270 ok to test. This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [spark] SparkQA commented on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
SparkQA commented on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743955859 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37317/ -

[GitHub] [spark] SparkQA commented on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
SparkQA commented on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743950164 **[Test build #132714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132714/testReport)** for PR 30643 at commit [`afbfdba`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30741: [SPARK-33764][SS] Make state store maintenance interval as SQL config

2020-12-12 Thread GitBox
SparkQA commented on pull request #30741: URL: https://github.com/apache/spark/pull/30741#issuecomment-743950072 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37316/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #30741: [SPARK-33764][SS] Make state store maintenance interval as SQL config

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30741: URL: https://github.com/apache/spark/pull/30741#issuecomment-743950073 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37316/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30741: [SPARK-33764][SS] Make state store maintenance interval as SQL config

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30741: URL: https://github.com/apache/spark/pull/30741#issuecomment-743950073 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37316/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743949767 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37314/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743949768 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30741: [SPARK-33764][SS] Make state store maintenance interval as SQL config

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30741: URL: https://github.com/apache/spark/pull/30741#issuecomment-743949771 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132713/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30741: [SPARK-33764][SS] Make state store maintenance interval as SQL config

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30741: URL: https://github.com/apache/spark/pull/30741#issuecomment-743949771 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132713/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743949767 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37314/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743949769 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #30741: [SPARK-33764][SS] Make state store maintenance interval as SQL config

2020-12-12 Thread GitBox
SparkQA removed a comment on pull request #30741: URL: https://github.com/apache/spark/pull/30741#issuecomment-743944705 **[Test build #132713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132713/testReport)** for PR 30741 at commit [`6a0574c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30741: [SPARK-33764][SS] Make state store maintenance interval as SQL config

2020-12-12 Thread GitBox
SparkQA commented on pull request #30741: URL: https://github.com/apache/spark/pull/30741#issuecomment-743948112 **[Test build #132713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132713/testReport)** for PR 30741 at commit [`6a0574c`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30741: [SPARK-33764][SS] Make state store maintenance interval as SQL config

2020-12-12 Thread GitBox
SparkQA commented on pull request #30741: URL: https://github.com/apache/spark/pull/30741#issuecomment-743948037 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37316/ -

[GitHub] [spark] yaooqinn commented on pull request #30643: [SPARK-33705][SQL][TEST][test-hadoop2.7] Fix HiveThriftHttpServerSuite flakiness

2020-12-12 Thread GitBox
yaooqinn commented on pull request #30643: URL: https://github.com/apache/spark/pull/30643#issuecomment-743947352 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA removed a comment on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA removed a comment on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743934727 **[Test build #132708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132708/testReport)** for PR 30735 at commit [`d3e52f4`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743947169 **[Test build #132708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132708/testReport)** for PR 30735 at commit [`d3e52f4`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
SparkQA commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743947122 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37314/ ---

[GitHub] [spark] SparkQA removed a comment on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA removed a comment on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743935336 **[Test build #132709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132709/testReport)** for PR 30735 at commit [`cc4ff8f`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743947062 **[Test build #132709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132709/testReport)** for PR 30735 at commit [`cc4ff8f`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA removed a comment on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743935844 **[Test build #132710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132710/testReport)** for PR 30735 at commit [`5f13c30`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743946596 **[Test build #132710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132710/testReport)** for PR 30735 at commit [`5f13c30`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743945189 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37315/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743945189 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37315/

[GitHub] [spark] SparkQA commented on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
SparkQA commented on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743945185 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37315/ ---

[GitHub] [spark] SparkQA commented on pull request #30741: [SPARK-33764][SS] Make state store maintenance interval as SQL config

2020-12-12 Thread GitBox
SparkQA commented on pull request #30741: URL: https://github.com/apache/spark/pull/30741#issuecomment-743944705 **[Test build #132713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132713/testReport)** for PR 30741 at commit [`6a0574c`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743944463 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743944463 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
SparkQA commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743944309 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37314/ -

[GitHub] [spark] SparkQA commented on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
SparkQA commented on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743943066 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37315/ -

[GitHub] [spark] SparkQA commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743942541 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37313/ ---

[GitHub] [spark] tnachen commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
tnachen commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743942486 Besides the comment it LGTM, I think if it's not set it's just using the default config so shouldn't break anything

[GitHub] [spark] tnachen commented on a change in pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
tnachen commented on a change in pull request #30739: URL: https://github.com/apache/spark/pull/30739#discussion_r541833752 ## File path: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala ## @@ -405,6 +405,20 @@ trait Mes

[GitHub] [spark] viirya commented on a change in pull request #30741: [SPARK-33764][SS] Make state store maintenance interval as SQL config

2020-12-12 Thread GitBox
viirya commented on a change in pull request #30741: URL: https://github.com/apache/spark/pull/30741#discussion_r541833066 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -1368,6 +1368,17 @@ object SQLConf { .intConf .creat

[GitHub] [spark] viirya commented on a change in pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
viirya commented on a change in pull request #30739: URL: https://github.com/apache/spark/pull/30739#discussion_r541690675 ## File path: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala ## @@ -405,6 +405,20 @@ trait Meso

[GitHub] [spark] SparkQA commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743941528 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37312/ ---

[GitHub] [spark] weixiuli edited a comment on pull request #30716: [SPARK-33747][CORE] Avoid calling unregisterMapOutput when the map stage is being rerunning.

2020-12-12 Thread GitBox
weixiuli edited a comment on pull request #30716: URL: https://github.com/apache/spark/pull/30716#issuecomment-743935047 @Ngone51 PTAL, ping @dongjoon-hyun Kindly review. This is an automated message from the Apache Git Serv

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743940704 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132711/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743940676 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132712/ -

[GitHub] [spark] SparkQA removed a comment on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
SparkQA removed a comment on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743939739 **[Test build #132712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132712/testReport)** for PR 30738 at commit [`e03b4b1`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
SparkQA removed a comment on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743939732 **[Test build #132711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132711/testReport)** for PR 30739 at commit [`8a5661c`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743940704 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132711/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743940676 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132712/ -

[GitHub] [spark] SparkQA commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
SparkQA commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743940674 **[Test build #132711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132711/testReport)** for PR 30739 at commit [`8a5661c`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
SparkQA commented on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743940647 **[Test build #132712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132712/testReport)** for PR 30738 at commit [`e03b4b1`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743940038 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37311/

[GitHub] [spark] SparkQA commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743940037 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37311/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743940038 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37311/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743364032 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743272130 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
SparkQA commented on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743939739 **[Test build #132712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132712/testReport)** for PR 30738 at commit [`e03b4b1`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
SparkQA commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743939732 **[Test build #132711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132711/testReport)** for PR 30739 at commit [`8a5661c`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743939666 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37313/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30748: [SPARK-33768][SQL] Remove `retainData` from `AlterTableDropPartition`

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30748: URL: https://github.com/apache/spark/pull/30748#issuecomment-743939451 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132705/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30746: [SPARK-33766][BUILD] Upgrade Jackson to 2.11.4

2020-12-12 Thread GitBox
AmplabJenkins removed a comment on pull request #30746: URL: https://github.com/apache/spark/pull/30746#issuecomment-743939450 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132707/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30746: [SPARK-33766][BUILD] Upgrade Jackson to 2.11.4

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30746: URL: https://github.com/apache/spark/pull/30746#issuecomment-743939450 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132707/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30748: [SPARK-33768][SQL] Remove `retainData` from `AlterTableDropPartition`

2020-12-12 Thread GitBox
AmplabJenkins commented on pull request #30748: URL: https://github.com/apache/spark/pull/30748#issuecomment-743939451 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132705/ -

[GitHub] [spark] SparkQA commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743939103 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37312/ -

[GitHub] [spark] SparkQA removed a comment on pull request #30746: [SPARK-33766][BUILD] Upgrade Jackson to 2.11.4

2020-12-12 Thread GitBox
SparkQA removed a comment on pull request #30746: URL: https://github.com/apache/spark/pull/30746#issuecomment-743928830 **[Test build #132707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132707/testReport)** for PR 30746 at commit [`bf4436b`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30746: [SPARK-33766][BUILD] Upgrade Jackson to 2.11.4

2020-12-12 Thread GitBox
SparkQA commented on pull request #30746: URL: https://github.com/apache/spark/pull/30746#issuecomment-743938729 **[Test build #132707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132707/testReport)** for PR 30746 at commit [`bf4436b`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30735: [SPARK-33748][K8S] Respect environment variables and configurations for Python executables

2020-12-12 Thread GitBox
SparkQA commented on pull request #30735: URL: https://github.com/apache/spark/pull/30735#issuecomment-743938002 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37311/ -

[GitHub] [spark] HyukjinKwon commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
HyukjinKwon commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743937646 @dmcwhorter mind keeping [the GitHub PR template](https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE)?

[GitHub] [spark] HyukjinKwon commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
HyukjinKwon commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-743937489 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

2020-12-12 Thread GitBox
HyukjinKwon commented on a change in pull request #30739: URL: https://github.com/apache/spark/pull/30739#discussion_r541825729 ## File path: resource-managers/mesos/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterSchedulerSuite.scala ## @@ -200,6 +201,60 @

[GitHub] [spark] HyukjinKwon commented on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
HyukjinKwon commented on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743937428 cc @vanzin FYI This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [spark] HyukjinKwon commented on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-12 Thread GitBox
HyukjinKwon commented on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-743937417 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30741: [SPARK-33764][SS] Make state store maintenance interval as SQL config

2020-12-12 Thread GitBox
HyukjinKwon commented on a change in pull request #30741: URL: https://github.com/apache/spark/pull/30741#discussion_r541825465 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -1368,6 +1368,17 @@ object SQLConf { .intConf .

[GitHub] [spark] HyukjinKwon edited a comment on pull request #30744: [SPARK-33589][SQL][FOLLOWUP] Replace Throwable with NonFatal

2020-12-12 Thread GitBox
HyukjinKwon edited a comment on pull request #30744: URL: https://github.com/apache/spark/pull/30744#issuecomment-743937032 cc @gatorsmile https://github.com/apache/spark/pull/30536#discussion_r541524866 This is an automated

[GitHub] [spark] HyukjinKwon commented on pull request #30744: [SPARK-33589][SQL][FOLLOWUP] Replace Throwable with NonFatal

2020-12-12 Thread GitBox
HyukjinKwon commented on pull request #30744: URL: https://github.com/apache/spark/pull/30744#issuecomment-743937032 cc @gatorsmile This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30745: [SPARK-33678][SQL] Product aggregation function

2020-12-12 Thread GitBox
HyukjinKwon commented on a change in pull request #30745: URL: https://github.com/apache/spark/pull/30745#discussion_r541824879 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Product.scala ## @@ -0,0 +1,93 @@ +/* + * Licensed to t

  1   2   3   >