Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1880/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88830/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #88830 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88830/testReport)**
for PR 20553 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1911/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1880/
---
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/20553
Merging once tests pass. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #88830 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88830/testReport)**
for PR 20553 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1879/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #88826 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88826/testReport)**
for PR 20553 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1879/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1910/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #88826 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88826/testReport)**
for PR 20553 at commit
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20553
Is this OK to merge?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1845/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88763/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #88763 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88763/testReport)**
for PR 20553 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1869/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1845/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #88763 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88763/testReport)**
for PR 20553 at commit
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20553
Agreed that `spark.kubernetes.executor.request.cores` is a better name.
Will change it.
---
-
To unsubscribe, e-mail:
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20553
Ah never mind sorry - thought we were referring to changing thread count
with `spark.kubernetes.executor.request.cores`. Think this config key makes
sense to communicate the exact K8s semantics
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20553
`spark.executor.cores` is the standard used by all the other cluster
managers, so we have to use that.
---
-
To unsubscribe,
Github user dvogelbacher commented on the issue:
https://github.com/apache/spark/pull/20553
Can we call this `spark.kubernetes.executor.request.cores`, similar to
`spark.kubernetes.executor.limit.cores` ?
I think this would make the naming a bit less confusing.
---
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20553
@foxish I think the confusion mostly comes from the property name. I don't
think we need to change the semantics of `spark.executor.cores` nor the way
dynamic resource allocation works in K8S
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/20553
Oops - knew we tried that before. Just paged in the discussion from
https://github.com/apache/spark/pull/20460. Looks like that approach might need
revisiting.
---
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/20553
@cloud-fan @jiangxb1987 I assume you're referring to
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20553
> Do you mean `spark.kubernetes.executor.cores` will only be used with k8s
for static allocation? It looks to me that if we wanna k8s work with Spark
dynamic allocation better, we have to change
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20553
> This is to avoid changing the semantics of spark.executor.cores and
spark.task.cpus and their role in task scheduling, task parallelism, dynamic
resource allocation, etc. The new configuration
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20553
The value of `spark.executor.cores` will be used to set cpu request for the
executor pods if `spark.kubernetes.executor.cores` is not set.
`spark.driver.cores` already allows fractional values
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20553
What is the default value if it is not configured, how do K8S control the
CPU usage by default?
Also it seems that user may configure how to differentiate between k8s
executor cores and
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20553
`spark.kubernetes.executor.cores` has nothing to do with dynamic resource
allocation. It's just a way of letting users specify a value for the cpu
resource request that conforms to Kubernetes
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20553
IIUC the `spark.kubernetes.executor.cores` here is just a special case for
`spark.executor.cores`, for k8s backend, you shall still have to handle float
values if you're to read the value of
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20553
also cc @cloud-fan @jerryshao
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20553
How do we plan to support dynamic allocation with k8s? Should we read
`spark.executor.cores` or `spark.kubernetes.executor.cores` ?
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1009/
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #87626 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87626/testReport)**
for PR 20553 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87626/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1009/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1015/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1008/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87625/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #87625 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87625/testReport)**
for PR 20553 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1008/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1014/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #87626 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87626/testReport)**
for PR 20553 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #87625 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87625/testReport)**
for PR 20553 at commit
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20553
Any comment or concern on this? Is this good to merge?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/950/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87538/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #87538 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87538/testReport)**
for PR 20553 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/950/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/956/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #87538 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87538/testReport)**
for PR 20553 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/724/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #87234 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87234/testReport)**
for PR 20553 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87234/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/724/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/729/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20553
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20553
**[Test build #87234 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87234/testReport)**
for PR 20553 at commit
73 matches
Mail list logo
