[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-179558866
  
**[Test build #50715 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50715/consoleFull)**
 for PR 8610 at commit 
[`db57a2e`](https://github.com/apache/spark/commit/db57a2e358e3e158338d531a4600be4a8cbc6a59).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-179561861
  
**[Test build #50719 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50719/consoleFull)**
 for PR 8610 at commit 
[`ae3e5e4`](https://github.com/apache/spark/commit/ae3e5e490aee08afde96875d91d16af60130c187).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-179563852
  
@andrewor14 thanks for reviewing this, I'm currently in traveling, I will 
move https://github.com/apache/spark/pull/4123 and the check here to 
SparkSubmit later 

I think we only need to check whether CoresPerExecutor is no less than 
CoresPerTask `only when the user has an explicit configuration on 
coresPerExecutor`

The reason is that when the user does not explicitly specify 
coresPerExecutor (default is 1), there are still `many chances that executors 
has more cores than coresPerTask`...if we simply throw exceptions to the user 
in this case, the user has to set an explicit value of coresPerExecutor 
`whenever they set a value on coresPerTask` which is not an ideal situtation in 
terms of user-friendly , your thoughts?




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-179561105
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50715/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-179561103
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-179558195
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-179558197
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50713/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-179561082
  
**[Test build #50715 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50715/consoleFull)**
 for PR 8610 at commit 
[`db57a2e`](https://github.com/apache/spark/commit/db57a2e358e3e158338d531a4600be4a8cbc6a59).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[andrewor14]

Github user andrewor14 commented on a diff in the pull request:

https://github.com/apache/spark/pull/8610#discussion_r51502059
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala 
---
@@ -571,7 +571,8 @@ private[deploy] class Master(
   usableWorkers: Array[WorkerInfo],
   spreadOutApps: Boolean): Array[Int] = {
 val coresPerExecutor = app.desc.coresPerExecutor
-val minCoresPerExecutor = coresPerExecutor.getOrElse(1)
+val coresPerTask = app.desc.coresPerTask
+val minCoresPerExecutor = math.max(coresPerExecutor.getOrElse(1), 
coresPerTask)
--- End diff --

I don't think we should do this. If the user explicitly specified 
`spark.executor.cores` they expect their executors to be scheduled with that 
many cores, no more no less.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[andrewor14]

Github user andrewor14 commented on a diff in the pull request:

https://github.com/apache/spark/pull/8610#discussion_r51502412
  
--- Diff: 
core/src/main/scala/org/apache/spark/deploy/ApplicationDescription.scala ---
@@ -26,10 +26,10 @@ private[spark] case class ApplicationDescription(
 command: Command,
 appUiUrl: String,
 eventLogDir: Option[URI] = None,
-// short name of compression codec used when writing event logs, if 
any (e.g. lzf)
--- End diff --

don't remove this...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[andrewor14]

Github user andrewor14 commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-178267213
  
By the way, I think it's totally fine to just do this for standalone mode 
first, since Mesos doesn't yet read `spark.executor.cores`. We can fix it for 
Mesos later.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[andrewor14]

Github user andrewor14 commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-178266813
  
@CodingCat thanks for working on this. Sorry that we haven't had the 
bandwidth to review this until now.

For this patch, I think it's better if we just throw an exception if 
`spark.task.cpus > spark.executor.cores`. We shouldn't start launching 
executors with more cores silently just because `spark.task.cpus` is big. That 
is not what I would expect as a user.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160196347
  
Jenkins, retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160218056
  
**[Test build #46829 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46829/consoleFull)**
 for PR 8610 at commit 
[`8232a80`](https://github.com/apache/spark/commit/8232a808e398f3644304822d1824aa0b923090dc).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160218097
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160218098
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46829/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160197699
  
**[Test build #46829 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46829/consoleFull)**
 for PR 8610 at commit 
[`8232a80`](https://github.com/apache/spark/commit/8232a808e398f3644304822d1824aa0b923090dc).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160001167
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46774/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160001142
  
**[Test build #46774 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46774/consoleFull)**
 for PR 8610 at commit 
[`8232a80`](https://github.com/apache/spark/commit/8232a808e398f3644304822d1824aa0b923090dc).
 * This patch **fails from timeout after a configured wait of \`250m\`**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160001166
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160005256
  
**[Test build #46785 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46785/consoleFull)**
 for PR 8610 at commit 
[`8232a80`](https://github.com/apache/spark/commit/8232a808e398f3644304822d1824aa0b923090dc).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160027038
  
Leave a mark here, there is something wrong with a particular test case 
causing the above failures, more details:   
https://issues.apache.org/jira/browse/SPARK-12021?jql=project%20%3D%20SPARK


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160004437
  
Jenkins, retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160017670
  
weird, something magical happened after Nov 23th 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160035436
  
**[Test build #46785 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46785/consoleFull)**
 for PR 8610 at commit 
[`8232a80`](https://github.com/apache/spark/commit/8232a808e398f3644304822d1824aa0b923090dc).
 * This patch **fails from timeout after a configured wait of \`250m\`**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160035458
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46785/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-160035457
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159924730
  
**[Test build #46768 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46768/consoleFull)**
 for PR 8610 at commit 
[`8232a80`](https://github.com/apache/spark/commit/8232a808e398f3644304822d1824aa0b923090dc).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159972641
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159972613
  
**[Test build #46768 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46768/consoleFull)**
 for PR 8610 at commit 
[`8232a80`](https://github.com/apache/spark/commit/8232a808e398f3644304822d1824aa0b923090dc).
 * This patch **fails from timeout after a configured wait of \`250m\`**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159972642
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46768/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159972730
  
Jenkins, retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159973804
  
**[Test build #46774 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46774/consoleFull)**
 for PR 8610 at commit 
[`8232a80`](https://github.com/apache/spark/commit/8232a808e398f3644304822d1824aa0b923090dc).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159792048
  
@dragos @tnachen  thanks for reviewing, I will handle your comments 
tomorrow 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[dragos]

Github user dragos commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159598342
  
I'll have a look at it tomorrow, thanks for pinging me.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[andrewor14]

Github user andrewor14 commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159041605
  
The 1.6 preview is already cut. This will have to come later. Until then, 
it would be good for @tnachen and @dragos to have a look.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159035418
  
Hi, @dragos , thanks for the comments, just sync the patch with master and 
addressed your comments

@andrewor14 any plan to fix this in the coming 1.6?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159016285
  
Jenkins, retest this please



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159053651
  
Jenkins, retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159053442
  
OK, thx

One more question, am I supposed to have the permission to trigger Jenkins 
to retest the patch, I thought  I was once able to do that...or it is due to 
the weird status of the bot?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159054509
  
**[Test build #46549 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46549/consoleFull)**
 for PR 8610 at commit 
[`f27f5d8`](https://github.com/apache/spark/commit/f27f5d88e30b31193ef6b45323eb1a60c6e64b49).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159054566
  
nvm, the bot fixed the relationship with me .


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[tnachen]

Github user tnachen commented on a diff in the pull request:

https://github.com/apache/spark/pull/8610#discussion_r45656480
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala 
---
@@ -639,10 +640,11 @@ private[deploy] class Master(
 // in the queue, then the second app, etc.
 for (app <- waitingApps if app.coresLeft > 0) {
   val coresPerExecutor: Option[Int] = app.desc.coresPerExecutor
+  val coresPerTask = app.desc.coresPerTask
   // Filter out workers that don't have enough resources to launch an 
executor
   val usableWorkers = workers.toArray.filter(_.state == 
WorkerState.ALIVE)
 .filter(worker => worker.memoryFree >= 
app.desc.memoryPerExecutorMB &&
-  worker.coresFree >= coresPerExecutor.getOrElse(1))
+  worker.coresFree >= math.max(coresPerExecutor.getOrElse(1), 
coresPerTask))
--- End diff --

IMO it's not clear what the relationship between coresPerTask and 
coresPerExecutor. Looking at the scheduling requirement we just need the 
maximum of the set to be able to schedule, and I thought cores per task is 
additional cpu resources on top of the executor. 
Can we perhaps comment where this is introduced what these two are? Or 
point to documentation?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159085372
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159085374
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46549/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-159085218
  
**[Test build #46549 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46549/consoleFull)**
 for PR 8610 at commit 
[`f27f5d8`](https://github.com/apache/spark/commit/f27f5d88e30b31193ef6b45323eb1a60c6e64b49).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-158994786
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46536/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-158994687
  
**[Test build #46536 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46536/consoleFull)**
 for PR 8610 at commit 
[`f27f5d8`](https://github.com/apache/spark/commit/f27f5d88e30b31193ef6b45323eb1a60c6e64b49).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-158995626
  
Jenkins, retest this please



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-158969995
  
**[Test build #46536 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46536/consoleFull)**
 for PR 8610 at commit 
[`f27f5d8`](https://github.com/apache/spark/commit/f27f5d88e30b31193ef6b45323eb1a60c6e64b49).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-158994783
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[dragos]

Github user dragos commented on a diff in the pull request:

https://github.com/apache/spark/pull/8610#discussion_r45551887
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala 
---
@@ -639,10 +640,11 @@ private[deploy] class Master(
 // in the queue, then the second app, etc.
 for (app <- waitingApps if app.coresLeft > 0) {
   val coresPerExecutor: Option[Int] = app.desc.coresPerExecutor
+  val coreNumPerTask = app.desc.coresPerTask
--- End diff --

nitpick: `coreNumPerTask` sounds weird.  `coresPerTaks` is what you've used 
everywhere else, and the name above is `coresPerExecutor`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-152505545
  
@andrewor14  would you have some chance to review this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[holdenk]

Github user holdenk commented on a diff in the pull request:

https://github.com/apache/spark/pull/8610#discussion_r43530497
  
--- Diff: 
core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala
 ---
@@ -83,8 +83,16 @@ private[spark] class SparkDeploySchedulerBackend(
   args, sc.executorEnvs, classPathEntries ++ testingClassPath, 
libraryPathEntries, javaOpts)
 val appUIAddress = sc.ui.map(_.appUIAddress).getOrElse("")
 val coresPerExecutor = 
conf.getOption("spark.executor.cores").map(_.toInt)
+if (coresPerExecutor.isDefined && (coresPerExecutor.get < 
scheduler.CPUS_PER_TASK ||
+  (coresPerExecutor.get % scheduler.CPUS_PER_TASK) != 0)) {
+  throw new SparkException(s"invalid configuration of " +
+s"spark.executor.cores(${coresPerExecutor.get}) and " +
+s"spark.task.cpus(${scheduler.CPUS_PER_TASK}}), 
spark.executor.cores has to be no less " +
+"than and folds of spark.task.cpus")
--- End diff --

clearer, thanks :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139066115
  
  [Test build #42217 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42217/console)
 for   PR 8610 at commit 
[`169141a`](https://github.com/apache/spark/commit/169141a36c5770f7cca1591e89ac61d2627461ab).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139066160
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139066161
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42217/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139095384
  
  [Test build #42229 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42229/consoleFull)
 for   PR 8610 at commit 
[`169141a`](https://github.com/apache/spark/commit/169141a36c5770f7cca1591e89ac61d2627461ab).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139094557
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139094447
  
Jenkins, retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139094542
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139092986
  
Jenkins, retest it please



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139116037
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139115777
  
  [Test build #42229 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42229/console)
 for   PR 8610 at commit 
[`169141a`](https://github.com/apache/spark/commit/169141a36c5770f7cca1591e89ac61d2627461ab).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139116039
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42229/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139036238
  
Jenkins, retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139036459
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139036439
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139036772
  
  [Test build #42217 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42217/consoleFull)
 for   PR 8610 at commit 
[`169141a`](https://github.com/apache/spark/commit/169141a36c5770f7cca1591e89ac61d2627461ab).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-138995243
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on a diff in the pull request:

https://github.com/apache/spark/pull/8610#discussion_r39076555
  
--- Diff: 
core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala
 ---
@@ -83,8 +83,16 @@ private[spark] class SparkDeploySchedulerBackend(
   args, sc.executorEnvs, classPathEntries ++ testingClassPath, 
libraryPathEntries, javaOpts)
 val appUIAddress = sc.ui.map(_.appUIAddress).getOrElse("")
 val coresPerExecutor = 
conf.getOption("spark.executor.cores").map(_.toInt)
+if (coresPerExecutor.isDefined && (coresPerExecutor.get < 
scheduler.CPUS_PER_TASK ||
+  (coresPerExecutor.get % scheduler.CPUS_PER_TASK) != 0)) {
+  throw new SparkException(s"invalid configuration of " +
+s"spark.executor.cores(${coresPerExecutor.get}) and " +
+s"spark.task.cpus(${scheduler.CPUS_PER_TASK}}), 
spark.executor.cores has to be no less " +
+"than and folds of spark.task.cpus")
--- End diff --

@holdenk how about now?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-138995204
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-138995607
  
  [Test build #42211 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42211/consoleFull)
 for   PR 8610 at commit 
[`169141a`](https://github.com/apache/spark/commit/169141a36c5770f7cca1591e89ac61d2627461ab).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139035887
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139035788
  
  [Test build #42211 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42211/console)
 for   PR 8610 at commit 
[`169141a`](https://github.com/apache/spark/commit/169141a36c5770f7cca1591e89ac61d2627461ab).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-139035888
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42211/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[holdenk]

Github user holdenk commented on a diff in the pull request:

https://github.com/apache/spark/pull/8610#discussion_r38983563
  
--- Diff: 
core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala
 ---
@@ -83,8 +83,16 @@ private[spark] class SparkDeploySchedulerBackend(
   args, sc.executorEnvs, classPathEntries ++ testingClassPath, 
libraryPathEntries, javaOpts)
 val appUIAddress = sc.ui.map(_.appUIAddress).getOrElse("")
 val coresPerExecutor = 
conf.getOption("spark.executor.cores").map(_.toInt)
+if (coresPerExecutor.isDefined && (coresPerExecutor.get < 
scheduler.CPUS_PER_TASK ||
+  (coresPerExecutor.get % scheduler.CPUS_PER_TASK) != 0)) {
+  throw new SparkException(s"invalid configuration of " +
+s"spark.executor.cores(${coresPerExecutor.get}) and " +
+s"spark.task.cpus(${scheduler.CPUS_PER_TASK}}), 
spark.executor.cores has to be no less " +
+"than and folds of spark.task.cpus")
--- End diff --

I don't think this error message is super clear, what is meant by "no less 
than and folds of spark.task.cups"?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

Github user CodingCat commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-138292843
  
Hi, @andrewor14 , I just added some test cases here, would you mind taking 
the review?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-138293310
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-138332631
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-138332538
  
  [Test build #42094 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42094/console)
 for   PR 8610 at commit 
[`44c6a03`](https://github.com/apache/spark/commit/44c6a03a4295db38e93cff150bc3ea84e0b7fa66).
 * This patch **passes all tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-138332632
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42094/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-138293269
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-138294877
  
  [Test build #42094 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42094/consoleFull)
 for   PR 8610 at commit 
[`44c6a03`](https://github.com/apache/spark/commit/44c6a03a4295db38e93cff150bc3ea84e0b7fa66).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137913133
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137913548
  
  [Test build #42037 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42037/consoleFull)
 for   PR 8610 at commit 
[`d3289fc`](https://github.com/apache/spark/commit/d3289fc331bd2e6bb19453f3489ff3ca4d865c90).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137924683
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42037/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137924668
  
  [Test build #42037 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42037/console)
 for   PR 8610 at commit 
[`d3289fc`](https://github.com/apache/spark/commit/d3289fc331bd2e6bb19453f3489ff3ca4d865c90).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137924682
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137913138
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[CodingCat]

GitHub user CodingCat opened a pull request:

https://github.com/apache/spark/pull/8610

[SPARK-5337][Mesos][Standalone] respect spark.task.cpus when scheduling 
Applications

https://issues.apache.org/jira/browse/SPARK-5337

Currently, we didn't consider spark.task.cpus when scheduling the 
applications in Master, so that we may fall into one of the following cases

the executor gets N cores but we need M cores to run a single task, where N 
< M

the executor gets N cores, we need M cores to run a single task, where N % 
M != 0 && N > M; so that we waste some cores in the executor

Patch for YARN is in submitted by @WangTaoTheTonic : #4123

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/CodingCat/spark SPARK-5337-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/8610.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #8610


commit d3289fc331bd2e6bb19453f3489ff3ca4d865c90
Author: CodingCat 
Date:   2015-09-05T06:01:05Z

respect spark.task.cpus when scheduling Applications




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137987456
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42059/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137989766
  
 Merged build triggered.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137989775
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137948696
  
Merged build started.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137962115
  
  [Test build #42054 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42054/console)
 for   PR 8610 at commit 
[`d3289fc`](https://github.com/apache/spark/commit/d3289fc331bd2e6bb19453f3489ff3ca4d865c90).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137962233
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137962236
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42054/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137987440
  
  [Test build #42059 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42059/console)
 for   PR 8610 at commit 
[`9c16dc7`](https://github.com/apache/spark/commit/9c16dc78c32d86444de9a88f01ba897edd9af87a).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/8610#issuecomment-137987454
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org