[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19515 LGTM, merging to master / 2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19515 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83648/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19515 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19515 **[Test build #83648 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83648/testReport)** for PR 19515 at commit [`33a8e68`](https://github.com/apache/spark/commit/33a8e6880a468335330a7cb6507493de8b125faa). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19515 @pmackles perhaps you could email this to d...@spark.apache.org to get some visibility to this and hopefully someone else on the mesos side can review? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19515 **[Test build #83648 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83648/testReport)** for PR 19515 at commit [`33a8e68`](https://github.com/apache/spark/commit/33a8e6880a468335330a7cb6507493de8b125faa). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19515 @susanxhuynh or anyone from the mesos side would you please review? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19515 Jenkins, test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user pmackles commented on the issue: https://github.com/apache/spark/pull/19515 @felixcheung - any chance of getting this tiny change merged and included in the upcoming 2.2.1 release? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/19515 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user pmackles commented on the issue: https://github.com/apache/spark/pull/19515 @ArtRand - yeah default would remain the same which is 1g --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/19515 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/19515 Yeah that sounds good. Thanks for this! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user pmackles commented on the issue: https://github.com/apache/spark/pull/19515 @ArtRand - WDYT? I was going to switch it to ```SPARK_DISPATCHER_MEMORY``` but then I noticed that the other env vars for MesosClusterDispatcher or also prefixed with ```SPARK_DAEMON_*``` so I thought it might be better to keep the names consistent. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user pmackles commented on the issue: https://github.com/apache/spark/pull/19515 Hi @ArtRand - Based on my testing and interpretation of the code, ```SPARK_DRIVER_MEMORY``` has no affect on MesosClusterDispatcher. Heap size always winds up being set to ```-Xmx1G``` (the default). I went with ```SPARK_DAEMON_MEMORY``` because that's what is used for the Master/Worker in a standalone cluster and I think Dispatcher is pretty similar in nature to those daemons. That said, I don't have a strong opinion and ```SPARK_DISPATCHER_MEMORY``` works fine for my use case so I am happy to switch it in the name of getting this PR accepted. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/19515 Hello @pmackles, thanks for this. It would be set to the value of `SPARK_DRIVER_MEMORY` by default correct? What to you think about introducing a new envvar (`SPARK_DISPATCHER_MEMORY`) so that it can be individually configured? Only motivated by the fact that the Dispatcher is a pretty unique beast compared to the other Spark daemons. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19515 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
