GitHub user yanji84 opened a pull request:
https://github.com/apache/spark/pull/21033
[SPARK-19320][MESOS][WIP]allow specifying a hard limit on number of gpus
required in each spark executor when running on mesos
## What changes were proposed in this pull request?
Currently, Spark only allows specifying overall gpu resources as an upper
limit, this adds a new conf parameter to allow specifying a hard limit on the
number of gpu cores for each executor while still respecting the overall gpu
resource constraint
## How was this patch tested?
Unit Testing
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yanji84/spark ji/hard_limit_on_gpu
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21033.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21033
----
commit cec434a1eba6227814ba5a842ff8f41103217539
Author: Ji Yan <jiyan@...>
Date: 2017-03-10T05:30:11Z
respect both gpu and maxgpu
commit c427e151dbf63815f25d20fe1b099a7b09e85f51
Author: Ji Yan <jiyan@...>
Date: 2017-05-14T20:02:16Z
fix gpu offer
commit 1e61996c31ff3a01396738fd91adf69952fd3558
Author: Ji Yan <jiyan@...>
Date: 2017-05-14T20:15:55Z
syntax fix
commit f24dbe17787acecd4c032e25d820cb59d8b6d491
Author: Ji Yan <jiyan@...>
Date: 2017-05-15T00:30:50Z
pass all tests
commit f89e5ccae02667d4f55e7aeb1f805a9cfaee1558
Author: Ji Yan <jiyan@...>
Date: 2018-04-10T18:37:14Z
remove redundant
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
