Re: Let MesosContainerizer support ramdisk.

2016-12-26 Thread haosdent
@bingqiang This patch looks may take a few time to review. Could you create an associate ticket in https://issues.apache.org/jira/browse/MESOS ? Thank you! On Tue, Dec 27, 2016 at 10:51 AM, pangbingqiang wrote: > Hi All: > > As now mesoscontainer don’t support

Mesos 1.2.0

2016-12-26 Thread Adam Bordelon
It's about time for Mesos 1.2.0, given our two-month release cadence. I volunteer to be release manager, and I'd like to get a new committer or two to help out. Any volunteers? I expect to cut rc1 in early January, once all Blockers (and most Critical issues) have been resolved/deferred. Here's

Re: Mesos Spark Fine Grained Execution - CPU count

2016-12-26 Thread Chawla,Sumit
What is the expected effect of reducing the mesosExecutor.cores to zero? What functionality of executor is impacted? Is the impact is just that it just behaves like a regular process? Regards Sumit Chawla On Mon, Dec 26, 2016 at 9:25 AM, Michael Gummelt wrote: > >

Re: Mesos Spark Fine Grained Execution - CPU count

2016-12-26 Thread Jacek Laskowski
Thanks a LOT, Michael! Pozdrawiam, Jacek Laskowski https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 https://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski On Mon, Dec 26, 2016 at 10:04 PM, Michael Gummelt wrote: > In

Re: Mesos Spark Fine Grained Execution - CPU count

2016-12-26 Thread Michael Gummelt
In fine-grained mode (which is deprecated), Spark tasks (which are threads) were implemented as Mesos tasks. When a Mesos task starts and stops, its underlying cgroup, and therefore the resources its consuming on the cluster, grows or shrinks based on the resources allocated to the tasks, which

Re: Mesos Spark Fine Grained Execution - CPU count

2016-12-26 Thread Jacek Laskowski
Hi Michael, That caught my attention... Could you please elaborate on "elastically grow and shrink CPU usage" and how it really works under the covers? It seems that CPU usage is just a "label" for an executor on Mesos. Where's this in the code? Pozdrawiam, Jacek Laskowski

Re: Mesos Spark Fine Grained Execution - CPU count

2016-12-26 Thread Michael Gummelt
> Using 0 for spark.mesos.mesosExecutor.cores is better than dynamic allocation Maybe for CPU, but definitely not for memory. Executors never shut down in fine-grained mode, which means you only elastically grow and shrink CPU usage, not memory. On Sat, Dec 24, 2016 at 10:14 PM, Davies Liu