@ Niklas/Jie, just sent the invitation to you :) ---- Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer Platform OpenSource Technology, STG, IBM GCG +86-10-8245 4084 | [email protected] | http://k82.me
On Tue, Mar 15, 2016 at 10:06 AM, Guangya Liu <[email protected]> wrote: > This is the google doc that we used to trace all discussion topics: > > https://docs.google.com/document/d/1B_v52zCOFcwCpqCPhgYi9h630a0NE-QM9Br0nCOZUR4/edit?usp=sharing > > This is the link for google hang out > video call Join video call > < > https://plus.google.com/hangouts/_/calendar/a2xhdXMxOTgyLmNuQGdtYWlsLmNvbQ.rgasirkilgmn1bm69kdcqjsb2s > >The > time is March 15th at 5pm PST > > Thanks, > > Guangya > > On Tue, Mar 15, 2016 at 9:47 AM, Benjamin Mahler <[email protected]> > wrote: > > > Sounds good, the next one is tomorrow March 15th at 5pm PST (they are at > > 5pm PST to accommodate China time zone). > > > > Will that work? > > > > On Mon, Mar 14, 2016 at 10:53 AM, Niklas Nielsen <[email protected]> wrote: > > > >> Ben, when do you have your next mesos allocator sync? We don't have our > >> next performance isolation sync lined up yet, so we could piggy back on > >> yours if you have it scheduled already. > >> > >> Niklas > >> > >> On Mon, Mar 14, 2016 at 9:32 AM, Jie Yu <[email protected]> wrote: > >> > >> > > > >> > > Just a quick note: Ian D. and the performance isolation working > group > >> are > >> > > discussing similar annotations and we should meet and talk about the > >> > > options. > >> > > >> > > >> > +1 > >> > > >> > Would love to understand the relationship between this and the > >> > task/executor level annotations. > >> > > >> > - Jie > >> > > >> > On Mon, Mar 14, 2016 at 9:29 AM, Niklas Nielsen <[email protected]> wrote: > >> > > >> > > Hi Ben, > >> > > > >> > > Just a quick note: Ian D. and the performance isolation working > group > >> are > >> > > discussing similar annotations and we should meet and talk about the > >> > > options. > >> > > > >> > > Niklas > >> > > > >> > > On Sat, Mar 12, 2016 at 12:05 AM, Klaus Ma <[email protected]> > >> > wrote: > >> > > > >> > > > Yes, I think that's true for now; so we define `ThrottleInfo` as > >> > message > >> > > to > >> > > > be more flexible. In Optimistic Offer Phase 1, we only use it to > >> > > > distinguish usage oversubscriptions and allocation > oversubscription, > >> > > > similar to bool :). > >> > > > > >> > > > Regarding the resources type, two questions after the discussion: > >> > > > > >> > > > 1. should we send different offer to the framework, so when > >> > > > usage/allocation oversubscription updated, only one type of offer > >> will > >> > be > >> > > > rescinded? > >> > > > 2. should we define framework's capability against `ThrottleInfo`? > >> > > > > >> > > > ---- > >> > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer > >> > > > Platform OpenSource Technology, STG, IBM GCG > >> > > > +86-10-8245 4084 | [email protected] | http://k82.me > >> > > > > >> > > > On Sat, Mar 12, 2016 at 12:03 PM, Guangya Liu <[email protected] > > > >> > > wrote: > >> > > > > >> > > > > > >> > > > > Hi Ben, > >> > > > > > >> > > > > I think that currently and even in the near future, the > >> > > __ThrottleInfo__ > >> > > > > will only be used by the usage oversubscriptions and the > >> > > oversubscription > >> > > > > for allocator (Both quota and reservations) will not use this > >> value > >> > but > >> > > > > only using __RevocableInfo__ is enough. > >> > > > > > >> > > > > I can even think that the __ThrottleInfo__ as a boolean value in > >> > > > > optimistic offer phase 1 as it is mainly used to distinguish > >> > resources > >> > > > > between usage oversubscriptions and allocation oversubscription > >> > (Quota > >> > > > and > >> > > > > Reservations), comments? > >> > > > > > >> > > > > Thanks, > >> > > > > > >> > > > > Guangya > >> > > > > > >> > > > > 在 2016年3月12日星期六 UTC+8上午11:09:46，Benjamin Mahler写道： > >> > > > > > >> > > > >> Hey folks, > >> > > > >> > >> > > > >> In the resource allocation working group we've been looking > into > >> a > >> > few > >> > > > >> projects that will make the allocator able to offer out > >> resources as > >> > > > >> revocable. For example: > >> > > > >> > >> > > > >> -We'll want to eventually allocate resources as revocable _by > >> > > default_, > >> > > > >> only allowing non-revocable when there are guarantees put in > >> place > >> > > > (static > >> > > > >> reservations or quota). > >> > > > >> > >> > > > >> -On the path to revocable by default, we can incrementally > start > >> to > >> > > > offer > >> > > > >> certain resources as revocable. Consider when quota is set but > >> the > >> > > role > >> > > > >> isn't using all of the quota. The unallocated quota can be > >> offered > >> > to > >> > > > other > >> > > > >> roles, but it should be revocable because we may revoke them > >> should > >> > > the > >> > > > >> quota'ed role want to use the resources. Unused reservations > fall > >> > > into a > >> > > > >> similar category. > >> > > > >> > >> > > > >> -Going revocable by default also allows us to enforce fairness > >> in a > >> > > > >> dynamically changing cluster by revoking resources as weights > are > >> > > > changed, > >> > > > >> frameworks are added or removed, etc. > >> > > > >> > >> > > > >> In this context, "revocable" means that the resources may be > >> taken > >> > > away > >> > > > >> and the container will be destroyed. The meaning of "revocable" > >> in > >> > the > >> > > > >> context of usage oversubscription includes this, but also the > >> > > container > >> > > > may > >> > > > >> experience a throttling (e.g. lower cpu shares, less network > >> > priority, > >> > > > etc). > >> > > > >> > >> > > > >> For this reason, and because we internally need to distinguish > >> > > revocable > >> > > > >> resources between the those that are generated by usage > >> > > oversubscription > >> > > > >> and those that are generated by the allocator, we're thinking > of > >> the > >> > > > >> following change to the API: > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> - message RevocableInfo {} > >> > > > >> + message RevocableInfo { > >> > > > >> + message ThrottleInfo {} > >> > > > >> + > >> > > > >> + // If set, indicates that the resources may be throttled > at > >> > > > >> + // any time. Throttle-able resoruces can be used for tasks > >> > > > >> + // that do not have strict performance requirements and > are > >> > > > >> + // capable of handling being throttled. > >> > > > >> + optional ThrottleInfo throttle_info; > >> > > > >> + } > >> > > > >> > >> > > > >> // If this is set, the resources are revocable, i.e., any > >> tasks > >> > or > >> > > > >> - // executors launched using these resources could get > >> preempted > >> > or > >> > > > >> - // throttled at any time. This could be used by frameworks > to > >> run > >> > > > >> - // best effort tasks that do not need strict uptime or > >> > performance > >> > > > >> + // executors launched using these resources could be > >> terminated > >> > at > >> > > > >> + // any time. This could be used by frameworks to run > >> > > > >> + // best effort tasks that do not need strict uptime > >> > > > >> // guarantees. Note that if this is set, 'disk' or > >> 'reservation' > >> > > > >> // cannot be set. > >> > > > >> optional RevocableInfo revocable = 9; > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> Essentially we want to distinguish between revocable and > >> revocable + > >> > > > >> throttle-able. This is because usage-oversubscription generates > >> > > > >> throttle-able revocable resources, whereas the allocator does > >> not. > >> > > This > >> > > > >> also solves our problem of distinguishing between these two > >> kinds of > >> > > > >> revocable resources internally. > >> > > > >> > >> > > > >> Feedback welcome! > >> > > > >> > >> > > > >> Ben > >> > > > >> > >> > > > >> -- > >> > > > > You received this message because you are subscribed to the > Google > >> > > Groups > >> > > > > "Mesos Resource Allocation Working Group" group. > >> > > > > To unsubscribe from this group and stop receiving emails from > it, > >> > send > >> > > an > >> > > > > email to [email protected]. > >> > > > > To post to this group, send email to > >> > [email protected] > >> > > . > >> > > > > To view this discussion on the web visit > >> > > > > > >> > > > > >> > > > >> > > >> > https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com > >> > > > > < > >> > > > > >> > > > >> > > >> > https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com?utm_medium=email&utm_source=footer > >> > > > > > >> > > > > . > >> > > > > For more options, visit https://groups.google.com/d/optout. > >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> > > -- > >> > > Niklas > >> > > > >> > > >> > >> > >> > >> -- > >> Niklas > >> > > > > >
