+1. On Sep 22, 2014, at 5:22 PM, Niklas Nielsen <nik...@mesosphere.io> wrote:
> Agreed - I can do early morning PST any day during this week (for people > dialing in from the east coast and Europe). > > Niklas > > On 22 September 2014 07:15, Tim St Clair <tstcl...@redhat.com> wrote: > >> Folks, >> >> I think this thread has fallen off the rails a bit, perhaps we could get a >> tcon/hangout going for this again. >> >> I would be happy to moderate if needed, but it's pretty clear to me to >> that trying to hash this out in an email thread is becoming >> couter-productive. >> >> Cheers, >> Tim >> >> ----- Original Message ----- >>> From: "Dominic Hamon" <dha...@twitter.com.INVALID> >>> To: "dev" <dev@mesos.apache.org> >>> Sent: Saturday, September 20, 2014 9:21:59 AM >>> Subject: Re: Mesos Modules Design >>> >>> On Fri, Sep 19, 2014 at 2:43 PM, Niklas Nielsen <nik...@mesosphere.io> >>> wrote: >>> >>>> Hi Dominic, >>>> >>>> (response inlined) >>>> >>>> On 19 September 2014 13:03, Dominic Hamon <dha...@twopensource.com> >> wrote: >>>> >>>>> I'm sorry, but I'm still having a hard time understanding why this >> needs >>>> to >>>>> be dynamic. >>>>> >>>>> If the mesos core is split into modules that are built as standalone >>>>> libraries (static) then at link time the right combination of >> libraries >>>> can >>>>> be bundled together to create the end result. If you want to get even >>>>> smarter, we can have default versions that are linked in to the mesos >>>> core >>>>> as weak symbols so later linked libraries can override the defaults. >> This >>>>> may mean that we move to static linking across the board, but frankly >>>> there >>>>> are a few benefits to that approach. >>>> >>>> >>>> This works for some, but not all use cases. >>>> One use-case where it _does_ make sense to statically bake into the >> image >>>> could be: https://issues.apache.org/jira/browse/MESOS-1330 >>>> That be, you probably rarely want to swap network transport >> implementations >>>> in and out on per-run basis but will know up front which one to >> configure >>>> and use. >>>> >>>> On the other hand, having to relink and rebuilding give users a poor >>>> experience and makes it hard to select (or unselect) custom components. >>>> Using a prebuilt package and point against libraries is a pretty common >>>> work-flow: Apache web server relies heavily on modules from dynamic >>>> libraries. >>>> >>>> >>> I'm going to argue against this, so please bear with me while I work >>> through it, and please point out anywhere that my assertions are wrong. >>> >>> The proposal you are suggesting requires the following: >>> >>> - callsites need to be modules aware to use the right factory method to >>> instantiate the modular object >>> - mesos-core owners need to be aware of modules and versioning when they >>> change the abstract API >>> - module owners need to be aware of the modules API and versioning >>> - customers (end-users) have to be modules aware and set their runtime >>> configuration correctly >>> - errors in versioning or modules will only be caught at runtime >>> >>> My alternative proposal requires the following: >>> >>> - module owners need to be aware of API changes during core upgrades >>> - issues are caught at compile time by module owners >>> - customers (end-users) are unaware of modules and only need to know >> about >>> per module configuration (kerberos attributes, for instance) not the >>> modules system and configuration >>> >>> In short, your proposal adds more complexity and shifts the burden of >>> knowledge and responsibility to every agent, instead of keeping it with >> the >>> module owner where it belongs. >>> >>> I think the solution that's been put up for review is clever, but I don't >>> think it solves the problem in the right way for anyone involved. Please >>> rethink your approach to take that into account. >>> >>> >>> >>>> TL;DR I don't see the two approaches to be mutually exclusive and we >> can >>>> get a lot of leverage with the current design but we want to be able >> to do >>>> this with static linking too. (To Tim St Clair's point) >>>> >>>> >>>>> >>>> >>>> >>>> >>>>> With the approach as defined, does this mean that the default >> versions >>>> will >>>>> also have to be reimplemented as modules? >>>>> >>>> >>>> Not necessary. Mesos default implementations stays as is. See Bernd's >>>> comment. >>>> >>>> >>>>> >>>>> Has any effort been put into determining the performance overhead of >> the >>>>> approach as specified? >>>>> >>>> >>>> It won't affect Mesos if you are not using modules. The call sites >> will be >>>> virtual dispatches no matter whether you are using modules or >>>> internal/default implementations. >>>> There is a performance penalty of not being able to do global >> optimizations >>>> within the module, but that is the trade-off of implementing a dynamic >>>> loadable module. >>>> >>>> >>>>> >>>>> On Fri, Sep 19, 2014 at 11:35 AM, Niklas Nielsen < >> nik...@mesosphere.io> >>>>> wrote: >>>>> >>>>>> Hi everyone, >>>>>> >>>>>> We have been iterating on a design for pluggable modules in Mesos >>>> lately >>>>>> and wanted to get a last round of feedback before putting out patch >>>> sets. >>>>>> >>>>>> Tim St Clair, Ben Hindman and I started the discussion (and work) >> on >>>> this >>>>>> subsystem https://issues.apache.org/jira/browse/MESOS-1224 and >>>>>> https://issues.apache.org/jira/browse/MESOS-1384. >>>>>> Kapil and Bernd took over the work (shepherded by Ben H and I) and >> have >>>>>> expanded on the original design to cope with api/modules/mesos >>>> versioning >>>>>> semantics and be extensible enough to cope future changes in the >>>> modules >>>>>> subsystem (dealing with modules dependencies, etc). >>>>>> >>>>>> The latest description of the modules system has been captured in: >>>>>> >>>>>> >>>>> >>>> >> https://cwiki.apache.org/confluence/display/MESOS/DRAFT+Design+Doc+-+Mesos+Modules >>>>>> (for those of you who don't want to read through the JIRA threads). >>>>>> >>>>>> We have an implementation ready based on this design and will be >>>> sharing >>>>> / >>>>>> starting review rounds start next week. >>>>>> >>>>>> Feel free to use this thread if you have any questions. >>>>>> >>>>>> Cheers, >>>>>> Niklas >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Dominic Hamon | @mrdo | Twitter >>>>> *There are no bad ideas; only good ideas that go horribly wrong.* >>>>> >>>> >>>> Hope that helps; if not, we'd be happy to jump on a call. >>>> >>>> Niklas >>>> >>> >>> >>> >>> -- >>> Dominic Hamon | @mrdo | Twitter >>> *There are no bad ideas; only good ideas that go horribly wrong.* >>> >> >> -- >> Cheers, >> Timothy St. Clair >> Red Hat Inc. >>
