resending from the right account. sorry. On Fri, Sep 19, 2014 at 2:43 PM, Niklas Nielsen <nik...@mesosphere.io> wrote:
> Hi Dominic, > > (response inlined) > > On 19 September 2014 13:03, Dominic Hamon <dha...@twopensource.com> wrote: > > > I'm sorry, but I'm still having a hard time understanding why this needs > to > > be dynamic. > > > > If the mesos core is split into modules that are built as standalone > > libraries (static) then at link time the right combination of libraries > can > > be bundled together to create the end result. If you want to get even > > smarter, we can have default versions that are linked in to the mesos > core > > as weak symbols so later linked libraries can override the defaults. This > > may mean that we move to static linking across the board, but frankly > there > > are a few benefits to that approach. > > > This works for some, but not all use cases. > One use-case where it _does_ make sense to statically bake into the image > could be: https://issues.apache.org/jira/browse/MESOS-1330 > That be, you probably rarely want to swap network transport implementations > in and out on per-run basis but will know up front which one to configure > and use. > > On the other hand, having to relink and rebuilding give users a poor > experience and makes it hard to select (or unselect) custom components. > Using a prebuilt package and point against libraries is a pretty common > work-flow: Apache web server relies heavily on modules from dynamic > libraries. > I'm going to argue against this, so please bear with me while I work through it, and please point out anywhere that my assertions are wrong. The proposal you are suggesting requires the following: - callsites need to be modules aware to use the right factory method to instantiate the modular object - mesos-core owners need to be aware of modules and versioning when they change the abstract API - module owners need to be aware of the modules API and versioning - customers (end-users) have to be modules aware and set their runtime configuration correctly - errors in versioning or modules will only be caught at runtime My alternative proposal requires the following: - module owners need to be aware of API changes during core upgrades - issues are caught at compile time by module owners - customers (end-users) are unaware of modules and only need to know about per module configuration (kerberos attributes, for instance) not the modules system and configuration In short, your proposal adds more complexity and shifts the burden of knowledge and responsibility to every agent, instead of keeping it with the module owner where it belongs. I think the solution that's been put up for review is clever, but I don't think it solves the problem in the right way for anyone involved. Please rethink your approach to take that into account. > > TL;DR I don't see the two approaches to be mutually exclusive and we can > get a lot of leverage with the current design but we want to be able to do > this with static linking too. (To Tim St Clair's point) > > > > > > > > > With the approach as defined, does this mean that the default versions > will > > also have to be reimplemented as modules? > > > > Not necessary. Mesos default implementations stays as is. See Bernd's > comment. > > > > > > Has any effort been put into determining the performance overhead of the > > approach as specified? > > > > It won't affect Mesos if you are not using modules. The call sites will be > virtual dispatches no matter whether you are using modules or > internal/default implementations. > There is a performance penalty of not being able to do global optimizations > within the module, but that is the trade-off of implementing a dynamic > loadable module. > > > > > > On Fri, Sep 19, 2014 at 11:35 AM, Niklas Nielsen <nik...@mesosphere.io> > > wrote: > > > > > Hi everyone, > > > > > > We have been iterating on a design for pluggable modules in Mesos > lately > > > and wanted to get a last round of feedback before putting out patch > sets. > > > > > > Tim St Clair, Ben Hindman and I started the discussion (and work) on > this > > > subsystem https://issues.apache.org/jira/browse/MESOS-1224 and > > > https://issues.apache.org/jira/browse/MESOS-1384. > > > Kapil and Bernd took over the work (shepherded by Ben H and I) and have > > > expanded on the original design to cope with api/modules/mesos > versioning > > > semantics and be extensible enough to cope future changes in the > modules > > > subsystem (dealing with modules dependencies, etc). > > > > > > The latest description of the modules system has been captured in: > > > > > > > > > https://cwiki.apache.org/confluence/display/MESOS/DRAFT+Design+Doc+-+Mesos+Modules > > > (for those of you who don't want to read through the JIRA threads). > > > > > > We have an implementation ready based on this design and will be > sharing > > / > > > starting review rounds start next week. > > > > > > Feel free to use this thread if you have any questions. > > > > > > Cheers, > > > Niklas > > > > > > > > > > > -- > > Dominic Hamon | @mrdo | Twitter > > *There are no bad ideas; only good ideas that go horribly wrong.* > > > > Hope that helps; if not, we'd be happy to jump on a call. > > Niklas > -- Dominic Hamon | @mrdo | Twitter *There are no bad ideas; only good ideas that go horribly wrong.*
