Great comments Vinod! ----- Original Message ----- > From: "Vinod Kone" <vinodk...@gmail.com> > To: "dev" <dev@mesos.apache.org> > Sent: Tuesday, September 23, 2014 1:59:26 PM > Subject: Re: Mesos Modules Design > > Ok. I finally had a chance to read the design doc, go through the comments > on this thread and glance at the review. Here are my comments. > > I like the concept of dynamic loading of libraries when it comes to making > the lives of end users/operators easy, esp when they want to mix and match > the modules. > > I agree with Dominic though that we should make sure to minimize the > burden/overhead of core developers when it comes to dealing with modules. > FWIW, the implementation that I've seen seems easy enough to deal with and > doesn't look like it adds much of a performance overhead.
+1, right now I think it's pretty non-intrusive. > > That said, the versioning part seems a bit complicated, as BenH alluded to. > Can we come up with something that's simple as the first cut but still > extensible? Also, it would be great if interface breakage could be > automatically deduced at load time. +1 - I think it's still POC at this point. You'll never know how an interface is going to evolve until it's actually written and used. The current semantics are pretty simple, but I think once we start iterating it will become apparent. Ideally I would like to have some way of denoting EXPERIMENTAL such that external clients don't depend on api until it's hardened. > But I guess we still need a module > version to deal with protocol changes without interface changes. To ensure > the devs update the versions in module manager, I suggest adding a comment > on top of the current components (Allocator, Isolator etc) mentioning that > there is a corresponding module that needs to be updated. Better yet, maybe > we could use a naming convention (e.g., s/Allocator/AllocatorModule/) for > easy visual cue. Good idea, perhaps using "Plugin" vs. Module. > > More importantly, I would like to understand how we are going to ensure the > quality of modules being written. In other words how do we, as a community, > ensure that a badly written module doesn't leave a bad taste in users > adopting Mesos. Simple semantics always work, MESOS FAILED TO LOAD (BOB'S) EXTERNAL MODULE.. Fail early. "Please check YOUR XYZ CONFIG FILE" > Should we come up with a test suite (functional and > performance) that runs on CI that module writers are required to test > against? What is the contract for support when something in modules break? > Should the users/operators ask on #mesos and dev lists or module writer's > lists? Would the users even know where the problem lies? Would Mesos devs > always know? I think this is where most of the burden/overhead lies. Of > course, this is the same problem with frameworks, but I think this is much > more relevant for modules because they (could) fundamentally change the > behavior of Mesos. Test suites are always good, perhaps we could breakout a couple of JIRAs around this. > > Even more fundamentally, how are we going to ensure that Mesos core gets > better features vs having the cool features developed as (possibly paid) > modules? For example, should Kerberos auth and SSL transport be modules or > should they be integrated into the core? As an open source project, how do > we ensure that the community resources are properly utilized in a fair and > neutral manner to help Mesos core grow? You've asked a very difficult question to answer. Stewardship depends almost entirely on a combination of contributor ethics + organizations business model. Luckily now a days, upstream 1st is the most common mode of operation as it ensures you have many eyeballs looking at it. (With enough people, all bugs are shallow). > Are we going to have > guidelines/opinions on what should/could be modularized or is everything > fair game? It would be great to understand how other successful open source > projects toe this line. Has anyone done any research regarding this? Right now I think jurisprudence is in order. Only make something a plugin if there is legitimate reason to do so. > > > On Tue, Sep 23, 2014 at 9:27 AM, George Sudarkoff <geo...@sudarkoff.com> > wrote: > > > Hello everybody! > > > > I'm new here. But why not jump in in the middle of the conversation and > > voice an opinion anyway, right? :) > > > > On 23 Sep 2014, at 09:17, Dominic Hamon <dha...@twopensource.com> wrote: > > > > > On Tue, Sep 23, 2014 at 5:57 AM, Tim St Clair <tstcl...@redhat.com> > > wrote: > > > > > >> > > >> > > >> ----- Original Message ----- > > >>> From: "Benjamin Hindman" <b...@eecs.berkeley.edu> > > >>> To: "dev" <dev@mesos.apache.org> > > >>> Sent: Tuesday, September 23, 2014 3:14:31 AM > > >>> Subject: Re: Mesos Modules Design > > >>> > > >>>> > > >>>> - create abstract classes to define interfaces to objects that should > > >> be > > >>>> modular > > >>>> > > >>> > > >>> We're all in agreement here! > > >>> > > >>> - build modules as static libraries that can be assembled at link time > > to > > >>>> create custom Mesos builds > > >>>> > > >>> > > >>> Okay, but unless I'm missing something here we'll still need a level of > > >>> indirection to wire everything together. What would that look like? > > >>> > > >>> Also, why ask an operator to go through the extra step of relinking > > >> Mesos? > > >>> Asking the operator to relink means they'll need a Mesos build > > >> environment, > > >>> while most folks will likely just have Mesos installed via an RPM (or > > >>> similar). I'm not convinced that getting a link error will be a better > > >> user > > >>> experience then getting a runtime error that cleanly prints out > > something > > >>> along the lines of "Version mismatch: the XXYYZZ module is not > > compatible > > >>> with this version of Mesos". > > >> > > >> To ask service operators to re-link and possibly re-deploy mesos is a > > >> non-starter imho. One of the goals of enabling "plugins" around key > > >> interfaces is to avoid this type of operation. > > >> > > > > > > ​What, concretely, does a service operator do if they have a bunch of > > > modules that give runtime version errors? What are there options to get a > > > running version? > > > > "Runtime" doesn't necessarily mean "four days after you start it". What > > I'd expect from a piece of software with plugins is to load the plugins and > > verify the versions/compatibility at launch time. If I drop a new plugin in > > and restart a client, I know what to do when the client complains about the > > incompatible versions. > > > > > > -- George > > > > > > > > > > > > > > > >> > > >> -- > > >> Cheers, > > >> Timothy St. Clair > > >> Red Hat Inc. > > >> > > > > > > > > > > > > -- > > > Dominic Hamon | @mrdo | Twitter > > > *There are no bad ideas; only good ideas that go horribly wrong.* > > > > > -- Cheers, Timothy St. Clair Red Hat Inc.
