Kapil— great your started this topic. I would like to add my 4¢ to the discussion.
First, I think we should support two use cases: installing 3rdparty packages and exposing them in the local build. As a Mesos (module) developer, I may have different versions of Mesos on my machine and I install neither of them to avoid conflicts. If I develop a module for a particular Mesos version (i.e. build), I would like to use artifacts of that build without installing them anywhere. Second, additionally 3rdparty packages, how about modules we provide with Mesos, e.g. FixedResourceEstimator? Also if in the future we decide to refactor default implementations into modules (e.g. hierarchical allocator), we should install them somewhere. On Wed, Jan 20, 2016 at 8:09 AM, Kapil Arya <[email protected]> wrote: > On Wed, Jan 20, 2016 at 2:04 AM, Marco Massenzio <[email protected]> > wrote: > > Great thinking, Kapil! > > (I'm one who got the headache :) > > > > However, having recently gone through the effort of having to figure out > it > > all, my +1 goes for *good documentation* of what is necessary. > > Totally with you on this :) > > > > When installing stuff / magic happening behind the scenes, it is always > > difficult to ensure it works on all "supported" platforms (and let's not > > even go into non-supported ones) and we would also run the risk that > future > > changes may inadvertently break it. > > > > Bear also in mind that folks (who may not be using the --prefix) may be > > "surprised" to find incompatible/unexpected versions of > glog/protobuf/etc. > > in the /usr/local system dir. > > This is the reason why we want to put it inside Mesos "owned" > directories (e.g., /usr/local/include/mesos/3rdparty, > /usr/lib64/mesos/3rdparty) so that this whole operation is side-effect > free for other applications/packages installed on the system. > > > We could also provide "exemplary scripts" that automate (most of) the > > tedious work and example build files, alongside documentation. > > If folks agree that this is a desirable alternative, I'm happy to help > out > > - as mentioned, I've recently been through this, so memory is still > fresh. > > I think several of us have developed such scripts by now. However, the > problem is maintainability as they get out-of-sync whenever a 3rdparty > component is updated in Mesos :-/. > > > > > My 2c. > > > > -- > > *Marco Massenzio* > > http://codetrips.com > > > > On Tue, Jan 19, 2016 at 10:22 PM, Kapil Arya <[email protected]> > wrote: > > > >> On Tue, Jan 19, 2016 at 11:58 PM, James Peach <[email protected]> wrote: > >> > > >> >> On Jan 19, 2016, at 2:03 PM, Kapil Arya <[email protected]> wrote: > >> >> > >> >> Hi All, > >> >> > >> >> I wanted to get your opinion on installing the 3rdparty packages > glog, > >> >> protobuf, boost and picojson[1] when installing Mesos itself. These > >> >> packages are required to build Mesos modules. > >> > > >> > An alternative approach could be to hide these headers so they are > >> internal to Mesos and not incidentally required by innocent modules. > IIRC > >> this should be fairly easy for picojson, but (much) harder for glog, > >> protobuf and boost. > >> > >> That's a lot of work and super hard to maintain IMHO :(. > >> > >> >> Currently, a module write has to manually install these 3rdparty > >> >> packages, either system-wide or locally, and update the compilation > >> >> flags such as CPPFLAGS to point to the installation which is > >> >> error-prone. Further, one might have a system-wide installation with > >> >> the wrong package version, causing even more headache. > >> > > >> > If you are looking at this, could you please also look at these: > >> > https://issues.apache.org/jira/browse/MESOS-2537 > >> > https://issues.apache.org/jira/browse/MESOS-4096 > >> > > >> > MESOS-2537 provides consistent behaviour for building against > optionally > >> bundled dependencies (and fixes the --enable-foo semantics). > >> > >> I'll take a look at this one. > >> > >> > MESOS-4096 makes it impossible to run stout tests against a protobuf > >> that is not version 2.5.0. > >> > >> At some point, AlexR and I tried to work on it, but with the stout > >> directory structure, it got harder to fix then it seemed at first. > >> > >> > > >> >> The proposal here is to install these 3rdparty packages when > >> >> installing Mesos. To avoid any conflicts with system-wide or local > >> >> installation, we can install them as follows: > >> >> > >> >> ${PREFIX}/include/mesos/3rdparty -- for header files > >> >> ${PREFIX}/<LIBDIR>/mesos/3rdparty -- for library files (LIBDIR can be > >> >> lib or lib64 depending upon the installation) > >> >> > >> >> where PREFIX refers to the `--prefix` flag for Mesos configure > script. > >> >> > >> >> We would then update `mesos.pc` with the correct flags so that a > >> >> module write can simply use `pkg-config` to get all the required > >> >> flags. > >> >> > >> >> I have created an issue > >> >> https://issues.apache.org/jira/browse/MESOS-4434 to track this. > >> >> > >> >> Best, > >> >> Kapil > >> >> > >> >> > >> >> [1]: picojson is currently installed in ${PREFIX}/include. See > >> >> https://issues.apache.org/jira/browse/MESOS-3909 > >> > > >> >
