Re: Proposal for Mesos Build Improvements

[Alex Clemmer]

Yes, that is right, PCHs would probably introduce some additional
dependencies for some object files, and if those PCHs become bloated
over time, then you can expect this to be expressed as diminishing time
savings.

This does imply that maintaining PCHs will require at least some work.


__
Transcribed by my voice-enabled refrigerator, please pardon chilly messages.

On Wed, 15 Feb 2017, Neil Conway wrote:


On Tue, Feb 14, 2017 at 11:28 AM, Jeff Coffler
 wrote:

For efficiency purposes, if a header file is included by 50% or more of the 
source files, it should be included in the precompiled header. If a header is 
included in fewer than 50% of the source files, then it can be separately 
included (and thus would not benefit from precompiled headers). Note that this 
is a guideline; even if a header is used by less than 50% of source files, if 
it's very large, we still may decide to throw it in the precompiled header.


It seems like this would have the effect of creating many false
dependencies: if file X doesn't currently include header Y but Y is
included in the precompiled header, the symbols in Y will now be
visible when X is compiled. It would also mean that X would need to be
recompiled when Y changes.

Related: the current policy is that headers and implementation files
should try to include all of their dependencies, without relying on
transitive includes. For example, if foo.cpp includes bar.hpp, which
includes , but foo.cpp also uses , both foo.cpp and
bar.hpp should "#include ". Adopting precompiled headers would
mean making an exception to this policy, right?

I wonder if we should instead use headers like:

<- mesos_common.h ->
#include 
#include 
#include 

<- xyz.cpp, which needs headers "b" and "d" ->
#include "mesos_common.h>

#include 
#include 

That way, the fact that "xyz.cpp" logically depends on  (but not
 or ) is not obscured (in other words, Mesos should continue to
compile if 'mesos_common.h' is replaced with an empty file). Does
anyone know whether the header guard in  _should_ make the repeated
inclusion of  relatively cheap?

Neil

Re: Proposal for Mesos Build Improvements

[Alex Clemmer]

Just to add a bit of context, the history of the issue of build time is
tracked in MESOS-1582[1], and most recently[2].

Speaking personally, I'm excited about _any_ progress in this area,
because (1) the Windows build times are completely unbearable, and (2)
because getting the build times down benefits the whole community.

When it was basically just me working on the Windows code paths, this
issue was tolerable, but now that we have multiple people working
full-time, it is really important to start fixing the issue.

[1] https://issues.apache.org/jira/browse/MESOS-1582
[2]
https://issues.apache.org/jira/browse/MESOS-1582?focusedCommentId=15828645=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15828645


__
Transcribed by my voice-enabled refrigerator, please pardon chilly messages.

On Tue, 14 Feb 2017, Jeff Coffler wrote:


Proposal For Build Improvements

The Mesos build process is in dire need of some build infrastructure 
improvements. These improvements will improve speed and ease of work in 
particular components, and dramatically improve overall build time, especially 
in the Windows environment, but likely in the Linux environment as well.


Background:

It is currently recommended to use the ccache project with the Mesos build 
process. This makes the Linux build process more tolerable in terms of speed, 
but unfortunately such software is not available on Windows. Ultimately, 
though, the caching software is covering up two fundamental flaws in the 
overall build process:

1. Lack of use of libraries
2. Lack of precompiled headers

By not allowing use of libraries, the overall build process is often much 
longer, particularly when a lot of work is being done in a particular 
component. If work is being done in a particular component, only that library 
need be rebuilt (and then the overall image relinked). Currently, since there 
is no such modularization, all source files must be considered at build time. 
Interestingly enough, there is such modularization in the source code layout; 
that modularization just isn't utilized at the compiler level.

Precompiled headers exist on both Windows and Linux. For Linux, you can refer to 
https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html. Straight from the GNU CC 
documentation: "The time the compiler takes to process these header files over and 
over again can account for nearly all of the time required to build the project."

In my prior use of precompiled headers, each C or C++ file generally took about 
4 seconds to compile. After switching to precompiled headers, the precompiled 
header creation took about 4 seconds, but each C/C++ file now took about 200 
milliseconds to compile. The overall build speed was thus dramatically reduced.


Scope of Changes:

These changes are only being proposed for the CMake system. Going forward, the 
CMake system is the easiest way to maintain some level of portability between 
the Linux and Windows platforms.


Details for Modularization:

For the modularization, the intent is to simply make each source directory of 
files, if functionally separate, to be compiled into an archive (.a) file. 
These archive files will then be linked together to form the actual 
executables. These changes will primarily be in the CMake system, and should 
have limited effect on any actual source code.

At a later date, if it makes sense, we can look at building shared library 
(.so) files. However, this only makes the most sense if the code is truly 
shared between different executable files. If that's not the case, then it 
likely makes sense just to stick with .a files. Regardless, generation of .so 
files is out of scope for this change.


Details for Precompiled Header Changes:

Precompiled headers will make use of stout (a very large header-only library) essentially 
"free" from a compile-time overhead point of view. Basically, precompiled headers will 
take a list of header files (including very long header files, like "windows.h"), and 
generate the compiler memory structures for their representation.

During precompiled header generation, these memory structures are flushed to 
disk. Then, when components are built, the memory structures are reloaded from 
disk, which is dramatically faster than actually parsing the tens of thousands 
of lines of header files and building the memory structures.

For precompiled headers to be useful, a relatively "consistent" set of headers 
must be included by all of the C/C++ files. So, for example, consider the following C 
file:

#if defined(windows)
#include 
#endif

#include 
#include 
#include 

< - Remainder of module - >

To make a precompiled header for this module, all of the #include files would 
be included in a new file, mesos_common.h. The C file would then be changed as 
follows:

#include "mesos_common.h"

< - Remainder of module - >

Structurally, the code is identical, and need not be built with precompiled 
headers. However, use of 

Re: Introduction of `int_fd`.

[Alex Clemmer]

It's great to see this shipped! Thanks, mpark.

As an aside: One of the impacts of this work is that it is the last
major change blocking the Windows Containers patches, which will be very
exciting to see finally lit up.

I will follow up this work with some more in-depth tests to prevent
regressions around some of the more subtle behavior of the Windows
codepaths. (The POSIX codepaths are supposed to remain unaffected, so
the risk is incurred only on the Windows side.)


__
Transcribed by my voice-enabled refrigerator, please pardon chilly messages.

On Mon, 6 Feb 2017, Michael Park wrote:


I'd like to announce that we've introduced an `int_fd` type in stout.
It is __not__ an RAII file descriptor type! It's actually not fancy
what-so-ever.

On POSIX, it's simply aliased to an `int`, so there's no change in usage at
all.
Just remember to use `int_fd` rather than `int` for file descriptors.

If you're wondering why we introduced this in the first place, continue
reading:

On Windows, `int_fd` is aliased to a type called `WindowsFD`. The reason is
that the abstraction on Windows has multiple types that represent
"file descriptors" from various APIs. Namely, WinCRT, Win32, WinSock.

Due to this, on Windows we have to deal with `int` from WinCRT, `HANDLE`
from Win32, and `SOCKET` from WinSock. In order to not require large changes
through the entire codebase, we've decided to rendezvous at the POSIX `int`.

`WindowsFD` is one of: `int`, `HANDLE`, and `SOCKET`, and tries its best to
provide the same API as the POSIX `int` file descriptor. Notably, the
comparison
operators are implemented to support expressions such as `fd < 0` and `fd
!= -1`.

Thanks to Daniel Pravat, Alex Clemmer, and Benjamin Hindman for helping out
with this work.

You know who you ping if you run into issues or have questions =D

Thanks,

MPark

Re: CMake and CLion

[Alex Clemmer]

Hey folks. Sorry about getting back to this so late. I've been out of
country for about 2.5 weeks, and we've been trying to make sure all the
Windows patches land in Mesos 1.0.

Just a quick note, the Windows MVP commit brings the CMake build system
much closer to being a viable alternative to autoconf:
https://reviews.apache.org/r/48000 As I type this I see that it has a
nice, round review number, 48000. A nice way to cap off that work.

Just as an aside, it might be useful for me to explain why the CMake
work has historically been incomplete and/or broken. When we started the
CMake work, the impetus was to build a compelling cross-platform build
system that in particular could support Windows. Its progress has
historically been pinned to the Windows work, both because we didn't
want to have to rewrite the build system to accommodate Windows, and
because historically I have done nearly all of the foundational work on
it, which means that development has been limited by my availability.

Looking into the future, as the Windows work has landed, my immediate
priority is equipping the community with the tools it needs to keep
the CMake (and CMake-on-Windows) builds stable -- integration with CI,
and so on.


__
Transcribed by my voice-enabled refrigerator, please pardon chilly messages.

On Mon, 30 May 2016, Frank Scholten wrote:


Cool, building it as we speak.

On Thu, May 26, 2016 at 1:20 PM, Juan Larriba <jlarr...@gmail.com> wrote:

-Hi Frank,

sorry for the delay, but my patch was submittede yestarday. Currently the
mesos agent is building using CMake either with gcc and clang. To build the
agent in Ubuntu or CentOS, you just have to create a new dir outside the
/mesos directory you cloned:

mkdir mesos_build

cd to it:
cd mesos_build

launch cmake on the mesos source dir:
cmake ../mesos

and then, make:

make

This will generate the agent, if you want to test it,

make check.

Cheers!

On 18 May 2016 at 15:06, Frank Scholten <fr...@frankscholten.nl> wrote:


Ok, thx. I pinged Alex on Twitter and I will ask him how I can help.

On Wed, May 18, 2016 at 2:40 PM, Jan Schlicht <j...@mesosphere.io> wrote:

Alex Clemmer is probably the right person to talk to regarding tasks for
improving the CMake build. Most (probably all?) devs still use
autoconf/automake for their every day work while MESOS-898 is still in
progress. That would also explain why certain things don't work yet. The
things that do work, though, should compile just fine. Judging from the
`CMakeLists.txt` that should be creating the `libmesos.so` library and

its

dependencies but no agent yet.

On Wed, May 18, 2016 at 1:21 PM, Frank Scholten <fr...@frankscholten.nl>
wrote:


How can I build the agent? I can't find an add_executable definition
in the CMakeLists.txt

I added this to src/CMakeLists.txt

{code}
# Agent executable
##
add_executable(${AGENT_TARGET} ${AGENT_SRC})
{code}

however it fails to build when I run

$ make mesos-agent

Also if I first run

$ make mesos-0.29.0

I get a lot of undefined references errors.


On Wed, May 18, 2016 at 11:42 AM, haosdent <haosd...@gmail.com> wrote:

I think you could refer to Bplotka's repo:
https://github.com/Bplotka/docker-mesos-clion

Juan have a patch to show how build Mesos via CMake as well.
https://reviews.apache.org/r/45668/

Noted that so far CMake only could build the Mesos Agent(Slave)

component,

don't include Mesos Master.

On Wed, May 18, 2016 at 4:58 PM, Frank Scholten <

fr...@frankscholten.nl>

wrote:


Hi all,

How can I help out with the CMake build of Mesos? This seems to be

the

epic related to it https://issues.apache.org/jira/browse/MESOS-898

Are there specific issues I can look at? I run Ubuntu 16.04

Cheers,

Frank





--
Best Regards,
Haosdent Huang






--
*Jan Schlicht*
Distributed Systems Engineer, Mesosphere

Re: Build Mesos on Windows

[Alex Clemmer]

Hey folks. Back from the insanity of //build.

Just wanted to poke my head in here and say: we will need to improve
the documentation here, but you actually *do* need to run the
`./bootstrap` for the CMake stuff. That is actually copying the git
post-commit hooks and stuff like that, and setting up the
`.gitignore`.

I will look at both Joerg's reviews and Juan's reviews shortly.

Please let me know what else shoudl be on my plate as I emerge from my burrow.

On Sun, Apr 3, 2016 at 11:58 PM, Juan Larriba  wrote:
> Submitted the review https://reviews.apache.org/r/45668/ and the Jira
> MESOS-5101. I cannot assign the issue to myself because I am not a
> contributor.
>
> On 3 April 2016 at 20:30, Juan Larriba  wrote:
>>
>> I submitted everything wrong it seems, so I am going to close the current
>> review and add a new one done correctly. Sorry for the inconvenience.
>>
>> On 3 April 2016 at 18:30, Juan Larriba  wrote:
>>>
>>> Thank you haosdent, I did not notice that the bug was already fixed.
>>>
>>> I have opened the Jira https://issues.apache.org/jira/browse/MESOS-5101
>>> and submitted a updated a different patch file to the review board.
>>>
>>> On 3 April 2016 at 18:08, haosdent  wrote:

 Hi, I think your build failure have already resolved by @Joerg
 https://reviews.apache.org/r/45582/

 Could you rebase it and fill a ticket for your docker_build.sh update in
 https://issues.apache.org/jira/browse/MESOS?

 On Sun, Apr 3, 2016 at 11:45 PM, Juan Larriba 
 wrote:

 > I have created the review https://reviews.apache.org/r/45656/ that
 > solves
 > the problem with CMake build in Linux (it was failing) and updates
 > docker_build.sh to support CMake build. I have enabled the CMake build
 > as a
 > new compiler option, so it can be built alongside GCC and clang.
 >
 > Please notice that for CMake builds,  the ENVIRONMENT env variable is
 > not
 > used, so it does not need to be set. The CONFIGURE variable must be,
 > at
 > least:
 >
 > export CONFIGURE='-G "Unix Makefiles"'
 >
 > and accepts also the parameter -DENABLE_LIBEVENT:BOOL="1" if libevent
 > wants
 > to be used instead of libev,
 >
 > The COMPILER variable must be "cmake".
 >
 > Also, the CMake compilation does not need to perform the ./bootstrap
 > action.
 >
 > It has been tested in ubuntu:14.04 and centos:7.
 >
 > On 29 March 2016 at 08:44, Juan Larriba  wrote:
 >
 > > CMake build currently fails on Linux, so I am working on fixing the
 > > Agent
 > > build before sending the review for the modified docker_build.sh
 > >
 > > On 27 March 2016 at 09:39, Juan Larriba  wrote:
 > >
 > >> OK Vinod, I will work on it.
 > >>
 > >> On 26 March 2016 at 20:45, Vinod Kone  wrote:
 > >>
 > >>> On Sat, Mar 26, 2016 at 3:12 AM, Juan Larriba 
 > >>> wrote:
 > >>>
 > >>> > For your advice on contribution, both tasks you have stated
 > >>> > sound
 > >>> great to
 > >>> > me, and adapting the CI system for CMake build sounds great. If
 > >>> > you
 > can
 > >>> > point me to the CI server the project is using for the builds
 > >>> > (as I
 > >>> don't
 > >>> > manage to find it by myself), I can clone the setting in my
 > >>> > local
 > test
 > >>> > environment to begin doing tests on CMake builds, identify the
 > >>> > needed
 > >>> > changes and ask for integrate them on the CI server.
 > >>> >
 > >>>
 > >>>
 > >>> I'm happy to help enable CMake for Linux on ASF CI
 > >>> . If you or Alex
 > >>> can
 > >>> update
 > >>> "support/docker_build.sh" and send a review, I'll merge it
 > >>> upstream.
 > >>>
 > >>
 > >>
 > >
 >



 --
 Best Regards,
 Haosdent Huang
>>>
>>>
>>
>



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Build Mesos on Windows

[Alex Clemmer]

Hey folks. Microsoft is putting on their annual developer conference,
"//build", next week, so I don't have time to respond in detail, but I
did want to address the immediate questions of the .patch file and
contributing.

For contributing at the state of the art, the two most valuable things
you can do are (1) help with code reviews, or (2) help change the CI
system to build and test from CMake on both Windows and Linux. Writing
new code is not the bottleneck at this point -- we have a feature
branch with a working agent, and a finite number of commits (less than
100) that need to be reviewed and checked in. The big obstacles are
that the Windows build is consistently inadvertently broken because
there is no CI integration, and the fact that I personally have to
review every Windows patch that comes through. If you want to help
with these, I am happy to direct you on how to proceed.

To your question about the .patch file, this sounds like a bug. Can
you please tell me the SHA of your head, and also check that the file
has CRLF line endings in your local repository? We are building out of
VisualStudio, by the way, but I don't expect that to make a
difference. If this is a line ending issue we may need to add a
.gitattributes entry to make an exception for the patch files.

On Fri, Mar 25, 2016 at 4:48 PM, Juan Larriba <jlarr...@gmail.com> wrote:
> Thank you Tommy and Benjamin, it would be very nice if someone could update
> with any news.
>
> However, I continued trying to build and, after unload the stout_tests
> project, I am stuck building the Zookeeper dependency.
>
> CMake is trying to patch zookeeper-06d3f3f (using that version as 3.4.5 does
> not compile in Windows) using GnuWin32 patch.exe, but I am receiving the
> traditional error:
>
> Assertion failed: hunk, file ../patch-2.5.9-src/patch.c, line 354
>
> Which means that we are trying to patch a file with Unix line end (LF)
> instead of DOS line end (CR-LF). Normally, this is solved using the --binary
> option during patching, but if I enable it in the CMake configuration, it
> fails to patch the vcxproj file:
>
> patching file src/c/zookeeper.vcxproj
> 2>  Hunk #1 FAILED at 1.
> 2>  Hunk #2 FAILED at 22.
> 2>  Hunk #3 FAILED at 81.
> 2>  Hunk #4 FAILED at 105.
> 2>  Hunk #5 FAILED at 125.
> 2>  Hunk #6 FAILED at 145.
> 2>  6 out of 6 hunks FAILED -- saving rejects to file
> src/c/zookeeper.vcxproj.rej
> 2>  patching file src/c/zookeeper-vs2015.sln
> 2>  patching file src/c/include/winconfig.h
> 2>  patching file src/c/include/winstdint.h
>
> Any kind of advice would be nice.
>
> Thank you very much.
>
> On 25 March 2016 at 23:36, Benjamin Mahler <bmah...@apache.org> wrote:
>>
>> + Alex Clemmer
>>
>> On Thu, Mar 24, 2016 at 7:50 PM, tommy xiao <xia...@gmail.com> wrote:
>>
>> > i remember microsoft team invovled in this case, but not found any
>> > update
>> > on it? anyone can update news?
>> >
>> > 2016-03-25 6:08 GMT+08:00 Juan Larriba <jlarr...@gmail.com>:
>> >
>> > > Hi all,
>> > >
>> > > I have been working with Mesos for the past months and I am very
>> > interested
>> > > in its possibilities and the capabilities it offers.
>> > >
>> > > Until now, I have been working with Mesos on Linux, and it works as a
>> > > charm, but I was interested in joining Windows based systems to the
>> > > clusters, and the possibilities it offer.
>> > >
>> > > When I searched for that possibility, I discovered that a functional
>> > > demo
>> > > had been done in August 2015 (
>> > >
>> > >
>> >
>> > https://mesosphere.com/blog/2015/08/20/mesos-everywhere-apache-mesos-for-windows-server/
>> > > )
>> > > and I reached a question in StackOverflow (
>> > >
>> > >
>> >
>> > http://stackoverflow.com/questions/33700919/roadmap-for-apache-mesos-on-windows
>> > > )
>> > > where they point out the Jira tickets that are being used to monitor
>> > > the
>> > > development progress.
>> > >
>> > > The StackOverflow answer even goes further and says that as of
>> > > November
>> > > 2015, the CMake-based build was pretty functional, even generating an
>> > agent
>> > > binary.
>> > >
>> > > So I downloaded the source and went into building Mesos on Windows
>> > > using
>> > > Visual Studio 2015, but I ended with some compilation errors and no
>> > binary
>> > > agent being generat

Re: Discussion about upgrading 3rdparty libraries

[Alex Clemmer]

So, at this point we have a bunch of different reviews open for
this[1], and I'd like to use this as an opportunity to start nudging
people towards thinking about possibly transitioning to a scheme where
the tarballs that are currently held in the `3rdparty/` directory are
moved to some external place, and retrieved for users out-of-band, as
necessary to build Mesos.

In particular, doing this (as you all likely know) is very expensive
because git stores a complete copy of the entire tarball, for each
different revision in history, so if you have updated a tarball twice,
you have two complete copies rolling around in the `.git/` folder. It
seems like there are not many benefits for keeping this scheme, other
than the fact that it's pretty easy to implement.

I'm not sure what it would take to transition the autotools build
system, but just recapping earlier what I said about the CMake build
system: The easiest thing to do (which we've already mostly done) is
to allow people to rope in tarballs from some mirror of the `3rdparty`
github repository[2]. Right now we have facilities that let you host
it either on your local FS or on a remote URL, and we'll download (if
necessary) and untar into the familliar place in the `build/` folder.
Easy! We could even have `bootstrap` clone the repository and make
CMake automatically pull in that repository if it's out of date.

Thoughts? I recognize that this might be overcomplicating the problem
a bit, but I figured I'd throw the hat in the ring because this has
always kind of bothered me. :)


[1] They are:
https://reviews.apache.org/r/44252/
https://reviews.apache.org/r/44382/
https://reviews.apache.org/r/44372/
https://reviews.apache.org/r/44378/
https://reviews.apache.org/r/44376/
https://reviews.apache.org/r/44257/

[2] https://github.com/3rdparty/mesos-3rdparty

On Tue, Mar 1, 2016 at 10:48 AM, Alex Clemmer
<clemmer.alexan...@gmail.com> wrote:
> It doesn't seem to be the case that these things are mutually
> exclusive -- it is well within our purview to accept only a specific
> range of versions for any particular dependency, and error out if
> someone tries to select a version outside that range. The only thing
> these commits add is more fine-grained control over which of the
> supported versions you are allowed to select.
>
> At this point, there are no such guards, but that is certainly
> something that can be added.
>
> On Tue, Mar 1, 2016 at 10:17 AM, Neil Conway <neil.con...@gmail.com> wrote:
>> The prospect of downloading dependencies from "rando" locations is
>> concerning to me :)
>>
>> Mesos can easily come to depend on implementation details of a
>> dependency that might change in a minor release. For example, a recent
>> change [1] depends on the connection retry logic in the Zk client
>> library in a fairly delicate way. I also wouldn't want users to
>> randomly upgrade to, say, protobuf 2.6.1 without it being thoroughly
>> tested. Increasing the support matrix of different users on different
>> platforms running arbitrarily different versions of third-party
>> dependencies doesn't seem like a net improvement to me.
>>
>> My two cents: if Windows requires additional dependencies that we
>> aren't currently vendoring, I would personally opt for (a) vendoring
>> those additional dependencies (b) ensuring that the vendored versions
>> we ship are modern enough to support all the platforms we care about.
>> Are there important use-cases that aren't supported by this scheme?
>>
>> Neil
>>
>> [1] 
>> https://github.com/apache/mesos/commit/c2d496ed430eaf7daee3e57edefa39c25af2aa43
>>
>> On Tue, Mar 1, 2016 at 10:00 AM, Alex Clemmer
>> <clemmer.alexan...@gmail.com> wrote:
>>> I guess a tl;dr might be in order.
>>>
>>> Basically: the CMake build system already supports roping in tarballs
>>> from rando places on the filesystem or Internet, so I think it makes
>>> sense to rope them in at configure time, and so I'm proposing we
>>> re-appropriate the sophisticated tools we already have to do this for
>>> WIndows, into a more general solution that is useful to other exotic
>>> platforms, rather than just Windows.
>>>
>>> As always, super interested to hear feedback, I'd love to know if I
>>> missed something.
>>>
>>> On Tue, Mar 1, 2016 at 9:58 AM, Alex Clemmer
>>> <clemmer.alexan...@gmail.com> wrote:
>>>> This is a great time to discuss the Mesos dependency channel story in
>>>> general, because it has had to evolve a bit to fit the requirements of
>>>> Windows, and some of the issues you describe are issues we had to
>>>> resolve (at least partially) to support the 

Re: Making 'curl' a prerequisite for installing Mesos

[Alex Clemmer]

Looks like the relevant review is this one:
https://reviews.apache.org/r/40418/diff/3#4

I _suspect_ this will work with Windows, but am not positive.
Optimistically, it's not clear to me whether it makes sense to add it
as a dependency, because I don't know how to get its location reliably
on Windows. Because Windows has no package manager, we actually rope
it in the libcurl dependency from CMake, at build time. Seems like the
thing to do might be to just build the exe as well and dispatch to
that but this will require some modifications to this code.

On Thu, Mar 3, 2016 at 5:46 PM, Guangya Liu  wrote:
> libcurl can automatically picks up certain environment variables and
> adjusts its settings accordingly, so libcurl support enabling http_proxy
> and https_proxy by default, this is important feature for someone who want
> to use a proxy to connect internet. One example is that I cannot get google
> docker images but need a proxy set in China.
>
> If we depend on "curl" (I saw that we already finished the this in
> MESOS-2840) when using fetcher, I think that we may also need to enable
> slave to pass a proxy to fetch curl to enable someone can pull google
> docker images under a firewall. Does it make sense file a JIRA to support
> http proxy?
>
> Thanks,
>
> Guangya
>
> On Fri, Mar 4, 2016 at 9:39 AM, Klaus Ma  wrote:
>
>> +1 to add 'curl' dependency firstly.
>>
>> 
>> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
>> Platform OpenSource Technology, STG, IBM GCG
>> +86-10-8245 4084 | klaus1982...@gmail.com | http://k82.me
>>
>> On Fri, Mar 4, 2016 at 5:04 AM, Jojy Varghese  wrote:
>>
>> > +1
>> >
>> > On Thu, Mar 3, 2016 at 12:52 PM Jake Farrell 
>> wrote:
>> >
>> > > +1
>> > >
>> > > -Jake
>> > >
>> > > On Thu, Mar 3, 2016 at 12:10 PM, Jie Yu  wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > I am proposing making 'curl' a prerequisite when installing Mesos.
>> > > > Currently, we require 'libcurl' being present when installing Mesos (
>> > > > http://mesos.apache.org/gettingstarted/). However, we found that it
>> > does
>> > > > not compose well with our asynchronous runtime environment (i.e.,
>> it'll
>> > > > block the current worker thread).
>> > > >
>> > > > Recent work on URI fetcher
>> > > >  uses 'curl'
>> > directly,
>> > > > instead of using 'libcurl' to fetch artifacts, because it composes
>> well
>> > > > with our async runtime env. 'curl' is installed by default in most
>> > > systems
>> > > > (e.g., OSX, centos, RHEL).
>> > > >
>> > > > So I am proposing adding 'curl' to our prerequisite list. Let me know
>> > if
>> > > > you have any concern on this. I'll update the Getting Started doc if
>> > you
>> > > > are OK with this change.
>> > > >
>> > > > Thanks,
>> > > > - Jie
>> > > >
>> > >
>> >
>>



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Discussion about upgrading 3rdparty libraries

[Alex Clemmer]

I guess a tl;dr might be in order.

Basically: the CMake build system already supports roping in tarballs
from rando places on the filesystem or Internet, so I think it makes
sense to rope them in at configure time, and so I'm proposing we
re-appropriate the sophisticated tools we already have to do this for
WIndows, into a more general solution that is useful to other exotic
platforms, rather than just Windows.

As always, super interested to hear feedback, I'd love to know if I
missed something.

On Tue, Mar 1, 2016 at 9:58 AM, Alex Clemmer
<clemmer.alexan...@gmail.com> wrote:
> This is a great time to discuss the Mesos dependency channel story in
> general, because it has had to evolve a bit to fit the requirements of
> Windows, and some of the issues you describe are issues we had to
> resolve (at least partially) to support the Windows integration work.
>
> More particularly, our problems are: first, Windows frequently
> requires newer versions of dependencies (due to poor support of MSVC
> 1900), so we have had to develop reasonably robust version-selection
> mechanisms, so that Windows can get specific versions of different
> packages. This means that the Mesos project does not have to evolve
> the dependency support story in lock step, which in the long term may
> actually be required, as some platforms (e.g., those run by
> governmental organizations) are more conservative about what
> dependencies are introduced on their clusters.
>
> Second, because Windows does not have a package manager, it became
> necessary for the CMake build system to support actually hitting some
> remote (possiblty the internet) to rope in the tarballs of arbitrary
> (and arbitrarily-versioned) dependencies that we normally expect to
> already be installed (such as APR or cURL).
>
> This last point is actually more convenient than it seems. Our CMake
> implementation recently[1][2] introduced a flag that lets you specify
> something like `cmake .. -D3RDPARTY_DEPENDENCIES=/some/path/or/url`
> and it will proactively look for tarballs in the location you give it
> -- and that location can be either a path on your filesystem, or a
> URI, like the 3rdparty remote in github[3] that is owned by the GitHub
> community. From the "exotic platform" perspective this is great
> because it makes it trivial for people building (say) Windows to
> upgrade to a version not supported by CMake:
>
> * Put a tarball of a new version somewhere on the filesystem. Say, we
> decide to use glog 0.3.4 instead of 0.3.3, so we just put that tarball
> for 0.3.4 in a well-known place in the filesystem.
> * Update the version of glog in Versions.cmake.
> * When you run cmake, just run `cmake ..
> -D3RDPARTY_DEPENDENCIES=/my/fancy/3rdparty/path`
> * Builds against new dep! Magic!
>
> Much of this was developed out of expediency, but going forward I
> think a reasonable approach to dealing with the third-party channel
> might be (and I would LOVE feedback on this):
>
> WORKFLOW THAT ASSUMES INTERNET ACCESS ON BUILD MACHINE:
> * Clone a copy of mesos.
> * (When we do a normal clone of Mesos, there are no tarballs in the
> `3rdparty/` directory.)
> * Run `bootstrap`.
> * `mkdir build && cd build && cmake ..`. Part of the CMake
> configuration step will be to `git clone` a copy of
> `https://github.com/3rdparty/mesos-3rdparty`. (If you don't know, the
> 3rdparty account is owned by the Mesos community, and the
> `mesos-3rdparty` is where we store canonical copies of all our
> third-party tarballs.)
> * This dumps all the tarballs into a folder, `mesos-3rdparty`.
> * We build against the tarballs we retrieved. Optionally you are
> allowed to set the versions in `Versions.cmake` and mesos will "just
> build" against those versions (as long as they are supported, and we
> will complain if they're not).
> * If you `git pull` and find that Mesos has upgraded its dependencies,
> and a version is out of date, then when you next build, CMake will
> explode automatically (even if you've built before) and ask you to
> `git pull` to update your `mesos-3rdparty` repository.
>
>
> WORKFLOW THAT DOES NOT ASSUME INTERNET ACCESS ON BUILD MACHINE:
> Much like the above, except when you run cmake, you do `cmake ..
> -D3RDPARTY_DEPENDENCIES="path/to/mesos-3rdparty/mirror"`. This will
> tell CMake to not clone the mirror itself, but to look for an existing
> mirror at the location specified.
>
>
> WHAT WE'VE IMPLEMENTED:
> We obviously haven't deleted the tarballs in 3rdparty, and the error
> reporting around `Versions.cmake` and asking people to re-pull when a
> version has been upgraded are not there, but a lot of the rest of this
> is already in place. For example, yesterday we checked in an

Re: Discussion about upgrading 3rdparty libraries

[Alex Clemmer]

This is a great time to discuss the Mesos dependency channel story in
general, because it has had to evolve a bit to fit the requirements of
Windows, and some of the issues you describe are issues we had to
resolve (at least partially) to support the Windows integration work.

More particularly, our problems are: first, Windows frequently
requires newer versions of dependencies (due to poor support of MSVC
1900), so we have had to develop reasonably robust version-selection
mechanisms, so that Windows can get specific versions of different
packages. This means that the Mesos project does not have to evolve
the dependency support story in lock step, which in the long term may
actually be required, as some platforms (e.g., those run by
governmental organizations) are more conservative about what
dependencies are introduced on their clusters.

Second, because Windows does not have a package manager, it became
necessary for the CMake build system to support actually hitting some
remote (possiblty the internet) to rope in the tarballs of arbitrary
(and arbitrarily-versioned) dependencies that we normally expect to
already be installed (such as APR or cURL).

This last point is actually more convenient than it seems. Our CMake
implementation recently[1][2] introduced a flag that lets you specify
something like `cmake .. -D3RDPARTY_DEPENDENCIES=/some/path/or/url`
and it will proactively look for tarballs in the location you give it
-- and that location can be either a path on your filesystem, or a
URI, like the 3rdparty remote in github[3] that is owned by the GitHub
community. From the "exotic platform" perspective this is great
because it makes it trivial for people building (say) Windows to
upgrade to a version not supported by CMake:

* Put a tarball of a new version somewhere on the filesystem. Say, we
decide to use glog 0.3.4 instead of 0.3.3, so we just put that tarball
for 0.3.4 in a well-known place in the filesystem.
* Update the version of glog in Versions.cmake.
* When you run cmake, just run `cmake ..
-D3RDPARTY_DEPENDENCIES=/my/fancy/3rdparty/path`
* Builds against new dep! Magic!

Much of this was developed out of expediency, but going forward I
think a reasonable approach to dealing with the third-party channel
might be (and I would LOVE feedback on this):

WORKFLOW THAT ASSUMES INTERNET ACCESS ON BUILD MACHINE:
* Clone a copy of mesos.
* (When we do a normal clone of Mesos, there are no tarballs in the
`3rdparty/` directory.)
* Run `bootstrap`.
* `mkdir build && cd build && cmake ..`. Part of the CMake
configuration step will be to `git clone` a copy of
`https://github.com/3rdparty/mesos-3rdparty`. (If you don't know, the
3rdparty account is owned by the Mesos community, and the
`mesos-3rdparty` is where we store canonical copies of all our
third-party tarballs.)
* This dumps all the tarballs into a folder, `mesos-3rdparty`.
* We build against the tarballs we retrieved. Optionally you are
allowed to set the versions in `Versions.cmake` and mesos will "just
build" against those versions (as long as they are supported, and we
will complain if they're not).
* If you `git pull` and find that Mesos has upgraded its dependencies,
and a version is out of date, then when you next build, CMake will
explode automatically (even if you've built before) and ask you to
`git pull` to update your `mesos-3rdparty` repository.


WORKFLOW THAT DOES NOT ASSUME INTERNET ACCESS ON BUILD MACHINE:
Much like the above, except when you run cmake, you do `cmake ..
-D3RDPARTY_DEPENDENCIES="path/to/mesos-3rdparty/mirror"`. This will
tell CMake to not clone the mirror itself, but to look for an existing
mirror at the location specified.


WHAT WE'VE IMPLEMENTED:
We obviously haven't deleted the tarballs in 3rdparty, and the error
reporting around `Versions.cmake` and asking people to re-pull when a
version has been upgraded are not there, but a lot of the rest of this
is already in place. For example, yesterday we checked in an
implementation of the `-D3RDPARTY_DEPENDENCIES` flag[1][2], which
allows you to tell CMake to build against third-party dependencies
mirrored either at a local path (e.g.,
`-D3RDPARTY_DEPENDENCIES="/your/path/here"`) or at a remote URI (e.g.,
`-D3RDPARTY_DEPENDENCIES=https://github.com/3rdparty/mesos-3rdparty`).


[1] 
https://github.com/apache/mesos/commit/6306b7d62dd5cbb34fa82636dfbb46cee46d0bf8
[2] 
https://github.com/apache/mesos/commit/3f7501b818662097f41b2d756b2389f6ed9fa5eb
[3] https://github.com/3rdparty/mesos-3rdparty

On Tue, Mar 1, 2016 at 7:56 AM, Kapil Arya  wrote:
>>
>> *3. 3rdparty/libprocess/3rdparty/stout/tests/protobuf_tests.pb.cc/h
>>  files.*
>>
>> Can anyone tell me why hardcode these two files in Mesos repo? I think
>> these two files can be dynamically generated during make check, this will
>> make it not depend on protoc version.
>>
>
> I think it's just due to the nature of the way dependencies are structured
> in 

Re: Reorganize 3rdparty directory

[Alex Clemmer]

Good question. At this point, libmesos should build, but the CMake
build has fallen a bit behind, since I've been mostly concentrating on
the integration work. That said, I do expect a bunch of patches to go
in tomorrow that will re-un-break the libmesos CMake build, and
following that, the patches that unbreak the agent and master should
follow, maybe in the next week, depending on the committer bandwidth.

On Fri, Feb 26, 2016 at 10:07 PM, Marco Massenzio <m.massen...@gmail.com> wrote:
>>
>> Overall, the changes are extremely small. We (apparently) did a pretty
>> good job of making the CMake configuration scripts modular and
>> consumable by arbitrary projects
>
>
> ah, the warm, fuzzy feeling that one experiences when all the hard work of
> abstracting stuff pays off :-)
> well done, Alex!
>
> A quick question for you (as you recall, I have great stakes in - but
> minimal understanding of - cmake) - which targets should be expected to
> work for Cmake?
> I had to add a few (minor) fixes to make CLion fully understand Mesos (it
> still has a few "bogus" errors, but it's by and large, greatly usable - and
> beats Eclipse CDT any day).
>
> Also, please let me know whether there's anything you'd like me to try out
> and report back.
>
> Thanks!
>
> --
> *Marco Massenzio*
> http://codetrips.com
>
> On Fri, Feb 26, 2016 at 11:44 AM, Alex Clemmer <clemmer.alexan...@gmail.com>
> wrote:
>
>> Ah, I see now that I coudl have done better by splitting the patch
>> into two patches -- one where we move everything, and one where we
>> change the cmake files.
>>
>> THe important parts of the patch are:
>>
>> * Adding the contents of `3rdparty/libprocess/3rdparty/CMakeLists.txt`
>> -> `3rdparty/CMakeLists.txt`; makes since, because we're merging the
>> two 3rdparty folders, so we only need one CMakeLists.txt. We just dump
>> the contents unchanged into the other one.
>> * `3rdparty/libprocess/cmake/Process3rdpartyConfigure.cmake` changing some
>> paths
>> * `3rdparty/libprocess/CMakeLists.txt` moving a call to `include`.
>>
>> And that's about it. Everything else is just moving.
>>
>> On Fri, Feb 26, 2016 at 11:36 AM, Alex Clemmer
>> <clemmer.alexan...@gmail.com> wrote:
>> > Folks:
>> >
>> > Took about 30 minutes to put together a prototype of the great
>> > `3rdparty` flattening. Please see the (extremely preliminary)
>> > review[1] or my working branch[2] for a really close approximation of
>> > the number of changes to the CMake build system we'd need to support
>> > this flattening.
>> >
>> > Overall, the changes are extremely small. We (apparently) did a pretty
>> > good job of making the CMake configuration scripts modular and
>> > consumable by arbitrary projects, so the changes are mostly things
>> > like "move the config `include` call over there instead of being over
>> > there" and "change a handful of path variables to reflect the new dir
>> > structure". (I hope, btw, that it doesn't seem arrogant to say this,
>> > but I think it is justified given the relatively minor changes in the
>> > actual CMake code.)
>> >
>> > Feel free to remix/take/throw away any subset of this. I don't need
>> > any credit for it in the final patch if someone marches off in that
>> > direction. :)
>> >
>> > [1] https://reviews.apache.org/r/44099/
>> > [2] https://github.com/hausdorff/mesos/commits/prototype_flattened_3rd
>> >
>> > On Thu, Feb 18, 2016 at 12:22 PM, Kevin Klues <klue...@gmail.com> wrote:
>> >> I am also a fan of git submodules in the long term, but avoiding them
>> >> in the short term.  We should get things organized as we want them
>> >> first, and then start thinking about pulling libprocess/stout out into
>> >> submodules later (while also preserving their history!)
>> >>
>> >> I disagree with moving libprocess and stout up to the same level as
>> >> src/. If we want to make sure they don't bleed into Mesos proper, we
>> >> really should treat them the same as any other 3rdparty code that we
>> >> depend on.  This will become more relevant when/if we move them into
>> >> submodules.
>> >>
>> >> Given all that, the only real challenge with flattening our 3rdparty
>> >> dependencies into a single folder should be the changes we have to
>> >> make to our configure.ac and Makefile.am scripts to know where to look
>> >> for their dependencies now.  In the e

Re: Reorganize 3rdparty directory

[Alex Clemmer]

Folks:

Took about 30 minutes to put together a prototype of the great
`3rdparty` flattening. Please see the (extremely preliminary)
review[1] or my working branch[2] for a really close approximation of
the number of changes to the CMake build system we'd need to support
this flattening.

Overall, the changes are extremely small. We (apparently) did a pretty
good job of making the CMake configuration scripts modular and
consumable by arbitrary projects, so the changes are mostly things
like "move the config `include` call over there instead of being over
there" and "change a handful of path variables to reflect the new dir
structure". (I hope, btw, that it doesn't seem arrogant to say this,
but I think it is justified given the relatively minor changes in the
actual CMake code.)

Feel free to remix/take/throw away any subset of this. I don't need
any credit for it in the final patch if someone marches off in that
direction. :)

[1] https://reviews.apache.org/r/44099/
[2] https://github.com/hausdorff/mesos/commits/prototype_flattened_3rd

On Thu, Feb 18, 2016 at 12:22 PM, Kevin Klues  wrote:
> I am also a fan of git submodules in the long term, but avoiding them
> in the short term.  We should get things organized as we want them
> first, and then start thinking about pulling libprocess/stout out into
> submodules later (while also preserving their history!)
>
> I disagree with moving libprocess and stout up to the same level as
> src/. If we want to make sure they don't bleed into Mesos proper, we
> really should treat them the same as any other 3rdparty code that we
> depend on.  This will become more relevant when/if we move them into
> submodules.
>
> Given all that, the only real challenge with flattening our 3rdparty
> dependencies into a single folder should be the changes we have to
> make to our configure.ac and Makefile.am scripts to know where to look
> for their dependencies now.  In the end these should be URLs to
> versioned tarballs that we host somewhere (or git repos that we can
> have forked and tagged with specific versions).  In the short term
> these can just be relative paths in the mesos tree though.
>
> On Tue, Feb 16, 2016 at 1:26 PM, Kapil Arya  wrote:
>> Thanks for bringing it up Alexander!
>>
>> I don't have a strong opinion wrt git submodules since I don't have
>> much experience with them personally. Having said that, I would like
>> to go conservative on this one (baby steps :-) ).
>>
>> Further, I do understand that moving libprocess and stout directories
>> will be painful for people who already have several branches and will
>> have conflicts. But I do think, there are some interim solutions as
>> well (for example, move libprocess/stout to wherever we want, but keep
>> a symlink from 3rdparty/libprocess, etc, to those new locations for
>> some time). I am sure there are better solutions out there, but this
>> should work too.
>>
>> Best,
>> Kapil
>>
>> On Tue, Feb 16, 2016 at 12:38 PM, Erik Weathers
>>  wrote:
>>> If we go to git submodules, please ensure there are good docs around how to
>>> update cloned repos.
>>>
>>> e.g., From ansible: https://docs.ansible.com/ansible/intro_installation.html
>>>
>>> Note when updating ansible, be sure to not only update the source tree, but
>>> also the “submodules” in git which point at Ansible’s own modules (not the
>>> same kind of modules, alas).
>>>
>>> $ git pull --rebase
>>> $ git submodule update --init --recursive
>>>
>>> Thanks,
>>>
>>> - Erik
>>>
>>> On Tue, Feb 16, 2016 at 8:54 AM, Alexander Rojas 
>>> wrote:
>>>
 +1
 I am one who is totally in for that change. It is not only the directories
 problem, but the structure which has led that the stout tests (which do
 need to be compiled) are actually managed in the libprocess Makefile, on
 top of all the things you have already mentioned.


 > On 09 Feb 2016, at 17:53, Kapil Arya  wrote:
 >
 > On Tue, Feb 9, 2016 at 8:23 PM, Jie Yu  wrote:
 >> Kapil,
 >>
 >> I guess what I want to understand is why the existing structure makes it
 >> hard for you to do the things that you want to do (installing
 >> module-specific 3rdparty dependencies into "${pkglibdir}/3rdparty" as
 part
 >> of "make install").
 >
 > Let me see if I can answer that :-).
 >
 > This is somewhat related. For example, if we want to install protobuf
 > in 3rdparty/{include,lib} (for module developers to use them without
 > doing a proper mesos installation), you need to provide the correct
 > "--prefix" flag that points to 3rdparty/. However, due to multiple
 > levels of configure.ac, the "--prefix" can at best be generated by
 > prepending "../../../" to get to the great-grandparent directory. This
 > is because we have a separate configure.ac which manages
 > 

Re: Anyone successfully setup CLION with mesos?

[Alex Clemmer]

CMake support is not quite mature yet. I think we're at least a couple
months out before it's really ready to rely on, but I'm quite happy to
hear about your bugs! Feel free to file them against me (I'm
`hausdorff` on the JIRA), and I'll make sure they get routed properly.

On Wed, Nov 25, 2015 at 8:22 PM, haosdent  wrote:
> I set up success for it. Because Mesos have cmake support now. Import it as
> a cmake project.
>
> On Thu, Nov 26, 2015 at 12:21 PM, Shiyao Ma  wrote:
>
>> Hi,
>>
>> I'd like to browse the mesos code with abilities such as  jump to
>> definitions, etc.
>>
>> Ctags and Youcompleteme fall short here.
>>
>> So I think CLION might be a good way to go, so anybody managed to do that
>> with CLion?
>>
>> What's the setup ?
>>
>>
>> Thanks.
>>
>>
>> shiyao
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Mesos in a Windows only environment

[Alex Clemmer]

We are still hoping to land things by the end of the year, but it will
depend on the bandwidth the committers have to actually devote time to
this work. They, understandably, are busy with other important
initiatives. :)

On Tue, Nov 24, 2015 at 7:43 AM, tommy xiao <xia...@gmail.com> wrote:
> when can we use the slave on windows?
>
> 2015-11-24 16:49 GMT+08:00 Alex Clemmer <clemmer.alexan...@gmail.com>:
>
>> The short answer is that the Windows integration work we have planned
>> currently is aimed specifically at getting the agent/slave working and
>> production-worthy on Windows. This plan does not specifically aim to
>> add Windows support to the master.
>>
>> That said, there are _provisional_ plans to add Windows support to the
>> master -- but, the specifics about when this work would happen, and
>> who would do it, is not yet decided.
>>
>> I do expect plans around the master to firm up in the first few months
>> of the next year, after a few other things (like the CMake build
>> system, which is required for convenient and ongoing Windows
>> development) become more solid and permanent parts of the code base.
>>
>> On Mon, Nov 23, 2015 at 10:34 PM, Kiran Anantha <kiran.anan...@gmail.com>
>> wrote:
>> > Hello,
>> > I would like to clarify if the Mesos on Windows plans include running the
>> > master on Windows as well. If not, I would like to understand why there
>> is
>> > such a constraint.
>> >
>> > I am interested in using Mesos for deploying a distributed java
>> application
>> > running on JBoss. Many of our smaller installations are at IT shops that
>> > run only Windows. Hence the question/clarification.
>> >
>> > Thanks in advance,
>> > Kiran.
>>
>>
>>
>> --
>> Alex
>>
>> Theory is the first term in the Taylor series of practice. -- Thomas M
>> Cover (1992)
>>
>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Mesos in a Windows only environment

[Alex Clemmer]

The short answer is that the Windows integration work we have planned
currently is aimed specifically at getting the agent/slave working and
production-worthy on Windows. This plan does not specifically aim to
add Windows support to the master.

That said, there are _provisional_ plans to add Windows support to the
master -- but, the specifics about when this work would happen, and
who would do it, is not yet decided.

I do expect plans around the master to firm up in the first few months
of the next year, after a few other things (like the CMake build
system, which is required for convenient and ongoing Windows
development) become more solid and permanent parts of the code base.

On Mon, Nov 23, 2015 at 10:34 PM, Kiran Anantha  wrote:
> Hello,
> I would like to clarify if the Mesos on Windows plans include running the
> master on Windows as well. If not, I would like to understand why there is
> such a constraint.
>
> I am interested in using Mesos for deploying a distributed java application
> running on JBoss. Many of our smaller installations are at IT shops that
> run only Windows. Hence the question/clarification.
>
> Thanks in advance,
> Kiran.



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Better error reporting in Mesos

[Alex Clemmer]

An additional argument for logging a struct and putting all the
to-string logic somewhere else is that it would allow us to
incorporate localization and internationalization relatively
painlessly. If we have to pass around a string, it is much more
painful to do this.

On Tue, Nov 10, 2015 at 12:28 PM, Michael Hopcroft <m...@hopcroft.org> wrote:
> There have been a lot of great suggestions on this thread. My
> recommendation is that I do an experiment to uncover any hidden issues with
> the macro approach outlined in my original email. I will try to design the
> API of the ErrorImpl class so that it would be, not only possible, but easy
> to add support for error chaining and message formatting, as has been
> suggested on this thread. If I don’t encounter any significant problems
> with the basic experiment, I will come back to this group with a patch and
> we can then discuss whether to push the design further forward. I’m a big
> fan of an error infrastructure because it gives us a centralized location
> in the code to add debugging hooks and apply various policies such as
> logging and formatting.
>
> On Fri, Nov 6, 2015 at 1:56 PM, Alex Clemmer <clemmer.alexan...@gmail.com>
> wrote:
>
>> In general, I think a good answer to my question is for the place that
>> is reporting or unboxing the error to aggregate data into an error
>> struct that holds things like the "error stack" (and other semantics
>> about the log, including perhaps error codes), and then let whoever is
>> logging, reporting or printing the message, handle the pretty-printing
>> semantics. This is a departure from our current log-the-error-message
>> approach, where the error message acts as the sole communicator of
>> error semantics and context, including the details about the line
>> number.
>>
>> In fact, I think it's worth wondering whether the place where we
>> create an error should even log the error message -- right now we add
>> a lot of strings together, and it makes more sense to me to have a
>> methodology for just populating an "error struct" with data about the
>> error, and then the `toString` method (or whatever) can simply
>> generate the error messages as necessary, in one holistic place. That
>> way, we have centralized place that determining how an error gets
>> transformed into a string, instead of a lot of ad hoc string
>> concatenations and error unboxing. Another advantage is that you'd
>> have very fine-grained control over how to respond to errors, as it is
>> a typesafe struct. In fact, you could even imagine having a system
>> where error stringifying is handled completely out-of-band, meaning
>> that logging could become much, much cheaper in general.
>>
>> But anyway, more broadly, what I'm trying to say is that the place
>> where we are returning an error should not know or care about how the
>> downstream sites pretty-print it or log it. Those are separate
>> concerns.
>>
>> On Fri, Nov 6, 2015 at 1:17 PM, Benjamin Mahler
>> <benjamin.mah...@gmail.com> wrote:
>> > Thanks for the proposal Michael. This has come up in discussions before
>> and
>> > I think it's really valuable, so it's nice to see folks thinking about
>> > this. Some things to consider:
>> >
>> > Don't forget 'Failure' which is unfortunately not unified with 'Error'.
>> >
>> > How does this affect error message composition (as Alex alluded to)?
>> > Currently we do manual error message composition (":
>> > : <callee's callee message>: ...") but in the future we
>> may
>> > add composition support to our synchronous types like Try,Result,etc to
>> be
>> > similar to Future composition with .then. See:
>> > https://issues.apache.org/jira/browse/MESOS-785
>> >
>> > Given composition, it sounds like we want to decouple the trace
>> information
>> > from the error message, so that we can store a "stack-trace" of multiple
>> > code locations?
>> >
>> > It would be good to be aware of the limitations around Future composition
>> > and failure messages. When a .then chain encounters a failure, no error
>> > message wrapping occurs as the failure bubbles up. The result is that we
>> > instead get "" as the failure message without
>> > the caller wrapping I mentioned above. Tracing will help alleviate only
>> > having the bottom most callee failure, but it would be great to think
>> about
>> > how to produce better messages as well (given MESOS-785 lands us in the
>> > same boat for Try,Result,etc

Re: [VOTE] Release Apache Mesos 0.26.0 (rc1)

[Alex Clemmer]

And, please do the same for the CMake configuration. :)

On Fri, Nov 20, 2015 at 7:40 AM, Benjamin Mahler
 wrote:
> Did you forget to replace 0.26.0 with 0.27.0 in configure.ac? Ideally all
> commits with configure.ac pointing to 0.26.0 land in 0.26.0.
>
> Please move configure.ac to 0.27.0 if you've decided on a cut point :)
>
> On Fri, Nov 13, 2015 at 12:14 PM, Till Toenshoff  wrote:
>>
>> Hi friends,
>>
>> Please vote on releasing the following candidate as Apache Mesos 0.26.0.
>>
>>
>> The CHANGELOG for the release is available at:
>>
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.0-rc1
>>
>> 
>>
>> The candidate for Mesos 0.26.0 release is available at:
>>
>> https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc1/mesos-0.26.0.tar.gz
>>
>> The tag to be voted on is 0.26.0-rc1:
>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.26.0-rc1
>>
>> The MD5 checksum of the tarball can be found at:
>>
>> https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc1/mesos-0.26.0.tar.gz.md5
>>
>> The signature of the tarball can be found at:
>>
>> https://dist.apache.org/repos/dist/dev/mesos/0.26.0-rc1/mesos-0.26.0.tar.gz.asc
>>
>> The PGP key used to sign the release is here:
>> https://dist.apache.org/repos/dist/release/mesos/KEYS
>>
>> The JAR is up in Maven in a staging repository here:
>> https://repository.apache.org/content/repositories/orgapachemesos-1085
>>
>> Please vote on releasing this package as Apache Mesos 0.26.0!
>>
>> The vote is open until Sun Nov 15 20:13:46 CET 2015 and passes if a
>> majority of at least 3 +1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Mesos 0.26.0
>> [ ] -1 Do not release this package because ...
>>
>> Thanks,
>> Bernd & Till
>
>



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: [4/5] mesos git commit: Replaced strerror calls in libprocess with os::strerror.

[Alex Clemmer]

It's worth noting that `os::strerror` does break the Windows build. I
have a fix here: https://reviews.apache.org/r/40382/ although it's in
my review chain, it actually doesn't have any dependencies.

On Thu, Nov 19, 2015 at 12:18 PM, James Peach  wrote:
> This commit reverts MESOS-3708. Can we please restore the previous ABORT 
> message?
>
>
>> On Nov 13, 2015, at 9:11 AM, bmah...@apache.org wrote:
>>
>> Replaced strerror calls in libprocess with os::strerror.
>>
>> Review: https://reviews.apache.org/r/39007
>>
>>
>> Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
>> Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/ba3be3f3
>> Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/ba3be3f3
>> Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/ba3be3f3
>>
>> Branch: refs/heads/master
>> Commit: ba3be3f37e6574c1a9882429fdd0939fd6406c8d
>> Parents: 2fe9f6b
>> Author: Benjamin Bannier 
>> Authored: Fri Nov 13 17:29:12 2015 +0100
>> Committer: Benjamin Mahler 
>> Committed: Fri Nov 13 17:54:19 2015 +0100
>>
>> --
>> 3rdparty/libprocess/src/io.cpp  | 13 +++--
>> 3rdparty/libprocess/src/poll_socket.cpp |  7 ---
>> 3rdparty/libprocess/src/process.cpp |  5 +++--
>> 3rdparty/libprocess/src/profiler.cpp|  9 +
>> 3rdparty/libprocess/src/subprocess.cpp  |  3 ++-
>> 5 files changed, 21 insertions(+), 16 deletions(-)
>> --
>>
>>
>> http://git-wip-us.apache.org/repos/asf/mesos/blob/ba3be3f3/3rdparty/libprocess/src/io.cpp
>> --
>> diff --git a/3rdparty/libprocess/src/io.cpp b/3rdparty/libprocess/src/io.cpp
>> index 26686e1..1730aaa 100644
>> --- a/3rdparty/libprocess/src/io.cpp
>> +++ b/3rdparty/libprocess/src/io.cpp
>> @@ -24,6 +24,7 @@
>> #include 
>> #include 
>> #include 
>> +#include 
>> #include 
>>
>> using std::string;
>> @@ -92,7 +93,7 @@ void read(
>>  WeakFuture(future)));
>>   } else {
>> // Error occurred.
>> -promise->fail(strerror(errno));
>> +promise->fail(os::strerror(errno));
>>   }
>> } else {
>>   promise->set(length);
>> @@ -183,7 +184,7 @@ void write(
>>  WeakFuture(future)));
>>   } else {
>> // Error occurred.
>> -promise->fail(strerror(errno));
>> +promise->fail(os::strerror(errno));
>>   }
>> } else {
>>   // TODO(benh): Retry if 'length' is 0?
>> @@ -276,7 +277,7 @@ Future peek(int fd, void* data, size_t size, 
>> size_t limit)
>>   // also make sure it's non-blocking and will close-on-exec. Start by
>>   // checking we've got a "valid" file descriptor before dup'ing.
>>   if (fd < 0) {
>> -return Failure(strerror(EBADF));
>> +return Failure(os::strerror(EBADF));
>>   }
>>
>>   fd = dup(fd);
>> @@ -429,7 +430,7 @@ Future read(int fd)
>>   // also make sure it's non-blocking and will close-on-exec. Start by
>>   // checking we've got a "valid" file descriptor before dup'ing.
>>   if (fd < 0) {
>> -return Failure(strerror(EBADF));
>> +return Failure(os::strerror(EBADF));
>>   }
>>
>>   fd = dup(fd);
>> @@ -475,7 +476,7 @@ Future write(int fd, const std::string& data)
>>   // also make sure it's non-blocking and will close-on-exec. Start by
>>   // checking we've got a "valid" file descriptor before dup'ing.
>>   if (fd < 0) {
>> -return Failure(strerror(EBADF));
>> +return Failure(os::strerror(EBADF));
>>   }
>>
>>   fd = dup(fd);
>> @@ -510,7 +511,7 @@ Future redirect(int from, Option to, 
>> size_t chunk)
>> {
>>   // Make sure we've got "valid" file descriptors.
>>   if (from < 0 || (to.isSome() && to.get() < 0)) {
>> -return Failure(strerror(EBADF));
>> +return Failure(os::strerror(EBADF));
>>   }
>>
>>   if (to.isNone()) {
>>
>> http://git-wip-us.apache.org/repos/asf/mesos/blob/ba3be3f3/3rdparty/libprocess/src/poll_socket.cpp
>> --
>> diff --git a/3rdparty/libprocess/src/poll_socket.cpp 
>> b/3rdparty/libprocess/src/poll_socket.cpp
>> index 85cd864..1daeb03 100644
>> --- a/3rdparty/libprocess/src/poll_socket.cpp
>> +++ b/3rdparty/libprocess/src/poll_socket.cpp
>> @@ -19,6 +19,7 @@
>> #include 
>>
>> #include 
>> +#include 
>>
>> #include "config.hpp"
>> #include "poll_socket.hpp"
>> @@ -72,7 +73,7 @@ Future accept(int fd)
>>   // Turn off Nagle (TCP_NODELAY) so pipelined requests don't wait.
>>   int on = 1;
>>   if (setsockopt(s, SOL_TCP, TCP_NODELAY, , sizeof(on)) < 0) {
>> -const char* error = strerror(errno);
>> +const string error = os::strerror(errno);
>> VLOG(1) << "Failed to turn off the Nagle algorithm: " << error;
>> os::close(s);
>> return Failure(
>> @@ -159,7 +160,7 @@ Future socket_send_data(int 

Re: Website update frequency

[Alex Clemmer]

Just a data point, I always apply reviews, not only to compile, but
also just because I like using my existing toolset to interact with
the code and understand it.

On Thu, Nov 12, 2015 at 12:59 AM, Artem Harutyunyan  wrote:
> Interesting, I always thought that people apply, compile and try out
> patches locally before giving them a 'Ship It'.
>
> Regarding pushing the code, I don't have enough of context, so could you
> please create a JIRA if you still think that it'd be great to have that
> feature added to the script?
>
> On Wed, Nov 11, 2015 at 6:18 PM, Vinod Kone  wrote:
>
>> AFAIK, apply-reviews is used mainly by committers. Maybe some one can use
>> to locally test a review chain of somebody else, but that is pretty rare.
>>
>> Regarding having apply-reviews to automatically push commits to the repo,
>> we need to figure out the credentials delegation aspect. Ideally, it would
>> be run by CI.
>>
>> On Tue, Nov 10, 2015 at 11:10 PM, Jonathon Rossi 
>> wrote:
>>
>> > We've also still got this one open that I intended would take us right
>> > through to the automatic site build:
>> >
>> > https://issues.apache.org/jira/browse/MESOS-3687
>> >
>> > On Wed, Nov 11, 2015 at 5:00 PM, Artem Harutyunyan 
>> > wrote:
>> >
>> > > Hey Vinod,
>> > >
>> > > Here is the JIRA https://issues.apache.org/jira/browse/MESOS-3883.
>> > >
>> > > I don't mean to hijack this thread, and I apologize for the off-topic,
>> > but
>> > > do you think it would make sense to have another script for committers
>> > that
>> > > will use apply-reviews.py? At some point you also mentioned that it
>> would
>> > > be good to have apply-reviews.py do the actual commits but we did not
>> > > pursue the idea because it was not a relevant feature for most of the
>> > > folks.
>> > >
>> > > Cheers,
>> > > Artem.
>> > >
>> > >
>> > > On Tue, Nov 10, 2015 at 7:30 PM, Vinod Kone 
>> wrote:
>> > >
>> > > > That said, this can be automated as a step in apply-reviews script.
>> For
>> > > > example, the script can check if something in site/ (or docs/ ?) is
>> > being
>> > > > committed and if yes, also do an svn update. @artem do you want to
>> take
>> > > > this on as you revamp the apply-reviews script?
>> > > >
>> > > > On Tue, Nov 10, 2015 at 1:23 AM, Adam Bordelon 
>> > > wrote:
>> > > >
>> > > > > Since it's still a manual process, the website is usually only
>> > updated
>> > > a)
>> > > > > when we have a new release to announce, or b) when some other
>> > > blog-worthy
>> > > > > content arises (e.g. MesosCon).
>> > > > >
>> > > > > On Tue, Nov 10, 2015 at 1:06 AM, Jonathon Rossi <
>> j...@jonorossi.com>
>> > > > > wrote:
>> > > > >
>> > > > > > It is currently a manual process performed by a committer,
>> however
>> > > > there
>> > > > > > are plans to make it automated. See this thread for the recent
>> > > > > discussion:
>> > > > > >
>> > > > > > http://www.mail-archive.com/dev@mesos.apache.org/msg33541.html
>> > > > > >
>> > > > > > On Tue, Nov 10, 2015 at 6:56 PM, Neil Conway <
>> > neil.con...@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Does anyone know how frequently the docs at mesos.apache.org
>> are
>> > > > > > > updated? I notice that some docs changes from > 1 week ago
>> aren't
>> > > > > > > reflected on the current site.
>> > > > > > >
>> > > > > > > Neil
>> > > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > Jono
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Jono
>> >
>>



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Add JIRA ticket# to `TODO`s in comments

[Alex Clemmer]

I like this proposal a lot, as I often end up making a point to
mention the MESOS- number in the comment anyway. I would rather
have the format `TODO(MESOS-XXX)` though, because (1) the JIRA should
capture the reporter as well as the assignee, and (2) it's not
immediately clear from the structure that the name should be the
reporter and not, say, the assignee.

On Sat, Nov 7, 2015 at 8:50 PM, Kapil Arya  wrote:
> Folks,
>
> I wanted to bring up a style issue related to the TODO tag in comments. I
> have filed a Jira ticket (https://issues.apache.org/jira/browse/MESOS-3850)
> with the following description:
>
> Currently, we have a TODO() tags to note stuff
> has "should be"/"has to be" done in future. While this provides us with
> some notion of accounting, it's not enough.
>
> The author listed in the TODO comment should be considered the "Reporter",
> but not necessarily the "Assignee". Further, since the stuff "should
> be"/"has to be" done, why not have a Jira issue tracking it?
>
> We can use TODO(MESOS-XXX) or TODO(:MESOS-XXX) or something
> similar. Finally, we might wan to consider adding this to the style guide
> to make it a soft/hard requirement.
>
>
> Are there any opinions/suggestions on this one?
>
> Best,
> Kapil



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: [jira] [Updated] (MESOS-3645) Implement stout/os/windows/stat.hpp

[Alex Clemmer]

Ah. I actually had no idea. Thanks!

On Fri, Nov 6, 2015 at 9:12 AM, Vinod Kone <vinodk...@gmail.com> wrote:
> Hi Alex,
>
> When you want to bulk edit issues (for example unset target version), there
> is an option in JIRA to "bulk edit and not send an email notification". I
> would recommend to use that option (in the future) to reduce the noise on
> the mailing list.
>
> Thanks,
>
> On Thu, Nov 5, 2015 at 10:16 PM, Alex Clemmer (JIRA) <j...@apache.org>
> wrote:
>>
>>
>>  [
>> https://issues.apache.org/jira/browse/MESOS-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>> ]
>>
>> Alex Clemmer updated MESOS-3645:
>> 
>> Target Version/s:   (was: 0.26.0)
>>
>> > Implement stout/os/windows/stat.hpp
>> > ---
>> >
>> > Key: MESOS-3645
>> > URL: https://issues.apache.org/jira/browse/MESOS-3645
>> > Project: Mesos
>> >  Issue Type: Task
>> >  Components: stout
>> >Reporter: Alex Clemmer
>> >Assignee: Alex Clemmer
>> >  Labels: mesosphere, windows
>> >
>>
>>
>>
>>
>> --
>> This message was sent by Atlassian JIRA
>> (v6.3.4#6332)
>
>



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Better error reporting in Mesos

[Alex Clemmer]

In general, I think a good answer to my question is for the place that
is reporting or unboxing the error to aggregate data into an error
struct that holds things like the "error stack" (and other semantics
about the log, including perhaps error codes), and then let whoever is
logging, reporting or printing the message, handle the pretty-printing
semantics. This is a departure from our current log-the-error-message
approach, where the error message acts as the sole communicator of
error semantics and context, including the details about the line
number.

In fact, I think it's worth wondering whether the place where we
create an error should even log the error message -- right now we add
a lot of strings together, and it makes more sense to me to have a
methodology for just populating an "error struct" with data about the
error, and then the `toString` method (or whatever) can simply
generate the error messages as necessary, in one holistic place. That
way, we have centralized place that determining how an error gets
transformed into a string, instead of a lot of ad hoc string
concatenations and error unboxing. Another advantage is that you'd
have very fine-grained control over how to respond to errors, as it is
a typesafe struct. In fact, you could even imagine having a system
where error stringifying is handled completely out-of-band, meaning
that logging could become much, much cheaper in general.

But anyway, more broadly, what I'm trying to say is that the place
where we are returning an error should not know or care about how the
downstream sites pretty-print it or log it. Those are separate
concerns.

On Fri, Nov 6, 2015 at 1:17 PM, Benjamin Mahler
<benjamin.mah...@gmail.com> wrote:
> Thanks for the proposal Michael. This has come up in discussions before and
> I think it's really valuable, so it's nice to see folks thinking about
> this. Some things to consider:
>
> Don't forget 'Failure' which is unfortunately not unified with 'Error'.
>
> How does this affect error message composition (as Alex alluded to)?
> Currently we do manual error message composition (":
> : <callee's callee message>: ...") but in the future we may
> add composition support to our synchronous types like Try,Result,etc to be
> similar to Future composition with .then. See:
> https://issues.apache.org/jira/browse/MESOS-785
>
> Given composition, it sounds like we want to decouple the trace information
> from the error message, so that we can store a "stack-trace" of multiple
> code locations?
>
> It would be good to be aware of the limitations around Future composition
> and failure messages. When a .then chain encounters a failure, no error
> message wrapping occurs as the failure bubbles up. The result is that we
> instead get "" as the failure message without
> the caller wrapping I mentioned above. Tracing will help alleviate only
> having the bottom most callee failure, but it would be great to think about
> how to produce better messages as well (given MESOS-785 lands us in the
> same boat for Try,Result,etc).
>
> Typed errors/failures has come up before as well, but might be orthogonal
> enough to discuss separately.
>
> On Fri, Nov 6, 2015 at 1:16 AM, Alex Clemmer <clemmer.alexan...@gmail.com>
> wrote:
>
>> I'm strongly in favor of moving down this path, or a path like it.
>>
>> As Mesos becomes increasingly important as core distributed systems
>> infrastructure, it will become correspondingly helpful and important
>> to have things like excellent core error reporting facilities, even if
>> they come at the cost of some technical risk in the short term. In
>> this case in particular, I believe this error reporting model, where
>> the developer need only fill in semantic data about the error (e.g.,
>> the error message), and the boilerplate context of the error (e.g.,
>> the line number of the error) is filled in automatically and opaquely
>> by the system, is clearly the "right" model. On the merits of simply
>> adding line numbers itself, I think this is worth prototyping, and if
>> anything, I'd ask for the model to go further -- I've worked on scale
>> systems where errors are all "stringly typed" and the call sites are
>> all tribal knowledge, and I've worked on systems where the errors are
>> highly semantically regimented, and it is extremely clear to me that
>> the latter is clearly, unequivocally better.
>>
>> It's true there are some subtleties to think through. We'll need to
>> think about things like ABI compatibility, and we'll need to think
>> about whether there are situations where we _don't_ want line numbers
>> to be reported (e.g., we often "unbox" the error message and repack

Re: Proposal: move towards #pragma and away from #include guards

[Alex Clemmer]

Just because I apparently was not very clear: in the "history" and
"summarized objections" sections of my original mail, I did attempt to
capture all of the objections from this thread and previous
discussions I found (and, just for the sake of thoroughness, I do
actually explicitly cite this thread in the footnotes). If you don't
want to dig through the thread yourself, I hope that this will give
you a good approximation of what happened in that thread.

I see now that I did fail to mention that this thread ended in
nonconsensus rather than an explicit decision against. Sorry about
that. You can also see evidence that the decision was nonconsensus
directly in the JIRA issues and the review I cite -- for example on
JIRA, Alex R mentions removing the newbie tag from the issue because
there is nonconsensus, and the review was discarded, with the cited
reason being nonconsensus.

Hopefully this clarifies these issues.

On Thu, Nov 5, 2015 at 8:32 AM, Alexander Rojas <alexan...@mesosphere.io> wrote:
> While I’m all in on the proposal, we did have this discussion almost a year 
> ago [1] at the end I think, the decision was not to use them, though I don’t 
> remember the exact reason. In any case, I vote +1 non-binding.
>
> [1] 
> http://mail-archives.apache.org/mod_mbox/mesos-dev/201501.mbox/%3CCA+fJHLjzxPTk_Ry7-KSj1B-02Rp8Jt4ZUkkM2fH6uhbwR=i...@mail.gmail.com%3E
>  
> <http://mail-archives.apache.org/mod_mbox/mesos-dev/201501.mbox/%3CCA+fJHLjzxPTk_Ry7-KSj1B-02Rp8Jt4ZUkkM2fH6uhbwR=i...@mail.gmail.com%3E>
>
>
>> On 05 Nov 2015, at 06:36, Alex Clemmer <clemmer.alexan...@gmail.com> wrote:
>>
>> Hey folks.
>>
>> In r/39803[1], Mike Hopcroft (in quintessential MSFT style, heh)
>> brought up the issue of moving away from #include guards and towards
>> `#pragma once`.
>>
>> As this has been brought up before, I will be brief: we think it's
>> revisiting because the primary objection in previous threads appears
>> to be that, though `#pragma once` is a cleaner solution to the
>> multiple-include problem, it's not so much better that it's worth the
>> code churn. However, the ongoing Windows integration work means we
>> have to touch these files anyway, so if we agree this is cleaner and
>> desirable, then this is an opportunity to obtain that additional code
>> clarity, without the cost of the churn.
>>
>> For the remainder of the email, I will summarize the history of our
>> discussion of this issue, who will do the work, and what the next
>> steps are.
>>
>> PROPOSAL: We propose that all new code use `#pragma once` instead of
>> #include guards; for existing files, we propose that you change
>> #include guards when you touch them.
>>
>> HISTORY: This has been discussed before, most recently a year ago on
>> the mailing list[2]. There is a relevant JIRA[3] and discarded
>> review[4] that changes style guide's recommendation on the matter.
>>
>> SUMMARIZED OBJECTIONS:
>> 1. The Google style guide explicitly forbids `#pragma once`.
>> 2. This results in a lot of code churn, but is only marginally better.
>> 3. It's not C++ standardized/it's platform dependent/IBM's compiler
>> doesn't support it.
>> 4. Popular projects like Chrome don't do `#pragma once` because of
>> history clutter.
>> 5. Intermediate state of inconsistency as we transition to `#pragma
>> once` from #include guards.
>>
>> OUR RESPONSE:
>> Objections (1), (2), and (4) are essentially the same -- Dominic Hamon
>> points out in a previous thread that the Google style guide was
>> canonized when `#pragma once` was Windows-only, and the guidance has
>> not changed since because of the history churn problem. As noted
>> above, we think the history churn problem is minimized by the fact
>> that it can be wrapped up into the Windows integration work.
>>
>> For objection (3), the consensus seems to be that the vast majority of
>> compilers we care about (in particular, the ones supporting C++ 11) do
>> support it.
>>
>> For objection (5) we believe the inconsistent state is likely to not
>> be long lived, as long as we commit to wrapping this work up into the
>> Windows integration work.
>>
>> SUMMARIZED ADVANTAGES:
>> * Basically fool-proof. Communicates simply what its function is (you
>> include this file once). Semantically it is "the right tool for the
>> job".
>> * No need for namespacing conventions for #include guards.
>> * No conflicts with reserved identifiers[5].
>> * No internal conflicts between include guards in Stout, Process
>> library, and Mesos (this is one reason we need the namespacing
&g

Re: Towards Mesos release 0.26.0

[Alex Clemmer]

Just for closure, it looks like a stunning 78% of these are due to me
(they are Windows integration-related). Since probably all of these
are not actually blocking 0.26, I will un/re-tag as appropriate today.

Also, I do apologize for making the issue graph a hockey stick.

On Thu, Nov 5, 2015 at 9:02 AM, Till Toenshoff  wrote:
> As announced, we will be cutting the next release tomorrow, 6th of November.
>
> We currently have 130 blockers (as shown at
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12327111).
>
> If you have any blocking issues assigned to yourself or if you are
> shepherding any of those, please make sure you update the target version
> unless you *really* want to block the 0.26 release. A quick fly-over seemed
> to suggest that only one of those actually should block 0.26; MESOS-2864
> which will likely get solved today / tomorrow (thanks Vinod!).
>
> Please answer this mail if you think your issues should be blocking the
> release, otherwise we will go ahead and move all those 129 issues to the
> next release.
>
> Thanks for your help!
> Till
>
> On Oct 19, 2015, at 3:44 PM, Bernd Mathiske  wrote:
>
> Dear Mesos fans,
>
> this is the tracking ticket for the upcoming next Mesos release.
>
> https://issues.apache.org/jira/browse/MESOS-3758
>
> Please read the ticket’s description for further instructions and tips on
> how to get features and other improvements included.
>
> The release managers will be Till and Bernd. If you have any questions,
> don’t hesitate to post them here or contact us directly:
>
> {till, bernd}mesosphere.io
>
> We aim at making the first cut with a tag around November 6 and at
> finalizing the release before November 17.
>
> Best greetings,
>
> Till and Bernd
>
>



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: New Cloud Foundry Framework - Mesos & CloudFoundry Integration

[Alex Clemmer]

The contribution documentation for the Mesos codebase is here: 
http://mesos.apache.org/documentation/latest/submitting-a-patch/
That said, it is not clear (to me at least) what you mean. It sounds like 
you've written a framework. Is that right? If so, the it seems like it might 
not fit in the scope of the core Mesos code base.

Sent from Outlook

_
From: Deepak Vij (A) 
Sent: Thursday, November 5, 2015 12:04
Subject: New Cloud Foundry Framework - Mesos & CloudFoundry Integration
To:  


Hi folks, I just joined the Apache Mesos mailing list. I wanted to reach out to 
the community regarding the new CloudFoundry/Mesos framework we have developed 
in our Santa Clara Lab. We would like to contribute this work to the Mesos open 
source community. I am pretty sure that this contribution would be quite 
valuable for the community. Please let me know the process for contributing our 
work to the community. Thanks.

Regards,
Deepak K. Vij
(Telecom Software Strategist, Huawei Software Lab, Santa Clara)

Proposal: move towards #pragma and away from #include guards

[Alex Clemmer]

Hey folks.

In r/39803[1], Mike Hopcroft (in quintessential MSFT style, heh)
brought up the issue of moving away from #include guards and towards
`#pragma once`.

As this has been brought up before, I will be brief: we think it's
revisiting because the primary objection in previous threads appears
to be that, though `#pragma once` is a cleaner solution to the
multiple-include problem, it's not so much better that it's worth the
code churn. However, the ongoing Windows integration work means we
have to touch these files anyway, so if we agree this is cleaner and
desirable, then this is an opportunity to obtain that additional code
clarity, without the cost of the churn.

For the remainder of the email, I will summarize the history of our
discussion of this issue, who will do the work, and what the next
steps are.

PROPOSAL: We propose that all new code use `#pragma once` instead of
#include guards; for existing files, we propose that you change
#include guards when you touch them.

HISTORY: This has been discussed before, most recently a year ago on
the mailing list[2]. There is a relevant JIRA[3] and discarded
review[4] that changes style guide's recommendation on the matter.

SUMMARIZED OBJECTIONS:
1. The Google style guide explicitly forbids `#pragma once`.
2. This results in a lot of code churn, but is only marginally better.
3. It's not C++ standardized/it's platform dependent/IBM's compiler
doesn't support it.
4. Popular projects like Chrome don't do `#pragma once` because of
history clutter.
5. Intermediate state of inconsistency as we transition to `#pragma
once` from #include guards.

OUR RESPONSE:
Objections (1), (2), and (4) are essentially the same -- Dominic Hamon
points out in a previous thread that the Google style guide was
canonized when `#pragma once` was Windows-only, and the guidance has
not changed since because of the history churn problem. As noted
above, we think the history churn problem is minimized by the fact
that it can be wrapped up into the Windows integration work.

For objection (3), the consensus seems to be that the vast majority of
compilers we care about (in particular, the ones supporting C++ 11) do
support it.

For objection (5) we believe the inconsistent state is likely to not
be long lived, as long as we commit to wrapping this work up into the
Windows integration work.

SUMMARIZED ADVANTAGES:
* Basically fool-proof. Communicates simply what its function is (you
include this file once). Semantically it is "the right tool for the
job".
* No need for namespacing conventions for #include guards.
* No conflicts with reserved identifiers[5].
* No internal conflicts between include guards in Stout, Process
library, and Mesos (this is one reason we need the namespacing
conventions)
* Reduces preprocessor definition clutter (we should rely on #define
as little as humanly possible).
* Optimized to be easy to read and reason about.

NEXT STEPS:
If we agree that this is the right thing to do, committers would ask
people to use `#pragma once` for new code when presented in code
reviews. For files that exist, I will take point on transitioning as
we complete the Windows integration work. I expect this work to
completely land before the new year.


Thanks,


[1] https://reviews.apache.org/r/39803/
[2] https://www.marc.info/?t=142540100400015=1=2
[3] https://issues.apache.org/jira/browse/MESOS-2211
[4] https://reviews.apache.org/r/30100/
[5] 
http://stackoverflow.com/questions/228783/what-are-the-rules-about-using-an-underscore-in-a-c-identifier


-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Backticks in comments

[Alex Clemmer]

+1. Additional note is that this is now the de facto syntax for code
snippets on the rest of our tools, too, including RB and JIRA.

On Mon, Nov 2, 2015 at 10:32 AM, Greg Mann  wrote:
> Hey folks!
> I wanted to bring up a style issue that I noticed recently. In some
> comments in the codebase, backticks are used to quote code excerpts and
> object names, while in other comments, single quotes are used. This doesn't
> seem to be documented in our style guide (nor in Google's), and I think it
> would be a good idea to establish a policy on this and document it, so that
> we can avoid wasted review cycles related to this in the future.
>
> It's likely that the backtick convention began because Doxygen will render
> backtick-enclosed text in monospace, and for this reason I would propose
> that we consistently use backticks to quote code excerpts and object names
> in comments from now on. What does everyone else think?
>
> Cheers,
> Greg



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Introducing a CMake-based build system for Mesos

[Alex Clemmer]

Another update, as of this weekend, we've checked in CMake changes
that will allow you to build a pretty wide swath of the master!

You can try it for yourself:

```
mkdir build && cd build
cmake .. && make -j 8
```

On Sun, Oct 4, 2015 at 9:50 AM, Alex Clemmer
<clemmer.alexan...@gmail.com> wrote:
> I like the idea, but sometimes it's not actually true that you want to
> touch a CMakeLists when you touch a makefile.am. For example, headers
> dependencies are automatically generated by CMake, so you don't have
> to list the .hpp files. If you were only adding an hpp file to stout,
> so this technique would make you change the CMakeLists even though you
> don't actually want to. BTW, another thing to consider is that we also
> need to find a way in general to tell people that if they modify a
> configure script they likely need to modify a CMakeLists file as well.
>
> So, for now I think it's fine that we have basically me going out and
> reminding people manually.
>
> Another thing worth thinking about on the horizon: it turns out
> maintaining the CMake build is a bit more complicated than it might
> seem. For example, adding simple changes to source files can break the
> CMake build because it makes slightly different assumptions about how
> to do things (like linking), so even if I monitor every review for
> touching particular files, sometimes the build just breaks and I have
> to find out why. We probably want to wrap CMake builds up into the
> bulid bot tests so that contributors get used to it being a thing to
> think about when you write code, and so the build doesn't just explode
> randomly when I pull down from the master branch. I think Artem is
> working on that.
>
> On Wed, Sep 30, 2015 at 1:31 AM, Alex Rukletsov <a...@mesosphere.com> wrote:
>> Can we extend a pre-commit hook in a way that it's not allowed to modify
>> makefile.am without touching CMakeLists and that any new file should
>> trigger touching CMakeLists? I think this can be part of reveiw r/38827.
>>
>> On Tue, Sep 29, 2015 at 6:46 PM, Alex Clemmer <clemmer.alexan...@gmail.com>
>> wrote:
>>
>>> Just as an update, we have expanded support for building the agent,
>>> and there is a review up[1] to support a large part of the master.
>>>
>>> The work is still anchored to the Windows work, so we expect the rest
>>> of the CMake solution to materialize sporadically until the
>>> November/December timeframe. In the mean time, I will be haunting all
>>> your reviews, asking you to make your makefile.am/configure changes in
>>> CMakeLists files also. :) (For more complicated changes, I will even
>>> attempt to help you write the code myself!)
>>>
>>> In the meantime, if you have the time, it would be great to have more
>>> people try it on their favorite platform. We've tried it on Windows
>>> 10, OS X 10.10, Ubuntu 14.04, and Ubuntu 15.04.
>>>
>>> [1] https://reviews.apache.org/r/38827/
>>>
>>> On Mon, Aug 10, 2015 at 12:13 PM, Alex Clemmer
>>> <clemmer.alexan...@gmail.com> wrote:
>>> > [... weeks later ...]
>>> >
>>> > Alex, I think that's a great idea, actually! The module you have
>>> > implemented is not a bad start. I've put it in a JIRA ticket so I
>>> > don't lose track of it[1].
>>> >
>>> > Also, just so we're clear: I'm not an expert at CMake. I wish I were,
>>> > it would have made this whole thing much easier.
>>> >
>>> > [1] https://issues.apache.org/jira/browse/MESOS-3249
>>> >
>>> > On Wed, Jul 29, 2015 at 2:31 AM, Alex Rukletsov <a...@mesosphere.com>
>>> wrote:
>>> >> One related thing I have started to work on but have never polished is
>>> >> FindMesos CMake script. The prototype lives here:
>>> >>
>>> https://github.com/rukletsov/mesos-modules/blob/master/cmake-modules/FindMesos.cmake
>>> >>
>>> >> My original idea was to support not only system Mesos, but also custom
>>> >> builds, so that writers of Mesos Modules can use the script as well.
>>> >>
>>> >> Alex, as CMake expert, let me know what you think!
>>> >>
>>> >> On Mon, Jul 27, 2015 at 7:48 PM, Alex Clemmer <
>>> clemmer.alexan...@gmail.com>
>>> >> wrote:
>>> >>
>>> >>> Yes, CMake is currently checking if the -std=c++11 flag exists. CMake
>>> >>> 3 supports the autotools-style "feature check" style (which would be
>>> >>> more

Re: Introducing a CMake-based build system for Mesos

[Alex Clemmer]

I like the idea, but sometimes it's not actually true that you want to
touch a CMakeLists when you touch a makefile.am. For example, headers
dependencies are automatically generated by CMake, so you don't have
to list the .hpp files. If you were only adding an hpp file to stout,
so this technique would make you change the CMakeLists even though you
don't actually want to. BTW, another thing to consider is that we also
need to find a way in general to tell people that if they modify a
configure script they likely need to modify a CMakeLists file as well.

So, for now I think it's fine that we have basically me going out and
reminding people manually.

Another thing worth thinking about on the horizon: it turns out
maintaining the CMake build is a bit more complicated than it might
seem. For example, adding simple changes to source files can break the
CMake build because it makes slightly different assumptions about how
to do things (like linking), so even if I monitor every review for
touching particular files, sometimes the build just breaks and I have
to find out why. We probably want to wrap CMake builds up into the
bulid bot tests so that contributors get used to it being a thing to
think about when you write code, and so the build doesn't just explode
randomly when I pull down from the master branch. I think Artem is
working on that.

On Wed, Sep 30, 2015 at 1:31 AM, Alex Rukletsov <a...@mesosphere.com> wrote:
> Can we extend a pre-commit hook in a way that it's not allowed to modify
> makefile.am without touching CMakeLists and that any new file should
> trigger touching CMakeLists? I think this can be part of reveiw r/38827.
>
> On Tue, Sep 29, 2015 at 6:46 PM, Alex Clemmer <clemmer.alexan...@gmail.com>
> wrote:
>
>> Just as an update, we have expanded support for building the agent,
>> and there is a review up[1] to support a large part of the master.
>>
>> The work is still anchored to the Windows work, so we expect the rest
>> of the CMake solution to materialize sporadically until the
>> November/December timeframe. In the mean time, I will be haunting all
>> your reviews, asking you to make your makefile.am/configure changes in
>> CMakeLists files also. :) (For more complicated changes, I will even
>> attempt to help you write the code myself!)
>>
>> In the meantime, if you have the time, it would be great to have more
>> people try it on their favorite platform. We've tried it on Windows
>> 10, OS X 10.10, Ubuntu 14.04, and Ubuntu 15.04.
>>
>> [1] https://reviews.apache.org/r/38827/
>>
>> On Mon, Aug 10, 2015 at 12:13 PM, Alex Clemmer
>> <clemmer.alexan...@gmail.com> wrote:
>> > [... weeks later ...]
>> >
>> > Alex, I think that's a great idea, actually! The module you have
>> > implemented is not a bad start. I've put it in a JIRA ticket so I
>> > don't lose track of it[1].
>> >
>> > Also, just so we're clear: I'm not an expert at CMake. I wish I were,
>> > it would have made this whole thing much easier.
>> >
>> > [1] https://issues.apache.org/jira/browse/MESOS-3249
>> >
>> > On Wed, Jul 29, 2015 at 2:31 AM, Alex Rukletsov <a...@mesosphere.com>
>> wrote:
>> >> One related thing I have started to work on but have never polished is
>> >> FindMesos CMake script. The prototype lives here:
>> >>
>> https://github.com/rukletsov/mesos-modules/blob/master/cmake-modules/FindMesos.cmake
>> >>
>> >> My original idea was to support not only system Mesos, but also custom
>> >> builds, so that writers of Mesos Modules can use the script as well.
>> >>
>> >> Alex, as CMake expert, let me know what you think!
>> >>
>> >> On Mon, Jul 27, 2015 at 7:48 PM, Alex Clemmer <
>> clemmer.alexan...@gmail.com>
>> >> wrote:
>> >>
>> >>> Yes, CMake is currently checking if the -std=c++11 flag exists. CMake
>> >>> 3 supports the autotools-style "feature check" style (which would be
>> >>> more appropriate on platforms like Windows, anyway) but it's not clear
>> >>> how far back we want to support just yet -- right now we support back
>> >>> to 2.8.
>> >>>
>> >>> One thing to consider is that the "default" install of CMake for
>> >>> apt-get on recent versions of Ubuntu is v2.8. In general, we'd like to
>> >>> have as smooth an install experience as possible for as many platforms
>> >>> as we can; I'm willing to be convinced that this isn't worth it, but
>> >>> to be safe we've developed with 2.8 in mind, becaus

Re: Introducing a CMake-based build system for Mesos

[Alex Clemmer]

Just as an update, we have expanded support for building the agent,
and there is a review up[1] to support a large part of the master.

The work is still anchored to the Windows work, so we expect the rest
of the CMake solution to materialize sporadically until the
November/December timeframe. In the mean time, I will be haunting all
your reviews, asking you to make your makefile.am/configure changes in
CMakeLists files also. :) (For more complicated changes, I will even
attempt to help you write the code myself!)

In the meantime, if you have the time, it would be great to have more
people try it on their favorite platform. We've tried it on Windows
10, OS X 10.10, Ubuntu 14.04, and Ubuntu 15.04.

[1] https://reviews.apache.org/r/38827/

On Mon, Aug 10, 2015 at 12:13 PM, Alex Clemmer
<clemmer.alexan...@gmail.com> wrote:
> [... weeks later ...]
>
> Alex, I think that's a great idea, actually! The module you have
> implemented is not a bad start. I've put it in a JIRA ticket so I
> don't lose track of it[1].
>
> Also, just so we're clear: I'm not an expert at CMake. I wish I were,
> it would have made this whole thing much easier.
>
> [1] https://issues.apache.org/jira/browse/MESOS-3249
>
> On Wed, Jul 29, 2015 at 2:31 AM, Alex Rukletsov <a...@mesosphere.com> wrote:
>> One related thing I have started to work on but have never polished is
>> FindMesos CMake script. The prototype lives here:
>> https://github.com/rukletsov/mesos-modules/blob/master/cmake-modules/FindMesos.cmake
>>
>> My original idea was to support not only system Mesos, but also custom
>> builds, so that writers of Mesos Modules can use the script as well.
>>
>> Alex, as CMake expert, let me know what you think!
>>
>> On Mon, Jul 27, 2015 at 7:48 PM, Alex Clemmer <clemmer.alexan...@gmail.com>
>> wrote:
>>
>>> Yes, CMake is currently checking if the -std=c++11 flag exists. CMake
>>> 3 supports the autotools-style "feature check" style (which would be
>>> more appropriate on platforms like Windows, anyway) but it's not clear
>>> how far back we want to support just yet -- right now we support back
>>> to 2.8.
>>>
>>> One thing to consider is that the "default" install of CMake for
>>> apt-get on recent versions of Ubuntu is v2.8. In general, we'd like to
>>> have as smooth an install experience as possible for as many platforms
>>> as we can; I'm willing to be convinced that this isn't worth it, but
>>> to be safe we've developed with 2.8 in mind, because it's easier to go
>>> from 2.8 to 3 than the reverse.
>>>
>>> On Mon, Jul 27, 2015 at 9:37 AM, James Peach <jor...@gmail.com> wrote:
>>> >
>>> >> On Jul 23, 2015, at 12:40 PM, Alex Clemmer <clemmer.alexan...@gmail.com>
>>> wrote:
>>> >>
>>> >> A fix is up for review here[1]. Thanks again for your feedback, this
>>> >> is very valuable!
>>> >>
>>> >> [1] https://reviews.apache.org/r/36743/
>>> >
>>> > AFAICT this just checks whether the -std=c++11 compiler option is
>>> accepted. The equivalent autoconf macro checks that various C++11 features
>>> compile, and people have added more over time ...
>>> 9eda4331dd23c3646aba1ec710e0dd3190e579ab,
>>> 623d6a0d0f0eb90be80b7e95c91ece89de513367,
>>> b930d5ce32b60b7c126844a3ef6ae119d36bc8d0, etc.
>>> >
>>> > Am I reading the cmake right?
>>> >
>>> >>
>>> >> On Thu, Jul 23, 2015 at 12:18 PM, Vinod Kone <vinodk...@gmail.com>
>>> wrote:
>>> >>> yup.
>>> >>>
>>> >>> checking for C++ compiler version... 4.1.2
>>> >>>
>>> >>> checking for C++ compiler vendor... (cached) gnu
>>> >>>
>>> >>> configure: error: GCC 4.8 or higher required (found 4.1.2)
>>> >>>
>>> >>> [vinod@smfd-atr-11-sr1 build-cmake]$ echo $?
>>> >>>
>>> >>> 1
>>> >>>
>>> >>> On Thu, Jul 23, 2015 at 12:17 PM, Alex Clemmer <
>>> clemmer.alexan...@gmail.com>
>>> >>> wrote:
>>> >>>
>>> >>>> We can easily change that to be a FATAL_ERROR or a WARNING. I
>>> >>>> recommend being at parity with autotools -- am I correct in assuming
>>> >>>> that it errors out?
>>> >>>>
>>> >>>> On Thu, Jul 23, 2015 at 12:12 PM, Vinod Kone <vinodk...@gmail.com>
>>> w

Re: Introducing a CMake-based build system for Mesos

[Alex Clemmer]

[... weeks later ...]

Alex, I think that's a great idea, actually! The module you have
implemented is not a bad start. I've put it in a JIRA ticket so I
don't lose track of it[1].

Also, just so we're clear: I'm not an expert at CMake. I wish I were,
it would have made this whole thing much easier.

[1] https://issues.apache.org/jira/browse/MESOS-3249

On Wed, Jul 29, 2015 at 2:31 AM, Alex Rukletsov a...@mesosphere.com wrote:
 One related thing I have started to work on but have never polished is
 FindMesos CMake script. The prototype lives here:
 https://github.com/rukletsov/mesos-modules/blob/master/cmake-modules/FindMesos.cmake

 My original idea was to support not only system Mesos, but also custom
 builds, so that writers of Mesos Modules can use the script as well.

 Alex, as CMake expert, let me know what you think!

 On Mon, Jul 27, 2015 at 7:48 PM, Alex Clemmer clemmer.alexan...@gmail.com
 wrote:

 Yes, CMake is currently checking if the -std=c++11 flag exists. CMake
 3 supports the autotools-style feature check style (which would be
 more appropriate on platforms like Windows, anyway) but it's not clear
 how far back we want to support just yet -- right now we support back
 to 2.8.

 One thing to consider is that the default install of CMake for
 apt-get on recent versions of Ubuntu is v2.8. In general, we'd like to
 have as smooth an install experience as possible for as many platforms
 as we can; I'm willing to be convinced that this isn't worth it, but
 to be safe we've developed with 2.8 in mind, because it's easier to go
 from 2.8 to 3 than the reverse.

 On Mon, Jul 27, 2015 at 9:37 AM, James Peach jor...@gmail.com wrote:
 
  On Jul 23, 2015, at 12:40 PM, Alex Clemmer clemmer.alexan...@gmail.com
 wrote:
 
  A fix is up for review here[1]. Thanks again for your feedback, this
  is very valuable!
 
  [1] https://reviews.apache.org/r/36743/
 
  AFAICT this just checks whether the -std=c++11 compiler option is
 accepted. The equivalent autoconf macro checks that various C++11 features
 compile, and people have added more over time ...
 9eda4331dd23c3646aba1ec710e0dd3190e579ab,
 623d6a0d0f0eb90be80b7e95c91ece89de513367,
 b930d5ce32b60b7c126844a3ef6ae119d36bc8d0, etc.
 
  Am I reading the cmake right?
 
 
  On Thu, Jul 23, 2015 at 12:18 PM, Vinod Kone vinodk...@gmail.com
 wrote:
  yup.
 
  checking for C++ compiler version... 4.1.2
 
  checking for C++ compiler vendor... (cached) gnu
 
  configure: error: GCC 4.8 or higher required (found 4.1.2)
 
  [vinod@smfd-atr-11-sr1 build-cmake]$ echo $?
 
  1
 
  On Thu, Jul 23, 2015 at 12:17 PM, Alex Clemmer 
 clemmer.alexan...@gmail.com
  wrote:
 
  We can easily change that to be a FATAL_ERROR or a WARNING. I
  recommend being at parity with autotools -- am I correct in assuming
  that it errors out?
 
  On Thu, Jul 23, 2015 at 12:12 PM, Vinod Kone vinodk...@gmail.com
 wrote:
  The one thing I found odd while testing was that some errors when
 running
  'cmake' do not result in a non-zero exit status.
  For example, when I tested with an older version of GCC it gave a
 warning
  about C++11 not being supported but went ahead otherwise.
 
  -- Performing Test COMPILER_SUPPORTS_CXX11 - Failed
 
  *--
 
 
 Thecompiler/usr/bin/c++doesnotsupportthe`-std=c++11`flag.PleaseuseadifferentC++compiler.*
 
  -- Looking for include file pthread.h
 
  -- Looking for include file pthread.h - found
 
  -- Looking for pthread_create
 
  -- Looking for pthread_create - not found
 
  -- Looking for pthread_create in pthreads
 
  -- Looking for pthread_create in pthreads - not found
 
  -- Looking for pthread_create in pthread
 
  -- Looking for pthread_create in pthread - found
 
  -- Found Threads: TRUE
 
  -- Found ZLIB: /usr/lib64/libz.so (found version 1.2.3)
 
  -- Found APR headers: /usr/include/apr-1
 
  -- Found APR library: /usr/lib64/libapr-1.so
 
  -- Found APRUTIL headers: /usr/include/apr-1
 
  -- Found APRUTIL library: /usr/lib64/libaprutil-1.so
 
  -- Found SVN lib: /usr/lib64/libsvn_client-1.so
 
  -- Found SVN lib: /usr/lib64/libsvn_delta-1.so
 
  -- Found SVN lib: /usr/lib64/libsvn_diff-1.so
 
  -- Found SVN lib: /usr/lib64/libsvn_fs-1.so
 
  -- Found SVN lib: /usr/lib64/libsvn_fs_base-1.so
 
 
 
 
  On Thu, Jul 23, 2015 at 12:07 PM, Alex Clemmer 
  clemmer.alexan...@gmail.com
  wrote:
 
  I've put up a pair of fixes, tested on OS X 10.10. They are here:
 
  (1) https://reviews.apache.org/r/36740/
  (2) https://reviews.apache.org/r/36741/
 
  This should resolve the issues, and thanks again for the bug report.
 
  On Thu, Jul 23, 2015 at 3:32 AM, haosdent haosd...@gmail.com
 wrote:
  Sure, I use OS X 10.10. Seems OS X don't have librt, don't add rt
 when
  the
  operate system is OSX?
 
  On Thu, Jul 23, 2015 at 6:22 PM, Alex Clemmer 
  clemmer.alexan...@gmail.com
  wrote:
 
  Thanks for reporting the issue! I appreciate it.
 
  This code is trying to find librt, which provides the POSIX.1b
  Realtime Extension (i.e., things like message passing, async

Re: mesos git commit: Candidate port of result.hpp.

[Alex Clemmer]

Yes sorry I pushed this out just as we went to dinner. I meant to redo the 
commit message.
This is actually a required change to compile on windows. The _try is (I think) 
a macro defined somewhere.
A better name would communicate this. I'll update the review (I haven't had 
time yet today).
Sent from Outlook

_
From: Benjamin Mahler benjamin.mah...@gmail.com
Sent: Friday, July 31, 2015 12:15
Subject: Re: mesos git commit: Candidate port of result.hpp.
To: dev dev@mesos.apache.org
Cc: Benjamin Hindman b...@mesosphere.io,  clemmer.alexan...@gmail.com,  
comm...@mesos.apache.org


   Hm.. this could have used a better commit name, what is a candidate 
port?   
  Looks like just a variable rename to me :)  
   On Fri, Jul 31, 2015 at 11:38 AM, b...@apache.org wrote:
 Repository: mesos 
 Updated Branches: 
   refs/heads/master d144fc7ba - d018eb712 
  
  
 Candidate port of result.hpp. 
  
 Review:  https://reviews.apache.org/r/36971 
  
  
 Project:  http://git-wip-us.apache.org/repos/asf/mesos/repo 
 Commit:  http://git-wip-us.apache.org/repos/asf/mesos/commit/d018eb71 
 Tree:  http://git-wip-us.apache.org/repos/asf/mesos/tree/d018eb71 
 Diff:  http://git-wip-us.apache.org/repos/asf/mesos/diff/d018eb71 
  
 Branch: refs/heads/master 
 Commit: d018eb712000888a996046fb9b5d8bb8e79e23e9 
 Parents: d144fc7 
 Author: Alex Clemmer  clemmer.alexan...@gmail.com 
 Authored: Fri Jul 31 11:37:57 2015 -0700 
 Committer: Benjamin Hindman  benjamin.hind...@gmail.com 
 Committed: Fri Jul 31 11:37:58 2015 -0700 
  
 -- 
  3rdparty/libprocess/3rdparty/stout/include/stout/result.hpp | 8  
  1 file changed, 4 insertions(+), 4 deletions(-) 
 -- 
  
  
  
http://git-wip-us.apache.org/repos/asf/mesos/blob/d018eb71/3rdparty/libprocess/3rdparty/stout/include/stout/result.hpp
 
 -- 
 diff --git a/3rdparty/libprocess/3rdparty/stout/include/stout/result.hpp 
b/3rdparty/libprocess/3rdparty/stout/include/stout/result.hpp 
 index 3d20614..f0b0a48 100644 
 --- a/3rdparty/libprocess/3rdparty/stout/include/stout/result.hpp 
 +++ b/3rdparty/libprocess/3rdparty/stout/include/stout/result.hpp 
 @@ -68,10 +68,10 @@ public: 
             TryOptionT(Some(option.get())) : 
             TryOptionT(None())) {} 
  
 -  Result(const TryT _try) 
 -    : data(_try.isSome() ? 
 -           TryOptionT(Some(_try.get())) : 
 -           TryOptionT(Error(_try.error( {} 
 +  Result(const TryT _t) 
 +    : data(_t.isSome() ? 
 +           TryOptionT(Some(_t.get())) : 
 +           TryOptionT(Error(_t.error( {} 
  
    Result(const None none) 
      : data(none) {}

Re: Introducing a CMake-based build system for Mesos

[Alex Clemmer]

Yes, CMake is currently checking if the -std=c++11 flag exists. CMake
3 supports the autotools-style feature check style (which would be
more appropriate on platforms like Windows, anyway) but it's not clear
how far back we want to support just yet -- right now we support back
to 2.8.

One thing to consider is that the default install of CMake for
apt-get on recent versions of Ubuntu is v2.8. In general, we'd like to
have as smooth an install experience as possible for as many platforms
as we can; I'm willing to be convinced that this isn't worth it, but
to be safe we've developed with 2.8 in mind, because it's easier to go
from 2.8 to 3 than the reverse.

On Mon, Jul 27, 2015 at 9:37 AM, James Peach jor...@gmail.com wrote:

 On Jul 23, 2015, at 12:40 PM, Alex Clemmer clemmer.alexan...@gmail.com 
 wrote:

 A fix is up for review here[1]. Thanks again for your feedback, this
 is very valuable!

 [1] https://reviews.apache.org/r/36743/

 AFAICT this just checks whether the -std=c++11 compiler option is accepted. 
 The equivalent autoconf macro checks that various C++11 features compile, and 
 people have added more over time ... 
 9eda4331dd23c3646aba1ec710e0dd3190e579ab, 
 623d6a0d0f0eb90be80b7e95c91ece89de513367, 
 b930d5ce32b60b7c126844a3ef6ae119d36bc8d0, etc.

 Am I reading the cmake right?


 On Thu, Jul 23, 2015 at 12:18 PM, Vinod Kone vinodk...@gmail.com wrote:
 yup.

 checking for C++ compiler version... 4.1.2

 checking for C++ compiler vendor... (cached) gnu

 configure: error: GCC 4.8 or higher required (found 4.1.2)

 [vinod@smfd-atr-11-sr1 build-cmake]$ echo $?

 1

 On Thu, Jul 23, 2015 at 12:17 PM, Alex Clemmer clemmer.alexan...@gmail.com
 wrote:

 We can easily change that to be a FATAL_ERROR or a WARNING. I
 recommend being at parity with autotools -- am I correct in assuming
 that it errors out?

 On Thu, Jul 23, 2015 at 12:12 PM, Vinod Kone vinodk...@gmail.com wrote:
 The one thing I found odd while testing was that some errors when running
 'cmake' do not result in a non-zero exit status.
 For example, when I tested with an older version of GCC it gave a warning
 about C++11 not being supported but went ahead otherwise.

 -- Performing Test COMPILER_SUPPORTS_CXX11 - Failed

 *--

 Thecompiler/usr/bin/c++doesnotsupportthe`-std=c++11`flag.PleaseuseadifferentC++compiler.*

 -- Looking for include file pthread.h

 -- Looking for include file pthread.h - found

 -- Looking for pthread_create

 -- Looking for pthread_create - not found

 -- Looking for pthread_create in pthreads

 -- Looking for pthread_create in pthreads - not found

 -- Looking for pthread_create in pthread

 -- Looking for pthread_create in pthread - found

 -- Found Threads: TRUE

 -- Found ZLIB: /usr/lib64/libz.so (found version 1.2.3)

 -- Found APR headers: /usr/include/apr-1

 -- Found APR library: /usr/lib64/libapr-1.so

 -- Found APRUTIL headers: /usr/include/apr-1

 -- Found APRUTIL library: /usr/lib64/libaprutil-1.so

 -- Found SVN lib: /usr/lib64/libsvn_client-1.so

 -- Found SVN lib: /usr/lib64/libsvn_delta-1.so

 -- Found SVN lib: /usr/lib64/libsvn_diff-1.so

 -- Found SVN lib: /usr/lib64/libsvn_fs-1.so

 -- Found SVN lib: /usr/lib64/libsvn_fs_base-1.so




 On Thu, Jul 23, 2015 at 12:07 PM, Alex Clemmer 
 clemmer.alexan...@gmail.com
 wrote:

 I've put up a pair of fixes, tested on OS X 10.10. They are here:

 (1) https://reviews.apache.org/r/36740/
 (2) https://reviews.apache.org/r/36741/

 This should resolve the issues, and thanks again for the bug report.

 On Thu, Jul 23, 2015 at 3:32 AM, haosdent haosd...@gmail.com wrote:
 Sure, I use OS X 10.10. Seems OS X don't have librt, don't add rt when
 the
 operate system is OSX?

 On Thu, Jul 23, 2015 at 6:22 PM, Alex Clemmer 
 clemmer.alexan...@gmail.com
 wrote:

 Thanks for reporting the issue! I appreciate it.

 This code is trying to find librt, which provides the POSIX.1b
 Realtime Extension (i.e., things like message passing, async I/O,
 mmap'd files, etc.). Assuming you're running some flavor of Linux,
 this _should_ exist on your system already, and `find_library` is the
 CMake-standard function to find it, so it is not immediately clear to
 me what went wrong here.

 Do you mind if I ask what system you are running?

 On Thu, Jul 23, 2015 at 1:16 AM, haosdent haosd...@gmail.com
 wrote:
 Hi, @Alex Clemmer  I try to build it on OS X 10.10

 ```
 mkdir build-cmake
 cmake ..
 make
 ```

 But have this error:
 ```
 CMake Error: The following variables are used in this project, but
 they
 are
 set to NOTFOUND.
 Please set them or make sure they are set and tested correctly in
 the
 CMake
 files:
 LIBRT_LIBRARIES
linked by target tests in directory
 /Users/haosdent/workspace/cpp/mesos/3rdparty/libprocess/src/tests

 -- Configuring incomplete, errors occurred!
 ```

 Any steps I wrong here?

 On Thu, Jul 23, 2015 at 11:27 AM, Marco Massenzio 
 ma...@mesosphere.io
 wrote:

 This is really cool!
 Eclipse CDT is becoming a bit tiresome to use, but JetLabs' CLion

Re: Where's Meses design documents

[Alex Clemmer]

Are you asking about the standard documentation (which is here[1]), or
are you asking about the design documents for every epic that has been
added historically?

[1] http://mesos.apache.org/documentation/latest/

On Sat, Jul 25, 2015 at 5:53 PM, Klaus Ma kl...@cguru.net wrote:
 Hi team,
 I can get design documents from each EPIC/Story, but where can I get ALL 
 design document of Mesos?

 Regards,Klaus Ma (马达), PMP® | http://www.cguru.net
 CallSend SMSCall from mobileAdd to SkypeYou'll need Skype CreditFree via 
 SkypeCallSend SMSCall from mobileAdd to SkypeYou'll need Skype CreditFree via 
 Skype



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Introducing a CMake-based build system for Mesos

[Alex Clemmer]

A fix is up for review here[1]. Thanks again for your feedback, this
is very valuable!

[1] https://reviews.apache.org/r/36743/

On Thu, Jul 23, 2015 at 12:18 PM, Vinod Kone vinodk...@gmail.com wrote:
 yup.

 checking for C++ compiler version... 4.1.2

 checking for C++ compiler vendor... (cached) gnu

 configure: error: GCC 4.8 or higher required (found 4.1.2)

 [vinod@smfd-atr-11-sr1 build-cmake]$ echo $?

 1

 On Thu, Jul 23, 2015 at 12:17 PM, Alex Clemmer clemmer.alexan...@gmail.com
 wrote:

 We can easily change that to be a FATAL_ERROR or a WARNING. I
 recommend being at parity with autotools -- am I correct in assuming
 that it errors out?

 On Thu, Jul 23, 2015 at 12:12 PM, Vinod Kone vinodk...@gmail.com wrote:
  The one thing I found odd while testing was that some errors when running
  'cmake' do not result in a non-zero exit status.
  For example, when I tested with an older version of GCC it gave a warning
  about C++11 not being supported but went ahead otherwise.
 
  -- Performing Test COMPILER_SUPPORTS_CXX11 - Failed
 
  *--
 
 Thecompiler/usr/bin/c++doesnotsupportthe`-std=c++11`flag.PleaseuseadifferentC++compiler.*
 
  -- Looking for include file pthread.h
 
  -- Looking for include file pthread.h - found
 
  -- Looking for pthread_create
 
  -- Looking for pthread_create - not found
 
  -- Looking for pthread_create in pthreads
 
  -- Looking for pthread_create in pthreads - not found
 
  -- Looking for pthread_create in pthread
 
  -- Looking for pthread_create in pthread - found
 
  -- Found Threads: TRUE
 
  -- Found ZLIB: /usr/lib64/libz.so (found version 1.2.3)
 
  -- Found APR headers: /usr/include/apr-1
 
  -- Found APR library: /usr/lib64/libapr-1.so
 
  -- Found APRUTIL headers: /usr/include/apr-1
 
  -- Found APRUTIL library: /usr/lib64/libaprutil-1.so
 
  -- Found SVN lib: /usr/lib64/libsvn_client-1.so
 
  -- Found SVN lib: /usr/lib64/libsvn_delta-1.so
 
  -- Found SVN lib: /usr/lib64/libsvn_diff-1.so
 
  -- Found SVN lib: /usr/lib64/libsvn_fs-1.so
 
  -- Found SVN lib: /usr/lib64/libsvn_fs_base-1.so
 
 
 
 
  On Thu, Jul 23, 2015 at 12:07 PM, Alex Clemmer 
 clemmer.alexan...@gmail.com
  wrote:
 
  I've put up a pair of fixes, tested on OS X 10.10. They are here:
 
  (1) https://reviews.apache.org/r/36740/
  (2) https://reviews.apache.org/r/36741/
 
  This should resolve the issues, and thanks again for the bug report.
 
  On Thu, Jul 23, 2015 at 3:32 AM, haosdent haosd...@gmail.com wrote:
   Sure, I use OS X 10.10. Seems OS X don't have librt, don't add rt when
  the
   operate system is OSX?
  
   On Thu, Jul 23, 2015 at 6:22 PM, Alex Clemmer 
  clemmer.alexan...@gmail.com
   wrote:
  
   Thanks for reporting the issue! I appreciate it.
  
   This code is trying to find librt, which provides the POSIX.1b
   Realtime Extension (i.e., things like message passing, async I/O,
   mmap'd files, etc.). Assuming you're running some flavor of Linux,
   this _should_ exist on your system already, and `find_library` is the
   CMake-standard function to find it, so it is not immediately clear to
   me what went wrong here.
  
   Do you mind if I ask what system you are running?
  
   On Thu, Jul 23, 2015 at 1:16 AM, haosdent haosd...@gmail.com
 wrote:
Hi, @Alex Clemmer  I try to build it on OS X 10.10
   
```
mkdir build-cmake
cmake ..
make
```
   
But have this error:
```
CMake Error: The following variables are used in this project, but
  they
   are
set to NOTFOUND.
Please set them or make sure they are set and tested correctly in
 the
   CMake
files:
LIBRT_LIBRARIES
linked by target tests in directory
/Users/haosdent/workspace/cpp/mesos/3rdparty/libprocess/src/tests
   
-- Configuring incomplete, errors occurred!
```
   
Any steps I wrong here?
   
On Thu, Jul 23, 2015 at 11:27 AM, Marco Massenzio 
  ma...@mesosphere.io
wrote:
   
This is really cool!
Eclipse CDT is becoming a bit tiresome to use, but JetLabs' CLion
  only
support cmake, so I definitely have a stake in this working :)
   
Please keep us posted on progress, I'll definitely try and give
 it a
   spin
on Ubuntu and OSX.
Thanks for doing it!
   
*Marco Massenzio*
*Distributed Systems Engineer*
   
On Wed, Jul 22, 2015 at 6:06 PM, Alex Clemmer 
   clemmer.alexan...@gmail.com

wrote:
   
 On Wed, Jul 22, 2015 at 3:47 PM, Vinod Kone 
 vinodk...@gmail.com
   wrote:
  This is exciting! Thanks for sharing the progress Alex.
 
  Mind sending us instructions on how to build/test with cmake
 for
   noobs
 like
  me?

 Ah, rats, I knew I was forgetting something.

 It actually looks pretty much like the autotools build system:

 1. Make sure you have all the normal system dependencies
  installed
 (like APR, etc.)
 2. Make sure you have CMake 2.8 or later installed on your
 machine.
 (On Ubuntu this looks like: `sudo apt-get install cmake

Re: Introducing a CMake-based build system for Mesos

[Alex Clemmer]

I've put up a pair of fixes, tested on OS X 10.10. They are here:

(1) https://reviews.apache.org/r/36740/
(2) https://reviews.apache.org/r/36741/

This should resolve the issues, and thanks again for the bug report.

On Thu, Jul 23, 2015 at 3:32 AM, haosdent haosd...@gmail.com wrote:
 Sure, I use OS X 10.10. Seems OS X don't have librt, don't add rt when the
 operate system is OSX?

 On Thu, Jul 23, 2015 at 6:22 PM, Alex Clemmer clemmer.alexan...@gmail.com
 wrote:

 Thanks for reporting the issue! I appreciate it.

 This code is trying to find librt, which provides the POSIX.1b
 Realtime Extension (i.e., things like message passing, async I/O,
 mmap'd files, etc.). Assuming you're running some flavor of Linux,
 this _should_ exist on your system already, and `find_library` is the
 CMake-standard function to find it, so it is not immediately clear to
 me what went wrong here.

 Do you mind if I ask what system you are running?

 On Thu, Jul 23, 2015 at 1:16 AM, haosdent haosd...@gmail.com wrote:
  Hi, @Alex Clemmer  I try to build it on OS X 10.10
 
  ```
  mkdir build-cmake
  cmake ..
  make
  ```
 
  But have this error:
  ```
  CMake Error: The following variables are used in this project, but they
 are
  set to NOTFOUND.
  Please set them or make sure they are set and tested correctly in the
 CMake
  files:
  LIBRT_LIBRARIES
  linked by target tests in directory
  /Users/haosdent/workspace/cpp/mesos/3rdparty/libprocess/src/tests
 
  -- Configuring incomplete, errors occurred!
  ```
 
  Any steps I wrong here?
 
  On Thu, Jul 23, 2015 at 11:27 AM, Marco Massenzio ma...@mesosphere.io
  wrote:
 
  This is really cool!
  Eclipse CDT is becoming a bit tiresome to use, but JetLabs' CLion only
  support cmake, so I definitely have a stake in this working :)
 
  Please keep us posted on progress, I'll definitely try and give it a
 spin
  on Ubuntu and OSX.
  Thanks for doing it!
 
  *Marco Massenzio*
  *Distributed Systems Engineer*
 
  On Wed, Jul 22, 2015 at 6:06 PM, Alex Clemmer 
 clemmer.alexan...@gmail.com
  
  wrote:
 
   On Wed, Jul 22, 2015 at 3:47 PM, Vinod Kone vinodk...@gmail.com
 wrote:
This is exciting! Thanks for sharing the progress Alex.
   
Mind sending us instructions on how to build/test with cmake for
 noobs
   like
me?
  
   Ah, rats, I knew I was forgetting something.
  
   It actually looks pretty much like the autotools build system:
  
   1. Make sure you have all the normal system dependencies installed
   (like APR, etc.)
   2. Make sure you have CMake 2.8 or later installed on your machine.
   (On Ubuntu this looks like: `sudo apt-get install cmake`)
   3. Go to the root of your Mesos source tree and do something like the
   following. Note that you will never have to run bootstrap or
   configure, so you should _only_ have to run the following commands.
  
   mkdir build-cmake
   cmake ..
   make
  
   4. Watch as it builds, and hopefully doesn't explode!
  
   Finally to run tests, you can do `make test ARGS=-V`. They run
   without ANSI colors right now, which is not ideal, but we know it's an
   issue.
  
  
   --
   Alex
  
   Theory is the first term in the Taylor series of practice. -- Thomas M
   Cover (1992)
  
 
 
 
 
  --
  Best Regards,
  Haosdent Huang



 --
 Alex

 Theory is the first term in the Taylor series of practice. -- Thomas M
 Cover (1992)




 --
 Best Regards,
 Haosdent Huang



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Introducing a CMake-based build system for Mesos

[Alex Clemmer]

We can easily change that to be a FATAL_ERROR or a WARNING. I
recommend being at parity with autotools -- am I correct in assuming
that it errors out?

On Thu, Jul 23, 2015 at 12:12 PM, Vinod Kone vinodk...@gmail.com wrote:
 The one thing I found odd while testing was that some errors when running
 'cmake' do not result in a non-zero exit status.
 For example, when I tested with an older version of GCC it gave a warning
 about C++11 not being supported but went ahead otherwise.

 -- Performing Test COMPILER_SUPPORTS_CXX11 - Failed

 *--
 Thecompiler/usr/bin/c++doesnotsupportthe`-std=c++11`flag.PleaseuseadifferentC++compiler.*

 -- Looking for include file pthread.h

 -- Looking for include file pthread.h - found

 -- Looking for pthread_create

 -- Looking for pthread_create - not found

 -- Looking for pthread_create in pthreads

 -- Looking for pthread_create in pthreads - not found

 -- Looking for pthread_create in pthread

 -- Looking for pthread_create in pthread - found

 -- Found Threads: TRUE

 -- Found ZLIB: /usr/lib64/libz.so (found version 1.2.3)

 -- Found APR headers: /usr/include/apr-1

 -- Found APR library: /usr/lib64/libapr-1.so

 -- Found APRUTIL headers: /usr/include/apr-1

 -- Found APRUTIL library: /usr/lib64/libaprutil-1.so

 -- Found SVN lib: /usr/lib64/libsvn_client-1.so

 -- Found SVN lib: /usr/lib64/libsvn_delta-1.so

 -- Found SVN lib: /usr/lib64/libsvn_diff-1.so

 -- Found SVN lib: /usr/lib64/libsvn_fs-1.so

 -- Found SVN lib: /usr/lib64/libsvn_fs_base-1.so




 On Thu, Jul 23, 2015 at 12:07 PM, Alex Clemmer clemmer.alexan...@gmail.com
 wrote:

 I've put up a pair of fixes, tested on OS X 10.10. They are here:

 (1) https://reviews.apache.org/r/36740/
 (2) https://reviews.apache.org/r/36741/

 This should resolve the issues, and thanks again for the bug report.

 On Thu, Jul 23, 2015 at 3:32 AM, haosdent haosd...@gmail.com wrote:
  Sure, I use OS X 10.10. Seems OS X don't have librt, don't add rt when
 the
  operate system is OSX?
 
  On Thu, Jul 23, 2015 at 6:22 PM, Alex Clemmer 
 clemmer.alexan...@gmail.com
  wrote:
 
  Thanks for reporting the issue! I appreciate it.
 
  This code is trying to find librt, which provides the POSIX.1b
  Realtime Extension (i.e., things like message passing, async I/O,
  mmap'd files, etc.). Assuming you're running some flavor of Linux,
  this _should_ exist on your system already, and `find_library` is the
  CMake-standard function to find it, so it is not immediately clear to
  me what went wrong here.
 
  Do you mind if I ask what system you are running?
 
  On Thu, Jul 23, 2015 at 1:16 AM, haosdent haosd...@gmail.com wrote:
   Hi, @Alex Clemmer  I try to build it on OS X 10.10
  
   ```
   mkdir build-cmake
   cmake ..
   make
   ```
  
   But have this error:
   ```
   CMake Error: The following variables are used in this project, but
 they
  are
   set to NOTFOUND.
   Please set them or make sure they are set and tested correctly in the
  CMake
   files:
   LIBRT_LIBRARIES
   linked by target tests in directory
   /Users/haosdent/workspace/cpp/mesos/3rdparty/libprocess/src/tests
  
   -- Configuring incomplete, errors occurred!
   ```
  
   Any steps I wrong here?
  
   On Thu, Jul 23, 2015 at 11:27 AM, Marco Massenzio 
 ma...@mesosphere.io
   wrote:
  
   This is really cool!
   Eclipse CDT is becoming a bit tiresome to use, but JetLabs' CLion
 only
   support cmake, so I definitely have a stake in this working :)
  
   Please keep us posted on progress, I'll definitely try and give it a
  spin
   on Ubuntu and OSX.
   Thanks for doing it!
  
   *Marco Massenzio*
   *Distributed Systems Engineer*
  
   On Wed, Jul 22, 2015 at 6:06 PM, Alex Clemmer 
  clemmer.alexan...@gmail.com
   
   wrote:
  
On Wed, Jul 22, 2015 at 3:47 PM, Vinod Kone vinodk...@gmail.com
  wrote:
 This is exciting! Thanks for sharing the progress Alex.

 Mind sending us instructions on how to build/test with cmake for
  noobs
like
 me?
   
Ah, rats, I knew I was forgetting something.
   
It actually looks pretty much like the autotools build system:
   
1. Make sure you have all the normal system dependencies
 installed
(like APR, etc.)
2. Make sure you have CMake 2.8 or later installed on your machine.
(On Ubuntu this looks like: `sudo apt-get install cmake`)
3. Go to the root of your Mesos source tree and do something like
 the
following. Note that you will never have to run bootstrap or
configure, so you should _only_ have to run the following commands.
   
mkdir build-cmake
cmake ..
make
   
4. Watch as it builds, and hopefully doesn't explode!
   
Finally to run tests, you can do `make test ARGS=-V`. They run
without ANSI colors right now, which is not ideal, but we know
 it's an
issue.
   
   
--
Alex
   
Theory is the first term in the Taylor series of practice. --
 Thomas M
Cover (1992)
   
  
  
  
  
   --
   Best Regards,
   Haosdent Huang

Re: Introducing a CMake-based build system for Mesos

[Alex Clemmer]

Thanks for reporting the issue! I appreciate it.

This code is trying to find librt, which provides the POSIX.1b
Realtime Extension (i.e., things like message passing, async I/O,
mmap'd files, etc.). Assuming you're running some flavor of Linux,
this _should_ exist on your system already, and `find_library` is the
CMake-standard function to find it, so it is not immediately clear to
me what went wrong here.

Do you mind if I ask what system you are running?

On Thu, Jul 23, 2015 at 1:16 AM, haosdent haosd...@gmail.com wrote:
 Hi, @Alex Clemmer  I try to build it on OS X 10.10

 ```
 mkdir build-cmake
 cmake ..
 make
 ```

 But have this error:
 ```
 CMake Error: The following variables are used in this project, but they are
 set to NOTFOUND.
 Please set them or make sure they are set and tested correctly in the CMake
 files:
 LIBRT_LIBRARIES
 linked by target tests in directory
 /Users/haosdent/workspace/cpp/mesos/3rdparty/libprocess/src/tests

 -- Configuring incomplete, errors occurred!
 ```

 Any steps I wrong here?

 On Thu, Jul 23, 2015 at 11:27 AM, Marco Massenzio ma...@mesosphere.io
 wrote:

 This is really cool!
 Eclipse CDT is becoming a bit tiresome to use, but JetLabs' CLion only
 support cmake, so I definitely have a stake in this working :)

 Please keep us posted on progress, I'll definitely try and give it a spin
 on Ubuntu and OSX.
 Thanks for doing it!

 *Marco Massenzio*
 *Distributed Systems Engineer*

 On Wed, Jul 22, 2015 at 6:06 PM, Alex Clemmer clemmer.alexan...@gmail.com
 
 wrote:

  On Wed, Jul 22, 2015 at 3:47 PM, Vinod Kone vinodk...@gmail.com wrote:
   This is exciting! Thanks for sharing the progress Alex.
  
   Mind sending us instructions on how to build/test with cmake for noobs
  like
   me?
 
  Ah, rats, I knew I was forgetting something.
 
  It actually looks pretty much like the autotools build system:
 
  1. Make sure you have all the normal system dependencies installed
  (like APR, etc.)
  2. Make sure you have CMake 2.8 or later installed on your machine.
  (On Ubuntu this looks like: `sudo apt-get install cmake`)
  3. Go to the root of your Mesos source tree and do something like the
  following. Note that you will never have to run bootstrap or
  configure, so you should _only_ have to run the following commands.
 
  mkdir build-cmake
  cmake ..
  make
 
  4. Watch as it builds, and hopefully doesn't explode!
 
  Finally to run tests, you can do `make test ARGS=-V`. They run
  without ANSI colors right now, which is not ideal, but we know it's an
  issue.
 
 
  --
  Alex
 
  Theory is the first term in the Taylor series of practice. -- Thomas M
  Cover (1992)
 




 --
 Best Regards,
 Haosdent Huang



-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Introducing a CMake-based build system for Mesos

[Alex Clemmer]

Yep! We sure do:

(1) https://issues.apache.org/jira/browse/MESOS-3098
(2) https://issues.apache.org/jira/browse/MESOS-3097
(3) https://issues.apache.org/jira/browse/MESOS-3102

On Wed, Jul 22, 2015 at 6:56 PM, haosdent haosd...@gmail.com wrote:
 do we have any tickets to record support Windows?
 On Jul 23, 2015 9:50 AM, haosdent haosd...@gmail.com wrote:

 cool!
 On Jul 23, 2015 5:12 AM, Alex Clemmer clemmer.alexan...@gmail.com
 wrote:

 Hey folks.

 For some time there has been vestigial interest in a CMake[1]-based
 alternative to the autotools-based build system that Mesos currently
 depends on (cf. MESOS-898[2]). In the near future we expect to start
 thinking about supporting little-known platforms such as Windows,
 (where a fully cross-platform build system is table stakes).

 To facilitate this, we implemented and recently committed an
 experimental MVP CMake-based build system that allows us to build the
 process library, as well as the tests for the process and stout
 libraries. Currently we have only seriously tested that this works
 Linux (specifically, we have asked CMake to compile the build spec to
 a standard make/gcc combination, much like autotools), but we expect
 this to anchor our thoughts on cross-platform support as we continue
 to think about our real x-plat story.

 The community can track completed and outstanding the work on
 MESOS-898[2].

 In the immediate term, we're hoping the community will help us work
 out the kinks by voicing their questions, concerns, proposing changes
 to our plan, or even (hopefully) attempting to build on their favorite
 flavor of Linux, and reporting bugs or irregularities.

 Finally, to get the ball rolling, I'd like to end with some FAQs:

 Q: Will autotools be deprecated soon?
 A: EMPHATIC NO. The CMake-based build system is an experimental work
 that we expect to inform the Mesos community's thoughts as we start to
 think about more robustly expanding support other platforms. If there
 are serious benefits, then the community may choose to standardize on
 this solution in the future, but that is a discussion to have far down
 the road.

 Q: Can I use this now?
 A: NO. You can use it to build the process library, the process
 library tests, and the stout tests. You cannot yet use it to
 completely build Mesos. That's intentional -- before we get too far
 down the road, we are hoping to get some early feedback.

 Q: How does CMake help the goal of cross-platform support?
 A: CMake seamlessly compiles to a variety of toolchains and build
 systems. This includes both Linux favorites like gcc and make, but
 also more exotic options like Eclipse, or even Visual Studio and MSVC.
 Additionally, CMake has fairly robust function and macro systems,
 which allows us to do things like support finding or building
 third-party dependencies on very heterogeneous systems

 Thanks a lot, and we look forward to hearing your feedback!

 [1] http://www.cmake.org/
 [2] https://issues.apache.org/jira/browse/MESOS-898


 --
 Alex

 Theory is the first term in the Taylor series of practice. -- Thomas M
 Cover (1992)





-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Introducing a CMake-based build system for Mesos

[Alex Clemmer]

Hey folks.

For some time there has been vestigial interest in a CMake[1]-based
alternative to the autotools-based build system that Mesos currently
depends on (cf. MESOS-898[2]). In the near future we expect to start
thinking about supporting little-known platforms such as Windows,
(where a fully cross-platform build system is table stakes).

To facilitate this, we implemented and recently committed an
experimental MVP CMake-based build system that allows us to build the
process library, as well as the tests for the process and stout
libraries. Currently we have only seriously tested that this works
Linux (specifically, we have asked CMake to compile the build spec to
a standard make/gcc combination, much like autotools), but we expect
this to anchor our thoughts on cross-platform support as we continue
to think about our real x-plat story.

The community can track completed and outstanding the work on MESOS-898[2].

In the immediate term, we're hoping the community will help us work
out the kinks by voicing their questions, concerns, proposing changes
to our plan, or even (hopefully) attempting to build on their favorite
flavor of Linux, and reporting bugs or irregularities.

Finally, to get the ball rolling, I'd like to end with some FAQs:

Q: Will autotools be deprecated soon?
A: EMPHATIC NO. The CMake-based build system is an experimental work
that we expect to inform the Mesos community's thoughts as we start to
think about more robustly expanding support other platforms. If there
are serious benefits, then the community may choose to standardize on
this solution in the future, but that is a discussion to have far down
the road.

Q: Can I use this now?
A: NO. You can use it to build the process library, the process
library tests, and the stout tests. You cannot yet use it to
completely build Mesos. That's intentional -- before we get too far
down the road, we are hoping to get some early feedback.

Q: How does CMake help the goal of cross-platform support?
A: CMake seamlessly compiles to a variety of toolchains and build
systems. This includes both Linux favorites like gcc and make, but
also more exotic options like Eclipse, or even Visual Studio and MSVC.
Additionally, CMake has fairly robust function and macro systems,
which allows us to do things like support finding or building
third-party dependencies on very heterogeneous systems

Thanks a lot, and we look forward to hearing your feedback!

[1] http://www.cmake.org/
[2] https://issues.apache.org/jira/browse/MESOS-898


-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)

Re: Introducing a CMake-based build system for Mesos

[Alex Clemmer]

On Wed, Jul 22, 2015 at 3:47 PM, Vinod Kone vinodk...@gmail.com wrote:
 This is exciting! Thanks for sharing the progress Alex.

 Mind sending us instructions on how to build/test with cmake for noobs like
 me?

Ah, rats, I knew I was forgetting something.

It actually looks pretty much like the autotools build system:

1. Make sure you have all the normal system dependencies installed
(like APR, etc.)
2. Make sure you have CMake 2.8 or later installed on your machine.
(On Ubuntu this looks like: `sudo apt-get install cmake`)
3. Go to the root of your Mesos source tree and do something like the
following. Note that you will never have to run bootstrap or
configure, so you should _only_ have to run the following commands.

mkdir build-cmake
cmake ..
make

4. Watch as it builds, and hopefully doesn't explode!

Finally to run tests, you can do `make test ARGS=-V`. They run
without ANSI colors right now, which is not ideal, but we know it's an
issue.


-- 
Alex

Theory is the first term in the Taylor series of practice. -- Thomas M
Cover (1992)