Re: Review Request 18341: Modified the Registrar to notify callers only _after_ the storage operation completes.

2014-02-26 Thread Benjamin Hindman

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18341/#review35536
---

Ship it!



src/master/registrar.cpp
https://reviews.apache.org/r/18341/#comment66132

It looks like 'applied' is true iff 'operator ()' returns Some or None. Can 
we simplify the semantics and not require setting 'applied'? What about another 
wrapper that calls 'operator ()' and sets 'applied' based on the result of 
'operator ()'. Also, do you perceive any cases of where Error won't turn into 
an 'applied = false' case? Finally, since maybe s/applied/success/ since some 
operations aren't technically applied as discussed in Vinod's previous 
review. 



src/master/registrar.cpp
https://reviews.apache.org/r/18341/#comment66130

Looks like mis-alignment here.



src/master/registrar.cpp
https://reviews.apache.org/r/18341/#comment66129

Minor nit: why ERROR instead of WARNING? This isn't an error, this is an 
expected case!

Also, when first reading this I was sort of surprised that we were just 
printing this out and not completing the promises/operations here. Perhaps a 
brief comment explaining that we set this first and do the same for the 
operations below?


- Benjamin Hindman


On Feb. 26, 2014, 2:19 a.m., Ben Mahler wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/18341/
 ---
 
 (Updated Feb. 26, 2014, 2:19 a.m.)
 
 
 Review request for mesos, Benjamin Hindman and Vinod Kone.
 
 
 Bugs: MESOS-764
 https://issues.apache.org/jira/browse/MESOS-764
 
 
 Repository: mesos-git
 
 
 Description
 ---
 
 We previously were setting the Promises in some cases _before_ writing to 
 storage. While this approach was correct, it requires some time and 
 explanation to convince oneself of this.
 
 This is why this patch simplifies the semantics by only setting the Promises 
 _after_ we've written to storage.
 
 Also, 'Mutation' has now become 'Operation' to better reflect the fact that 
 some operations do not mutate state when applied.
 
 
 Diffs
 -
 
   src/master/registrar.cpp ee16121035db21d966ee151483dd23cbc4a495c2 
 
 Diff: https://reviews.apache.org/r/18341/diff/
 
 
 Testing
 ---
 
 make check
 
 
 Thanks,
 
 Ben Mahler
 




Re: Path forward to C++11.

2014-02-26 Thread Benjamin Mahler
Update on MESOS-750:

0.17.0 is being released with the --with-cxx11 capability at configure time.

I chatted with some of the committers and we'd like to move forward with
the final two phases of MESOS-750 as follows:

0.18.0: --with-cxx11 will become the _default_. --without-cxx11 will be
available for those who wish to use older compilers.

0.19.0: c++11 required; Mesos will not build with a compiler that does not
have c++11 support.

If there are any objections please speak now. If not, I'll update the
ticket to reflect the current path towards C++11.


On Mon, Oct 28, 2013 at 8:09 PM, Benjamin Mahler
benjamin.mah...@gmail.comwrote:

 Thanks for the heads up, we're likely to upgrade to 3.3.6, but I've yet to
 confirm that it builds with c++11.


 On Thu, Oct 24, 2013 at 8:10 AM, Tim St Clair tstcl...@redhat.com wrote:

 +1 to the approach.

 FWIW made a quick update to the zookeeper.spec (
 http://koji.fedoraproject.org/koji/buildinfo?buildID=465529) and updated:

 export CC='gcc -std=c11'
 export CXX='g++ -std=c++11'

 prior to 'configure', and it failed compilation pretty quickly.
 It may make sense to track integration issues and versions...somewhere.

 So when --with-cpp11 is enabled, it can hard check on version =
 fixed_version.

 Cheers,
 Tim

 - Original Message -
  From: Benjamin Mahler benjamin.mah...@gmail.com
  To: dev@mesos.apache.org
  Sent: Wednesday, October 23, 2013 4:46:49 PM
  Subject: Path forward to C++11.
 
  As discussed in MESOS-750 
 https://issues.apache.org/jira/browse/MESOS-750,
  we would like to move to C++11.
 
  The proposed plan is to do this in phases to allow everyone time to
 adjust
  to this change:
 
 
  Phase 1: Send an email to the dev@ list soliciting objections /
 feedback.
  We're currently here. :)
 
  Phase 2: Add an option --with-cpp11 that defaults to false.
 
  Phase 3: At this point, developers can and should begin using
 --with-cpp11.
  In this phase, we'll need to make sure the project and dependencies can
  compile with c++11.
 
  Phase 4: Default to --with-cpp11 but still leave the option in order to
  allow folks to turn this off if needed. At this point, we still cannot
 rely
  on C++11.
 
  Phase 5: Remove --with-cpp11 leaving no ability to turn it off. From
 this
  point forward, we can begin to use C++11 features unconditionally.
 
 
  Please reply to this thread with any objections, comments, or advice! To
  get a better sense of how others feel about this (I currently only know
  that some of the other core contributors are on board), please feel
 free to
  reply with something as simple as a +1 as well.
 
  Thanks!
  Ben
 

 --
 Cheers,
 Tim





Re: Review Request 18403: Added support for launching tasks by TaskInfo.

2014-02-26 Thread Ian Downes

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18403/#review35504
---


I think we need to work on this a little. It seems that making the launching 
state explicit and then cleaning up behavior (killTask) when in this state is 
necessary.


src/slave/containerizer/containerizer.hpp
https://reviews.apache.org/r/18403/#comment66140

Should this be in mesos_containerizer? I was thinking if the task didn't 
have task.executor() then the determination of the executor would be 
containerizer implementation dependent.



src/slave/containerizer/mesos_containerizer.cpp
https://reviews.apache.org/r/18403/#comment66141

Why is this a Future? Can you not just pass the ExecutorInfo to exec() 
and have it return?



src/slave/containerizer/mesos_containerizer.cpp
https://reviews.apache.org/r/18403/#comment66143

const ExecutorInfo ?



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66056

Could you swap this logic? i.e.,
if (contains) {
  launch = framework-launchingExecutors[];
} else {
  launch = framework-launch();
}



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66057

What happens to Executor* if the future is ready?



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66058

s/rebust/robust/



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66060

Previously executor == NULL was unexpected but now it seems weird to have 
NULL as a 'normal' value. Can we capture this is a better way?



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66062

We shouldn't be returning TASK_LOST and claiming we cannot find the 
executor here if the executor is actually launching. If we haven't sent a 
launching executor the task then we should prevent that and act appropriately. 



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66063

What about introducing a new executor state - Executor::LAUNCHING?



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66065

s/We might be in a situation where the executor is launching but not yet 
ready. The second half is scheduled until executor is ready, or called directly 
if executor is running./If the executor is still launching we'll defer 
registering until it is ready./?



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66078

ditto - what happens to Executor* if the future is ready?



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66134

ditto - Executor*



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66135

okay :-p



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66136

perhaps not in this change, but this should done when the 
FutureExecutorInfo is ready rather than now.



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66137

ditto - should start timeout when the launch is ready.



src/slave/slave.cpp
https://reviews.apache.org/r/18403/#comment66138

The determination of executor resources should be done by each 
containerizer implementation if they choose an executor.


- Ian Downes


On Feb. 26, 2014, 12:04 a.m., Niklas Nielsen wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/18403/
 ---
 
 (Updated Feb. 26, 2014, 12:04 a.m.)
 
 
 Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Ian Downes, 
 Till Toenshoff, and Vinod Kone.
 
 
 Bugs: MESOS-922
 https://issues.apache.org/jira/browse/MESOS-922
 
 
 Repository: mesos-git
 
 
 Description
 ---
 
 This patch delegates the choice of executor to the containerizer by removing 
 executorInfo dependencies up until Containerizer::launch().
 Containerizer::launch() now returns a future to the executor info that is 
 being run and the slave creates the corresponding executor structure when 
 launch completes.
 This means message handling from the running executor to the slave in the 
 interim where the executor structure has not created, need to be enqueued 
 until executor is ready. So far, registerExecutor() and reregisterExecutor() 
 has been split into two continuations to deal with this issue.
 
 
 Diffs
 -
 
   src/slave/containerizer/containerizer.hpp d9ae326 
   src/slave/containerizer/containerizer.cpp d0a1023 
   src/slave/containerizer/mesos_containerizer.hpp ee1fd30 
   src/slave/containerizer/mesos_containerizer.cpp 6d990cb 
   src/slave/slave.hpp 01b80df 
   src/slave/slave.cpp 4f5349b 
   src/tests/containerizer.hpp 5686398 
   src/tests/containerizer.cpp bfb9341 
 
 Diff: https://reviews.apache.org/r/18403/diff/
 
 
 Testing
 ---
 
 make check
 
 
 Thanks,
 
 Niklas Nielsen
 




Re: Path forward to C++11.

2014-02-26 Thread Dominic Hamon
What do you mean by 'C++11 support' in the compiler? g++4.7 supports some
features, not all.

I think we need to define the set of C++11 features we want to support
first before making any decisions regarding timeline. Then we can look at
the state of the nation in regards to C++11 feature support in different
compilers on different OSs, and which are available out of the box. We
don't want to limit our community involvement by shutting out certain
platforms for development.

Ideally, we'd be able to say something like 0.19.0 builds on g++4.7 and
clang 3.3 or greater and 0.20.0 builds on g++4.8 and clang 3.4 or
greater instead of blanket C++11 support.




On Wed, Feb 26, 2014 at 10:20 AM, Benjamin Mahler benjamin.mah...@gmail.com
 wrote:

 Update on MESOS-750:

 0.17.0 is being released with the --with-cxx11 capability at configure
 time.

 I chatted with some of the committers and we'd like to move forward with
 the final two phases of MESOS-750 as follows:

 0.18.0: --with-cxx11 will become the _default_. --without-cxx11 will be
 available for those who wish to use older compilers.

 0.19.0: c++11 required; Mesos will not build with a compiler that does not
 have c++11 support.

 If there are any objections please speak now. If not, I'll update the
 ticket to reflect the current path towards C++11.


 On Mon, Oct 28, 2013 at 8:09 PM, Benjamin Mahler
 benjamin.mah...@gmail.comwrote:

  Thanks for the heads up, we're likely to upgrade to 3.3.6, but I've yet
 to
  confirm that it builds with c++11.
 
 
  On Thu, Oct 24, 2013 at 8:10 AM, Tim St Clair tstcl...@redhat.com
 wrote:
 
  +1 to the approach.
 
  FWIW made a quick update to the zookeeper.spec (
  http://koji.fedoraproject.org/koji/buildinfo?buildID=465529) and
 updated:
 
  export CC='gcc -std=c11'
  export CXX='g++ -std=c++11'
 
  prior to 'configure', and it failed compilation pretty quickly.
  It may make sense to track integration issues and versions...somewhere.
 
  So when --with-cpp11 is enabled, it can hard check on version =
  fixed_version.
 
  Cheers,
  Tim
 
  - Original Message -
   From: Benjamin Mahler benjamin.mah...@gmail.com
   To: dev@mesos.apache.org
   Sent: Wednesday, October 23, 2013 4:46:49 PM
   Subject: Path forward to C++11.
  
   As discussed in MESOS-750 
  https://issues.apache.org/jira/browse/MESOS-750,
   we would like to move to C++11.
  
   The proposed plan is to do this in phases to allow everyone time to
  adjust
   to this change:
  
  
   Phase 1: Send an email to the dev@ list soliciting objections /
  feedback.
   We're currently here. :)
  
   Phase 2: Add an option --with-cpp11 that defaults to false.
  
   Phase 3: At this point, developers can and should begin using
  --with-cpp11.
   In this phase, we'll need to make sure the project and dependencies
 can
   compile with c++11.
  
   Phase 4: Default to --with-cpp11 but still leave the option in order
 to
   allow folks to turn this off if needed. At this point, we still cannot
  rely
   on C++11.
  
   Phase 5: Remove --with-cpp11 leaving no ability to turn it off. From
  this
   point forward, we can begin to use C++11 features unconditionally.
  
  
   Please reply to this thread with any objections, comments, or advice!
 To
   get a better sense of how others feel about this (I currently only
 know
   that some of the other core contributors are on board), please feel
  free to
   reply with something as simple as a +1 as well.
  
   Thanks!
   Ben
  
 
  --
  Cheers,
  Tim
 
 
 




-- 
Dominic Hamon | @mrdo | Twitter
*There are no bad ideas; only good ideas that go horribly wrong.*


Re: Review Request 18361: Implemented MESOS-945: Added hostname attribute to framework, display it in master's web ui

2014-02-26 Thread Vinod Kone

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18361/#review35560
---

Ship it!


Ship It!

- Vinod Kone


On Feb. 25, 2014, 2:16 p.m., Bernd Mathiske wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/18361/
 ---
 
 (Updated Feb. 25, 2014, 2:16 p.m.)
 
 
 Review request for mesos and Vinod Kone.
 
 
 Bugs: MESOS-945
 https://issues.apache.org/jira/browse/MESOS-945
 
 
 Repository: mesos-git
 
 
 Description
 ---
 
 - Added a field to FrameworkInfo in protobuf spec 
 - Added table fields in html of web ui that display framework's hostname for 
 running and terminated frameworks
 - Added line to scheduler constructor that sets framework hostname. This is 
 just about the first centralized piece of code that every framework executes.
 - Refactored scheduler constructor so that code duplication is avoided.
 
 
 Diffs
 -
 
   include/mesos/mesos.proto 69a4a60 
   include/mesos/scheduler.hpp 2e4707e 
   src/master/http.cpp c941274 
   src/sched/sched.cpp dcb3158 
   src/slave/http.cpp 7c4cfba 
   src/webui/master/static/frameworks.html 8686b00 
 
 Diff: https://reviews.apache.org/r/18361/diff/
 
 
 Testing
 ---
 
 Built (with Clang) and ran local Mesos on Mac. Ran C++ and Java test 
 frameworks. Observed new hostname display in web UI. Built Mesos on Ubuntu 
 13.10 with gcc 4.8.1.
 
 
 Thanks,
 
 Bernd Mathiske
 




Re: Review Request 18341: Modified the Registrar to notify callers only _after_ the storage operation completes.

2014-02-26 Thread Ben Mahler

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18341/
---

(Updated Feb. 26, 2014, 7:54 p.m.)


Review request for mesos, Benjamin Hindman and Vinod Kone.


Changes
---

Updates per benh review.


Bugs: MESOS-764
https://issues.apache.org/jira/browse/MESOS-764


Repository: mesos-git


Description
---

We previously were setting the Promises in some cases _before_ writing to 
storage. While this approach was correct, it requires some time and explanation 
to convince oneself of this.

This is why this patch simplifies the semantics by only setting the Promises 
_after_ we've written to storage.

Also, 'Mutation' has now become 'Operation' to better reflect the fact that 
some operations do not mutate state when applied.


Diffs (updated)
-

  src/master/registrar.cpp ee16121035db21d966ee151483dd23cbc4a495c2 

Diff: https://reviews.apache.org/r/18341/diff/


Testing
---

make check


Thanks,

Ben Mahler



Re: Review Request 18341: Modified the Registrar to notify callers only _after_ the storage operation completes.

2014-02-26 Thread Ben Mahler


 On Feb. 26, 2014, 6:04 p.m., Benjamin Hindman wrote:
  src/master/registrar.cpp, lines 79-83
  https://reviews.apache.org/r/18341/diff/3/?file=504150#file504150line79
 
  It looks like 'applied' is true iff 'operator ()' returns Some or None. 
  Can we simplify the semantics and not require setting 'applied'? What about 
  another wrapper that calls 'operator ()' and sets 'applied' based on the 
  result of 'operator ()'. Also, do you perceive any cases of where Error 
  won't turn into an 'applied = false' case? Finally, since maybe 
  s/applied/success/ since some operations aren't technically applied as 
  discussed in Vinod's previous review.

I've made some updates here. Now Operation is still a functor, but the () 
operator wraps a call to a protected 'apply' function. This is done in order to 
set 'success' based on the result of the 'apply' call. To answer your question, 
'success' is true iff !apply().isError() and I think this will generalize to 
all operations.

Now each Operation overrides the pure virtual apply() call and does not have to 
deal with setting the 'success' boolean, which is in private scope anyway and 
is thus hidden from the subclasses of Operation.

Let me know how it looks, it's a bit unfortunate that we've introduced a copy 
across () and apply().


- Ben


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18341/#review35536
---


On Feb. 26, 2014, 2:19 a.m., Ben Mahler wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/18341/
 ---
 
 (Updated Feb. 26, 2014, 2:19 a.m.)
 
 
 Review request for mesos, Benjamin Hindman and Vinod Kone.
 
 
 Bugs: MESOS-764
 https://issues.apache.org/jira/browse/MESOS-764
 
 
 Repository: mesos-git
 
 
 Description
 ---
 
 We previously were setting the Promises in some cases _before_ writing to 
 storage. While this approach was correct, it requires some time and 
 explanation to convince oneself of this.
 
 This is why this patch simplifies the semantics by only setting the Promises 
 _after_ we've written to storage.
 
 Also, 'Mutation' has now become 'Operation' to better reflect the fact that 
 some operations do not mutate state when applied.
 
 
 Diffs
 -
 
   src/master/registrar.cpp ee16121035db21d966ee151483dd23cbc4a495c2 
 
 Diff: https://reviews.apache.org/r/18341/diff/
 
 
 Testing
 ---
 
 make check
 
 
 Thanks,
 
 Ben Mahler
 




Re: Review Request 18370: Refactor Cluster::Master::start methods

2014-02-26 Thread Vinod Kone

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18370/#review35561
---



src/tests/cluster.hpp
https://reviews.apache.org/r/18370/#comment66163

s/normal production/default/ ?



src/tests/cluster.hpp
https://reviews.apache.org/r/18370/#comment66165

Why this name change? I rather liked the former.


- Vinod Kone


On Feb. 25, 2014, 9:23 p.m., Charlie Carson wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/18370/
 ---
 
 (Updated Feb. 25, 2014, 9:23 p.m.)
 
 
 Review request for mesos, Benjamin Hindman, Ben Mahler, and Vinod Kone.
 
 
 Bugs: MESOS-1024
 https://issues.apache.org/jira/browse/MESOS-1024
 
 
 Repository: mesos-git
 
 
 Description
 ---
 
 Refactor Cluster::Master::start methods
 
 There are currently two overloads of the Cluster::Masters::start
 function.  One takes a argument of AllocatorProcess* and the other
 does not.  The AllocatorProcess* overload serves two purposes:
 1) it allows an alternative implementation of AllocatorProcess to be
 passed (i.e. a mock)
 2) it changes the destruction timing so that the passed in argument
 can outlive the master.
 
 Beyond that, the two functions are identical.   This changes the
 parameter to be OptionAllocatorProcess* and allows all the logic
 to be in one method.  The old function exists for back-compat but
 now simply forward by passing in None() to the other function.
 
 This is for two purposes:
 1) reduce code duplication
 2) position the code so that we can also optionally pass in a mock
repaier.
 
 Review: https://reviews.apache.org/r/18370
 
 
 Diffs
 -
 
   src/tests/cluster.hpp d1bf680ed3f6a0fb16af85a21409d653d44522ca 
 
 Diff: https://reviews.apache.org/r/18370/diff/
 
 
 Testing
 ---
 
 make check
 
 
 Thanks,
 
 Charlie Carson
 




Re: Review Request 18370: Refactor Cluster::Master::start methods

2014-02-26 Thread Charlie Carson

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18370/
---

(Updated Feb. 26, 2014, 7:56 p.m.)


Review request for mesos and Vinod Kone.


Changes
---

assigning to myself (@vinodkone) for shepherding this.


Bugs: MESOS-1024
https://issues.apache.org/jira/browse/MESOS-1024


Repository: mesos-git


Description
---

Refactor Cluster::Master::start methods

There are currently two overloads of the Cluster::Masters::start
function.  One takes a argument of AllocatorProcess* and the other
does not.  The AllocatorProcess* overload serves two purposes:
1) it allows an alternative implementation of AllocatorProcess to be
passed (i.e. a mock)
2) it changes the destruction timing so that the passed in argument
can outlive the master.

Beyond that, the two functions are identical.   This changes the
parameter to be OptionAllocatorProcess* and allows all the logic
to be in one method.  The old function exists for back-compat but
now simply forward by passing in None() to the other function.

This is for two purposes:
1) reduce code duplication
2) position the code so that we can also optionally pass in a mock
   repaier.

Review: https://reviews.apache.org/r/18370


Diffs
-

  src/tests/cluster.hpp d1bf680ed3f6a0fb16af85a21409d653d44522ca 

Diff: https://reviews.apache.org/r/18370/diff/


Testing
---

make check


Thanks,

Charlie Carson



[jira] [Resolved] (MESOS-945) Show framework host name in the WebUI

2014-02-26 Thread Vinod Kone (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kone resolved MESOS-945.
--

Resolution: Fixed

commit f19db574d4e644beed88100b198354733706422d
Author: Bernd Mathiske be...@mesosphere.io
Date:   Wed Feb 26 11:45:57 2014 -0800

Added hostname attribute to framework and displayed it in master's web
ui.

Review: https://reviews.apache.org/r/18361


 Show framework host name in the WebUI
 -

 Key: MESOS-945
 URL: https://issues.apache.org/jira/browse/MESOS-945
 Project: Mesos
  Issue Type: Improvement
  Components: webui
Reporter: Vinod Kone
Assignee: Bernd Mathiske
  Labels: newbie
 Fix For: 0.19.0


 It would be great if we could see the hostname of the framework that is 
 currently registered with the master in the webui.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


Fwd: Path forward to C++11.

2014-02-26 Thread Dominic Hamon
Forwarding from the right account to avoid Spam filters ...

-- Forwarded message --
From: Dominic Hamon dha...@twitter.com
Date: Wed, Feb 26, 2014 at 11:46 AM
Subject: Re: Path forward to C++11.
To: dev@mesos.apache.org


What do you mean by 'C++11 support' in the compiler? g++4.7 supports some
features, not all.

I think we need to define the set of C++11 features we want to support
first before making any decisions regarding timeline. Then we can look at
the state of the nation in regards to C++11 feature support in different
compilers on different OSs, and which are available out of the box. We
don't want to limit our community involvement by shutting out certain
platforms for development.

Ideally, we'd be able to say something like 0.19.0 builds on g++4.7 and
clang 3.3 or greater and 0.20.0 builds on g++4.8 and clang 3.4 or
greater instead of blanket C++11 support.




On Wed, Feb 26, 2014 at 10:20 AM, Benjamin Mahler benjamin.mah...@gmail.com
 wrote:

 Update on MESOS-750:

 0.17.0 is being released with the --with-cxx11 capability at configure
 time.

 I chatted with some of the committers and we'd like to move forward with
 the final two phases of MESOS-750 as follows:

 0.18.0: --with-cxx11 will become the _default_. --without-cxx11 will be
 available for those who wish to use older compilers.

 0.19.0: c++11 required; Mesos will not build with a compiler that does not
 have c++11 support.

 If there are any objections please speak now. If not, I'll update the
 ticket to reflect the current path towards C++11.


 On Mon, Oct 28, 2013 at 8:09 PM, Benjamin Mahler
 benjamin.mah...@gmail.comwrote:

  Thanks for the heads up, we're likely to upgrade to 3.3.6, but I've yet
 to
  confirm that it builds with c++11.
 
 
  On Thu, Oct 24, 2013 at 8:10 AM, Tim St Clair tstcl...@redhat.com
 wrote:
 
  +1 to the approach.
 
  FWIW made a quick update to the zookeeper.spec (
  http://koji.fedoraproject.org/koji/buildinfo?buildID=465529) and
 updated:
 
  export CC='gcc -std=c11'
  export CXX='g++ -std=c++11'
 
  prior to 'configure', and it failed compilation pretty quickly.
  It may make sense to track integration issues and versions...somewhere.
 
  So when --with-cpp11 is enabled, it can hard check on version =
  fixed_version.
 
  Cheers,
  Tim
 
  - Original Message -
   From: Benjamin Mahler benjamin.mah...@gmail.com
   To: dev@mesos.apache.org
   Sent: Wednesday, October 23, 2013 4:46:49 PM
   Subject: Path forward to C++11.
  
   As discussed in MESOS-750 
  https://issues.apache.org/jira/browse/MESOS-750,
   we would like to move to C++11.
  
   The proposed plan is to do this in phases to allow everyone time to
  adjust
   to this change:
  
  
   Phase 1: Send an email to the dev@ list soliciting objections /
  feedback.
   We're currently here. :)
  
   Phase 2: Add an option --with-cpp11 that defaults to false.
  
   Phase 3: At this point, developers can and should begin using
  --with-cpp11.
   In this phase, we'll need to make sure the project and dependencies
 can
   compile with c++11.
  
   Phase 4: Default to --with-cpp11 but still leave the option in order
 to
   allow folks to turn this off if needed. At this point, we still cannot
  rely
   on C++11.
  
   Phase 5: Remove --with-cpp11 leaving no ability to turn it off. From
  this
   point forward, we can begin to use C++11 features unconditionally.
  
  
   Please reply to this thread with any objections, comments, or advice!
 To
   get a better sense of how others feel about this (I currently only
 know
   that some of the other core contributors are on board), please feel
  free to
   reply with something as simple as a +1 as well.
  
   Thanks!
   Ben
  
 
  --
  Cheers,
  Tim
 
 
 




-- 
Dominic Hamon | @mrdo | Twitter
*There are no bad ideas; only good ideas that go horribly wrong.*


Re: Review Request 18341: Modified the Registrar to notify callers only _after_ the storage operation completes.

2014-02-26 Thread Vinod Kone

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18341/#review35563
---

Ship it!



src/master/registrar.cpp
https://reviews.apache.org/r/18341/#comment66166

Does this have to be virtual?


- Vinod Kone


On Feb. 26, 2014, 7:54 p.m., Ben Mahler wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/18341/
 ---
 
 (Updated Feb. 26, 2014, 7:54 p.m.)
 
 
 Review request for mesos, Benjamin Hindman and Vinod Kone.
 
 
 Bugs: MESOS-764
 https://issues.apache.org/jira/browse/MESOS-764
 
 
 Repository: mesos-git
 
 
 Description
 ---
 
 We previously were setting the Promises in some cases _before_ writing to 
 storage. While this approach was correct, it requires some time and 
 explanation to convince oneself of this.
 
 This is why this patch simplifies the semantics by only setting the Promises 
 _after_ we've written to storage.
 
 Also, 'Mutation' has now become 'Operation' to better reflect the fact that 
 some operations do not mutate state when applied.
 
 
 Diffs
 -
 
   src/master/registrar.cpp ee16121035db21d966ee151483dd23cbc4a495c2 
 
 Diff: https://reviews.apache.org/r/18341/diff/
 
 
 Testing
 ---
 
 make check
 
 
 Thanks,
 
 Ben Mahler
 




Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME #1919

2014-02-26 Thread Apache Jenkins Server
See 
https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/1919/changes

Changes:

[niklas] Added new documentation page for frameworks.

--
[...truncated 4703 lines...]
I0226 19:18:08.511400 24627 master.cpp:2339] Queuing up authentication request 
from scheduler(1)@67.195.138.9:38323 because authentication is still in progress
I0226 19:18:08.932227 24633 master.cpp:96] No whitelist given. Advertising 
offers for all slaves
I0226 19:18:09.135534 24631 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 58907ns
I0226 19:18:10.135767 24634 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 31688ns
I0226 19:18:11.136059 24632 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 50719ns
I0226 19:18:12.137267 24633 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 32653ns
I0226 19:18:13.137533 24629 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 30145ns
W0226 19:18:13.511966 24629 sched.cpp:362] Authentication timed out
I0226 19:18:13.512111 24629 sched.cpp:322] Failed to authenticate with master 
master@67.195.138.9:38323: Authentication discarded
I0226 19:18:13.512151 24629 sched.cpp:267] Authenticating with master 
master@67.195.138.9:38323
I0226 19:18:13.512291 24627 authenticatee.hpp:128] Creating new client SASL 
connection
I0226 19:18:13.512517 24634 master.cpp:2339] Queuing up authentication request 
from scheduler(1)@67.195.138.9:38323 because authentication is still in progress
I0226 19:18:13.932324 24627 master.cpp:96] No whitelist given. Advertising 
offers for all slaves
I0226 19:18:14.138712 24629 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 52245ns
I0226 19:18:15.138974 24630 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 37203ns
I0226 19:18:16.139312 24628 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 38662ns
I0226 19:18:17.139726 24631 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 40628ns
I0226 19:18:18.139964 24627 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 29378ns
W0226 19:18:18.512387 24634 sched.cpp:362] Authentication timed out
I0226 19:18:18.512657 24629 sched.cpp:322] Failed to authenticate with master 
master@67.195.138.9:38323: Authentication discarded
I0226 19:18:18.512722 24629 sched.cpp:267] Authenticating with master 
master@67.195.138.9:38323
I0226 19:18:18.512908 24628 authenticatee.hpp:128] Creating new client SASL 
connection
I0226 19:18:18.513178 24629 master.cpp:2339] Queuing up authentication request 
from scheduler(1)@67.195.138.9:38323 because authentication is still in progress
I0226 19:18:18.932801 24632 master.cpp:96] No whitelist given. Advertising 
offers for all slaves
I0226 19:18:19.141152 24632 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 31871ns
I0226 19:18:20.141477 24632 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 29318ns
I0226 19:18:21.141680 24632 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 30149ns
I0226 19:18:22.141899 24632 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 29267ns
I0226 19:18:23.142094 24632 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 28831ns
W0226 19:18:23.513563 24632 sched.cpp:362] Authentication timed out
I0226 19:18:23.513694 24632 sched.cpp:322] Failed to authenticate with master 
master@67.195.138.9:38323: Authentication discarded
I0226 19:18:23.513725 24632 sched.cpp:267] Authenticating with master 
master@67.195.138.9:38323
I0226 19:18:23.513818 24632 authenticatee.hpp:128] Creating new client SASL 
connection
I0226 19:18:23.513983 24632 master.cpp:2339] Queuing up authentication request 
from scheduler(1)@67.195.138.9:38323 because authentication is still in progress
I0226 19:18:23.933203 24627 master.cpp:96] No whitelist given. Advertising 
offers for all slaves
I0226 19:18:24.142500 24627 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 35792ns
I0226 19:18:25.142745 24633 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 25794ns
I0226 19:18:26.143082 24633 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 31622ns
I0226 19:18:27.143302 24633 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 25548ns
I0226 19:18:28.143506 24633 hierarchical_allocator_process.hpp:688] Performed 
allocation for 1 slaves in 26022ns
W0226 19:18:28.513929 24633 sched.cpp:362] Authentication timed out
I0226 19:18:28.514027 24633 sched.cpp:322] Failed to authenticate with master 
master@67.195.138.9:38323: Authentication discarded
I0226 19:18:28.514053 24633 sched.cpp:267] Authenticating with master 

Review Request 18532: Fix ROOT_CGROUPS_Write test.

2014-02-26 Thread Ian Downes

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18532/
---

Review request for mesos, Ben Mahler and Vinod Kone.


Repository: mesos-git


Description
---

Fixed test by moving cgroups::create before fork so only the parent creates it.

Also cleaned up the code paths for the parent/child.


Diffs
-

  src/tests/cgroups_tests.cpp 6a111e500b3824b87569d6014f966fb35e62ccd0 

Diff: https://reviews.apache.org/r/18532/diff/


Testing
---

make check # linux


Thanks,

Ian Downes



Jenkins build is back to normal : Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME #1920

2014-02-26 Thread Apache Jenkins Server
See 
https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/1920/changes



Re: Review Request 18532: Fix ROOT_CGROUPS_Write test.

2014-02-26 Thread Ian Downes

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18532/
---

(Updated Feb. 26, 2014, 9:55 p.m.)


Review request for mesos and Vinod Kone.


Changes
---

assigned to myself (@vinodkone) for review.


Repository: mesos-git


Description
---

Fixed test by moving cgroups::create before fork so only the parent creates it.

Also cleaned up the code paths for the parent/child.


Diffs
-

  src/tests/cgroups_tests.cpp 6a111e500b3824b87569d6014f966fb35e62ccd0 

Diff: https://reviews.apache.org/r/18532/diff/


Testing
---

make check # linux


Thanks,

Ian Downes



Re: Review Request 18532: Fix ROOT_CGROUPS_Write test.

2014-02-26 Thread Vinod Kone

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18532/#review35568
---

Ship it!


Is there a bug that needs to be attached?


src/tests/cgroups_tests.cpp
https://reviews.apache.org/r/18532/#comment66175

align errno with write ?



- Vinod Kone


On Feb. 26, 2014, 8:19 p.m., Ian Downes wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/18532/
 ---
 
 (Updated Feb. 26, 2014, 8:19 p.m.)
 
 
 Review request for mesos, Ben Mahler and Vinod Kone.
 
 
 Repository: mesos-git
 
 
 Description
 ---
 
 Fixed test by moving cgroups::create before fork so only the parent creates 
 it.
 
 Also cleaned up the code paths for the parent/child.
 
 
 Diffs
 -
 
   src/tests/cgroups_tests.cpp 6a111e500b3824b87569d6014f966fb35e62ccd0 
 
 Diff: https://reviews.apache.org/r/18532/diff/
 
 
 Testing
 ---
 
 make check # linux
 
 
 Thanks,
 
 Ian Downes
 




Re: Path forward to C++11.

2014-02-26 Thread Benjamin Mahler
Being explicit about compiler version requirements sounds great!

GCC is still our explicitly supported compiler toolchain and it's how we
test releases for production purposes, at least at Twitter.

We have been supporting Clang as a best effort so far, but I agree it would
be great to document supported versions. Taking a look at some of our CI, I
see Clang 3.3. is building Mesos.

 Looking at CI for gcc, I see the following building with C++11:

*Mesos builds with C++11:*
CentOS 5 - with devtoolset-2 gcc 4.8.1
CentOS 6 - with devtoolset-2 gcc 4.8.1
Fedora 19 - gcc 4.8.2
Fedora 20 - gcc 4.8.2
Ubuntu 13.10 - gcc 4.8.x

*Mesos does not build with C++11:*
CentOS 5 - gcc 4.1.2
CentOS 6 - gcc 4.4.7
Ubuntu 12.10 - gcc 4.7.x
Ubuntu 12.04 - gcc 4.6.x

So we already do not support gcc 4.7 for C++11.

It's important here to make the distinction that this is for development,
ideally those are running from RPMs are agnostic to the gcc version used.
But, I suppose the c++11 shared library will not present on the machine.

On Wed, Feb 26, 2014 at 12:08 PM, Dominic Hamon dha...@twopensource.comwrote:

 Forwarding from the right account to avoid Spam filters ...

 -- Forwarded message --
 From: Dominic Hamon dha...@twitter.com
 Date: Wed, Feb 26, 2014 at 11:46 AM
 Subject: Re: Path forward to C++11.
 To: dev@mesos.apache.org


 What do you mean by 'C++11 support' in the compiler? g++4.7 supports some
 features, not all.

 I think we need to define the set of C++11 features we want to support
 first before making any decisions regarding timeline. Then we can look at
 the state of the nation in regards to C++11 feature support in different
 compilers on different OSs, and which are available out of the box. We
 don't want to limit our community involvement by shutting out certain
 platforms for development.

 Ideally, we'd be able to say something like 0.19.0 builds on g++4.7 and
 clang 3.3 or greater and 0.20.0 builds on g++4.8 and clang 3.4 or
 greater instead of blanket C++11 support.




 On Wed, Feb 26, 2014 at 10:20 AM, Benjamin Mahler 
 benjamin.mah...@gmail.com
  wrote:

  Update on MESOS-750:
 
  0.17.0 is being released with the --with-cxx11 capability at configure
  time.
 
  I chatted with some of the committers and we'd like to move forward with
  the final two phases of MESOS-750 as follows:
 
  0.18.0: --with-cxx11 will become the _default_. --without-cxx11 will be
  available for those who wish to use older compilers.
 
  0.19.0: c++11 required; Mesos will not build with a compiler that does
 not
  have c++11 support.
 
  If there are any objections please speak now. If not, I'll update the
  ticket to reflect the current path towards C++11.
 
 
  On Mon, Oct 28, 2013 at 8:09 PM, Benjamin Mahler
  benjamin.mah...@gmail.comwrote:
 
   Thanks for the heads up, we're likely to upgrade to 3.3.6, but I've yet
  to
   confirm that it builds with c++11.
  
  
   On Thu, Oct 24, 2013 at 8:10 AM, Tim St Clair tstcl...@redhat.com
  wrote:
  
   +1 to the approach.
  
   FWIW made a quick update to the zookeeper.spec (
   http://koji.fedoraproject.org/koji/buildinfo?buildID=465529) and
  updated:
  
   export CC='gcc -std=c11'
   export CXX='g++ -std=c++11'
  
   prior to 'configure', and it failed compilation pretty quickly.
   It may make sense to track integration issues and
 versions...somewhere.
  
   So when --with-cpp11 is enabled, it can hard check on version =
   fixed_version.
  
   Cheers,
   Tim
  
   - Original Message -
From: Benjamin Mahler benjamin.mah...@gmail.com
To: dev@mesos.apache.org
Sent: Wednesday, October 23, 2013 4:46:49 PM
Subject: Path forward to C++11.
   
As discussed in MESOS-750 
   https://issues.apache.org/jira/browse/MESOS-750,
we would like to move to C++11.
   
The proposed plan is to do this in phases to allow everyone time to
   adjust
to this change:
   
   
Phase 1: Send an email to the dev@ list soliciting objections /
   feedback.
We're currently here. :)
   
Phase 2: Add an option --with-cpp11 that defaults to false.
   
Phase 3: At this point, developers can and should begin using
   --with-cpp11.
In this phase, we'll need to make sure the project and dependencies
  can
compile with c++11.
   
Phase 4: Default to --with-cpp11 but still leave the option in order
  to
allow folks to turn this off if needed. At this point, we still
 cannot
   rely
on C++11.
   
Phase 5: Remove --with-cpp11 leaving no ability to turn it off. From
   this
point forward, we can begin to use C++11 features unconditionally.
   
   
Please reply to this thread with any objections, comments, or
 advice!
  To
get a better sense of how others feel about this (I currently only
  know
that some of the other core contributors are on board), please feel
   free to
reply with something as simple as a +1 as well.
   
Thanks!
Ben
   
  
   --
   Cheers,
   Tim
  
  
  
 



 --
 

Re: Path forward to C++11.

2014-02-26 Thread Dominic Hamon
Thank you for the breakdown, that's really helpful.

It looks like with 4.8.1 as the minimum supported version, we're looking
very good for all features: http://gcc.gnu.org/gcc-4.8/cxx0x_status.html

We may need to consider distributing the runtime library or statically
linking it to allow users to run the binary from the rpm.


On Wed, Feb 26, 2014 at 1:59 PM, Benjamin Mahler
benjamin.mah...@gmail.comwrote:

 Being explicit about compiler version requirements sounds great!

 GCC is still our explicitly supported compiler toolchain and it's how we
 test releases for production purposes, at least at Twitter.

 We have been supporting Clang as a best effort so far, but I agree it would
 be great to document supported versions. Taking a look at some of our CI, I
 see Clang 3.3. is building Mesos.

  Looking at CI for gcc, I see the following building with C++11:

 *Mesos builds with C++11:*
 CentOS 5 - with devtoolset-2 gcc 4.8.1
 CentOS 6 - with devtoolset-2 gcc 4.8.1
 Fedora 19 - gcc 4.8.2
 Fedora 20 - gcc 4.8.2
 Ubuntu 13.10 - gcc 4.8.x

 *Mesos does not build with C++11:*
 CentOS 5 - gcc 4.1.2
 CentOS 6 - gcc 4.4.7
 Ubuntu 12.10 - gcc 4.7.x
 Ubuntu 12.04 - gcc 4.6.x

 So we already do not support gcc 4.7 for C++11.

 It's important here to make the distinction that this is for development,
 ideally those are running from RPMs are agnostic to the gcc version used.
 But, I suppose the c++11 shared library will not present on the machine.

 On Wed, Feb 26, 2014 at 12:08 PM, Dominic Hamon dha...@twopensource.com
 wrote:

  Forwarding from the right account to avoid Spam filters ...
 
  -- Forwarded message --
  From: Dominic Hamon dha...@twitter.com
  Date: Wed, Feb 26, 2014 at 11:46 AM
  Subject: Re: Path forward to C++11.
  To: dev@mesos.apache.org
 
 
  What do you mean by 'C++11 support' in the compiler? g++4.7 supports some
  features, not all.
 
  I think we need to define the set of C++11 features we want to support
  first before making any decisions regarding timeline. Then we can look at
  the state of the nation in regards to C++11 feature support in different
  compilers on different OSs, and which are available out of the box. We
  don't want to limit our community involvement by shutting out certain
  platforms for development.
 
  Ideally, we'd be able to say something like 0.19.0 builds on g++4.7 and
  clang 3.3 or greater and 0.20.0 builds on g++4.8 and clang 3.4 or
  greater instead of blanket C++11 support.
 
 
 
 
  On Wed, Feb 26, 2014 at 10:20 AM, Benjamin Mahler 
  benjamin.mah...@gmail.com
   wrote:
 
   Update on MESOS-750:
  
   0.17.0 is being released with the --with-cxx11 capability at configure
   time.
  
   I chatted with some of the committers and we'd like to move forward
 with
   the final two phases of MESOS-750 as follows:
  
   0.18.0: --with-cxx11 will become the _default_. --without-cxx11 will be
   available for those who wish to use older compilers.
  
   0.19.0: c++11 required; Mesos will not build with a compiler that does
  not
   have c++11 support.
  
   If there are any objections please speak now. If not, I'll update the
   ticket to reflect the current path towards C++11.
  
  
   On Mon, Oct 28, 2013 at 8:09 PM, Benjamin Mahler
   benjamin.mah...@gmail.comwrote:
  
Thanks for the heads up, we're likely to upgrade to 3.3.6, but I've
 yet
   to
confirm that it builds with c++11.
   
   
On Thu, Oct 24, 2013 at 8:10 AM, Tim St Clair tstcl...@redhat.com
   wrote:
   
+1 to the approach.
   
FWIW made a quick update to the zookeeper.spec (
http://koji.fedoraproject.org/koji/buildinfo?buildID=465529) and
   updated:
   
export CC='gcc -std=c11'
export CXX='g++ -std=c++11'
   
prior to 'configure', and it failed compilation pretty quickly.
It may make sense to track integration issues and
  versions...somewhere.
   
So when --with-cpp11 is enabled, it can hard check on version =
fixed_version.
   
Cheers,
Tim
   
- Original Message -
 From: Benjamin Mahler benjamin.mah...@gmail.com
 To: dev@mesos.apache.org
 Sent: Wednesday, October 23, 2013 4:46:49 PM
 Subject: Path forward to C++11.

 As discussed in MESOS-750 
https://issues.apache.org/jira/browse/MESOS-750,
 we would like to move to C++11.

 The proposed plan is to do this in phases to allow everyone time
 to
adjust
 to this change:


 Phase 1: Send an email to the dev@ list soliciting objections /
feedback.
 We're currently here. :)

 Phase 2: Add an option --with-cpp11 that defaults to false.

 Phase 3: At this point, developers can and should begin using
--with-cpp11.
 In this phase, we'll need to make sure the project and
 dependencies
   can
 compile with c++11.

 Phase 4: Default to --with-cpp11 but still leave the option in
 order
   to
 allow folks to turn this off if needed. At this point, we still
  cannot

Review Request 18540: assign bool directly to JSON::Value

2014-02-26 Thread Charlie Carson

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18540/
---

Review request for mesos and Benjamin Hindman.


Repository: mesos-git


Description
---

assign bool directly to JSON::Value

This change removes the workaround that was necessary now that
https://issues.apache.org/jira/browse/MESOS-939 is committed.

Review: https://reviews.apache.org/r/18540


Diffs
-

  src/master/http.cpp e64093c2aa9d54ad615050ed0339ab0d6d4c6e41 
  src/tests/repair_tests.cpp ba6f50ad67ebe834ae4af5fc10bd39270ce91471 

Diff: https://reviews.apache.org/r/18540/diff/


Testing
---

make check


Thanks,

Charlie Carson



Re: Review Request 18540: assign bool directly to JSON::Value

2014-02-26 Thread Dominic Hamon

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18540/#review35571
---

Ship it!


Ship It!

- Dominic Hamon


On Feb. 26, 2014, 2:14 p.m., Charlie Carson wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/18540/
 ---
 
 (Updated Feb. 26, 2014, 2:14 p.m.)
 
 
 Review request for mesos and Benjamin Hindman.
 
 
 Repository: mesos-git
 
 
 Description
 ---
 
 assign bool directly to JSON::Value
 
 This change removes the workaround that was necessary now that
 https://issues.apache.org/jira/browse/MESOS-939 is committed.
 
 Review: https://reviews.apache.org/r/18540
 
 
 Diffs
 -
 
   src/master/http.cpp e64093c2aa9d54ad615050ed0339ab0d6d4c6e41 
   src/tests/repair_tests.cpp ba6f50ad67ebe834ae4af5fc10bd39270ce91471 
 
 Diff: https://reviews.apache.org/r/18540/diff/
 
 
 Testing
 ---
 
 make check
 
 
 Thanks,
 
 Charlie Carson
 




[jira] [Commented] (MESOS-890) Figure out a way to migrate a live Mesos cluster to a different ZooKeeper cluster

2014-02-26 Thread David Robinson (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13913694#comment-13913694
 ] 

David Robinson commented on MESOS-890:
--

I think option #1 is the better approach. Writing a heap of code to correctly 
handle option #2 seems like a lot of effort for something which should very 
rarely happen. IIRC the 75 second constraint can be adjusted in Aurora too. 
[~rgs], how difficult is #1 from the ZK perspective? I'm happy to test this in 
a dev/test cluster.

 Figure out a way to migrate a live Mesos cluster to a different ZooKeeper 
 cluster
 -

 Key: MESOS-890
 URL: https://issues.apache.org/jira/browse/MESOS-890
 Project: Mesos
  Issue Type: Improvement
  Components: master
Reporter: Raul Gutierrez Segales
Assignee: Raul Gutierrez Segales

 I've been chatting with [~vinodkone] about approaching a live ZK cluster 
 migration. Here are the options we came up with.
 For the descriptions we treat `zk1` as the current working cluster, `obs` as 
 a  bunch of ZooKeeper Observers [1] and `zk2` as the new cluster to which we 
 need to migrate. 
 Approach #1: Using Observers
 With this option we need to:
 * add obs to zk1
 * restart slaves to have them use obs to find their master
 * restart the framework having it use obs to find the mesos master
 * restart the mesos masters having them use obs to perform their election
 * we then stop all ZK obs and remove their data (since they will need to sync 
 up with an entirely new cluster, we need to lose the old data)
 * we restart ZK obs having them be part of zk2
 * at this point the slaves, the framework and the masters can reach the ZK 
 obs again and an election happens
 * optionally you can restart slaves, the framework and masters again using 
 zk2 instead of the ZK obs if you wanted to decommission them. 
 This assumes that we can do the last three steps in  75 secs (75 secs being 
 the slave health check timeout). This is a reasonable assumption if the data 
 size in zk2 is small enough to ensure that the ZK obs can sync up quickly 
 with zk2. If zk2 is a new cluster with no data then this should be very fast.
 The good things of this approach are:
 * no mesos code change
 * it is very easy to rollback half way through, if need be
 The hard issues are:
 * Manipulating the ZK obs (i.e.: stopping, removing the data from zk1 and 
 starting again) needs to be done with care. Messing up configs or not 
 removing the data from zk1 on any of the ZK obs will cause problems
 * we need to restart all slaves to have them use the ZK obs instead of 
 connecting to zk1 directly. But with slave recovery this isn't an issue, just 
 an extra step.
 * same thing for the framework and the masters
 Approach #2: Dual publishing from mesos masters
 With this option we would augment the election handling code in mesos masters 
 to have it deal with the notion of a primary and secondary ZK clusters. 
 Master registration and election would then work as follows:
 * create an ephemeral|sequential znode in zk1 (i.e.:  
 /path/to/znode/mesos_23)
 * create an ephemeral, but not sequential, znode in zk2 with the exact same 
 path as what was created in zk1 (i.e.: /path/to/znode/mesos_23)
 * make sure both sessions, in zk1 and zk2, are always in the same state 
 (i.e.: if one expires, the other one should be closed, etc.)
 For now, lets omit a few implementation details which might need extra care 
 and assume we can make this work consistently in such a way that zk2 reflects 
 accurately elections that happen in zk1. This means that regardless of being 
 connected to zk1 or zk2, you always get the same master. Once we have this 
 the migration steps would be:
 * restart slaves to have them use zk2 where masters can be found by virtue of 
 what we implemented above
 * restart the framework so that it finds the mesos master in zk2
 * stop all mesos masters (they all need to be stopped before moving to the 
 next step)
 * start all mesos masters using zk2 as its primary and only cluster
 Again, this assumes we can do the last two steps in  75 secs (or if we 
 needed to, we could bump the slave health check timeout). Which, again, 
 sounds achievable given that masters have no state and their start-up time is 
 very short.
 The good things of this approach are:
 - no tinkering with extra ZK servers nor with ZK configs 
 The hard issues are:
 - extra code needs to be added to the election handling bits of mesos master 
 to address a very rare, but probable, use-case of cluster migration. It might 
 take a bit of time to get that code right. 
 - it's easier to end up with a bad state if any of the mesos masters ends up 
 with a bad config or is restarted earlier and ends up publishing differently 
 than 

Re: Review Request 18532: Fix ROOT_CGROUPS_Write test.

2014-02-26 Thread Ian Downes

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18532/
---

(Updated Feb. 27, 2014, 12:04 a.m.)


Review request for mesos and Vinod Kone.


Bugs: MESOS-1042
https://issues.apache.org/jira/browse/MESOS-1042


Repository: mesos-git


Description
---

Fixed test by moving cgroups::create before fork so only the parent creates it.

Also cleaned up the code paths for the parent/child.


Diffs
-

  src/tests/cgroups_tests.cpp 6a111e500b3824b87569d6014f966fb35e62ccd0 

Diff: https://reviews.apache.org/r/18532/diff/


Testing
---

make check # linux


Thanks,

Ian Downes



Re: Review Request 18386: Option reference cleanup in mesos

2014-02-26 Thread Ben Mahler


 On Feb. 24, 2014, 11:52 p.m., Ben Mahler wrote:
  src/linux/fs.hpp, line 63
  https://reviews.apache.org/r/18386/diff/2/?file=500797#file500797line63
 
  Well I see how you stumbled upon this fix.. :)
  
  Does this mean you did a grep to ensure all of the Option usage doesn't 
  use the reference after the Option is re-assigned?
 
 Dominic Hamon wrote:
 I did not. I'll do that after posting this just to make sure (though all 
 tests pass.. that means something, right? ;))

Let me know when you've done the search (might be good to spell this out in the 
'Testing Done').


- Ben


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18386/#review35338
---


On Feb. 25, 2014, 12:10 a.m., Dominic Hamon wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/18386/
 ---
 
 (Updated Feb. 25, 2014, 12:10 a.m.)
 
 
 Review request for mesos and Ben Mahler.
 
 
 Bugs: MESOS-1008
 https://issues.apache.org/jira/browse/MESOS-1008
 
 
 Repository: mesos-git
 
 
 Description
 ---
 
 See summary
 
 
 Diffs
 -
 
   src/linux/fs.hpp 1d86dd0d24c3daae957b5eec387638d1e8e6d7db 
   src/linux/fs.cpp e5f4f9a16becd4e5960d0cbb7f988736188b2426 
   src/log/log.cpp 62dc9286285d5981aa5fc63e125d3c3f51d4f457 
   src/sched/sched.cpp dcb3158d2acd93a107fda51b19e92899a092e628 
 
 Diff: https://reviews.apache.org/r/18386/diff/
 
 
 Testing
 ---
 
 make check
 
 
 Thanks,
 
 Dominic Hamon
 




[jira] [Created] (MESOS-1045) Unrecognized file extension in CommandInfo.URI causes executor to exit

2014-02-26 Thread Vinod Kone (JIRA)
Vinod Kone created MESOS-1045:
-

 Summary: Unrecognized file extension in CommandInfo.URI causes 
executor to exit
 Key: MESOS-1045
 URL: https://issues.apache.org/jira/browse/MESOS-1045
 Project: Mesos
  Issue Type: Bug
Affects Versions: 0.18.0
Reporter: Vinod Kone
 Fix For: 0.18.0


Previously only files with known extensions (zip and [compressed] tar) were 
extracted.

Since 0.18.0, unknown file extensions causes the fetcher to exit. This is a 
regression because 1) it is not the intended behavior per our documentation in 
mesos.proto and 2) it breaks Jenkins framework which uses a jar file with 
command executor.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


Review Request 18565: Fixed fetcher to ignore unrecognized file names.

2014-02-26 Thread Vinod Kone

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18565/
---

Review request for mesos and Ian Downes.


Bugs: MESOS-1045
https://issues.apache.org/jira/browse/MESOS-1045


Repository: mesos-git


Description
---

See bug for details.


Diffs
-

  src/launcher/fetcher.cpp 9c9f07d11b69fb08d435ea130f4bc53148ed7036 

Diff: https://reviews.apache.org/r/18565/diff/


Testing
---

make check


Thanks,

Vinod Kone



[jira] [Assigned] (MESOS-1045) Unrecognized file extension in CommandInfo.URI causes executor to exit

2014-02-26 Thread Vinod Kone (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kone reassigned MESOS-1045:
-

Assignee: Vinod Kone

 Unrecognized file extension in CommandInfo.URI causes executor to exit
 --

 Key: MESOS-1045
 URL: https://issues.apache.org/jira/browse/MESOS-1045
 Project: Mesos
  Issue Type: Bug
Affects Versions: 0.18.0
Reporter: Vinod Kone
Assignee: Vinod Kone
 Fix For: 0.18.0


 Previously only files with known extensions (zip and [compressed] tar) were 
 extracted.
 Since 0.18.0, unknown file extensions causes the fetcher to exit. This is a 
 regression because 1) it is not the intended behavior per our documentation 
 in mesos.proto and 2) it breaks Jenkins framework which uses a jar file 
 with command executor.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


[jira] [Commented] (MESOS-1045) Unrecognized file extension in CommandInfo.URI causes executor to exit

2014-02-26 Thread Vinod Kone (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13914106#comment-13914106
 ] 

Vinod Kone commented on MESOS-1045:
---

https://reviews.apache.org/r/18565/

 Unrecognized file extension in CommandInfo.URI causes executor to exit
 --

 Key: MESOS-1045
 URL: https://issues.apache.org/jira/browse/MESOS-1045
 Project: Mesos
  Issue Type: Bug
Affects Versions: 0.18.0
Reporter: Vinod Kone
 Fix For: 0.18.0


 Previously only files with known extensions (zip and [compressed] tar) were 
 extracted.
 Since 0.18.0, unknown file extensions causes the fetcher to exit. This is a 
 regression because 1) it is not the intended behavior per our documentation 
 in mesos.proto and 2) it breaks Jenkins framework which uses a jar file 
 with command executor.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)