date:20140731

Re: Review Request 24123: Fixed a flaky test: ZooKeeperTest.LeaderDetectorTimeoutHandling

2014-07-31 Thread Jiang Yan Xu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24123/
---

(Updated July 31, 2014, 11:23 p.m.)


Review request for mesos and Ben Mahler.


Changes
---

Split the the test into two and removed the part that waits several seconds for 
ZooKeeperTestServer to expire the session which is unnecessary to test the 
detector and slows down the test.


Summary (updated)
-

Fixed a flaky test: ZooKeeperTest.LeaderDetectorTimeoutHandling


Bugs: MESOS-1655
https://issues.apache.org/jira/browse/MESOS-1655


Repository: mesos-git


Description (updated)
---

Fixed a flaky test: ZooKeeperTest.LeaderDetectorTimeoutHandling

- The original ZK session timeout was the same as AWAIT_READY timeout so it's 
possible that AWAIT_READY timed out in a race.
- Split the the test into two and removed the part that waits several seconds 
for ZooKeeperTestServer to expire the session which is unnecessary to test the 
detector and slows down the test.


Diffs (updated)
-

  src/tests/zookeeper_tests.cpp be9fa06818b96e5170c68810fe16cc472f1f8b28 

Diff: https://reviews.apache.org/r/24123/diff/


Testing (updated)
---

Ran the test for 2000 iterations.


Thanks,

Jiang Yan Xu

Review Request 24178: Basic filesystem isolator for Linux.

2014-07-31 Thread Ian Downes


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24178/
---

Review request for mesos, Ben Mahler and Jie Yu.


Bugs: MESOS-1586
https://issues.apache.org/jira/browse/MESOS-1586


Repository: mesos-git


Description
---

Does not report usage or enforce quota but can create 'private' directories for 
each container which mask parts of the root filesystem.


Diffs
-

  src/Makefile.am b660d912c92594ef679b71caf508134f20511dae 
  src/slave/containerizer/isolators/filesystem.hpp PRE-CREATION 
  src/slave/containerizer/isolators/filesystem.cpp PRE-CREATION 
  src/slave/containerizer/linux_launcher.cpp 
f3cc813c64e9298f43c9691de9d51ba29c3a8e91 
  src/slave/containerizer/mesos/containerizer.cpp 
2c394e2c8702166266f5d20ff005abb218da8a6c 
  src/slave/flags.hpp 146c4011eb6a64dbb03555cece1c5303338ae240 
  src/tests/isolator_tests.cpp c38f87632cb6984543cb3767dbd656cde7459610 

Diff: https://reviews.apache.org/r/24178/diff/


Testing
---

# added two tests for the FilesystemIsolator
make check


Thanks,

Ian Downes

Review Request 24177: Pass executor directory to Isolator::prepare().

2014-07-31 Thread Ian Downes


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24177/
---

Review request for mesos, Ben Mahler and Jie Yu.


Repository: mesos-git


Description
---

Pass executor directory to Isolator::prepare().

Will be used for FilesystemIsolator.


Diffs
-

  src/slave/containerizer/isolator.hpp e52e8b15c740c62ef64b49897d3d6ae5179d4719 
  src/slave/containerizer/isolator.cpp 5e61bf2e3cf14be53d41aa657b4a78ab2dd6ecb0 
  src/slave/containerizer/isolators/cgroups/cpushare.hpp 
780037b093cb5b61254e61ddcfb9ecd4b0551a77 
  src/slave/containerizer/isolators/cgroups/cpushare.cpp 
3265a8084dfab6f3c9244f8f227e4024d8afe68a 
  src/slave/containerizer/isolators/cgroups/mem.hpp 
8c476c7847baa923821bc89a1e04f50f019c6ecb 
  src/slave/containerizer/isolators/cgroups/mem.cpp 
e8d1e35a1acdb3beb03a91cfd734a59417657b20 
  src/slave/containerizer/isolators/cgroups/perf_event.hpp 
4ceb07a6e4f171788cfbabca78c14ae23ff183db 
  src/slave/containerizer/isolators/cgroups/perf_event.cpp 
6f65b722c5b22a71a7c334c2b63db1584cb9fe29 
  src/slave/containerizer/isolators/posix.hpp 
17bbd10c565aa88048ec3a845014ceee582396d2 
  src/slave/containerizer/mesos/containerizer.cpp 
2c394e2c8702166266f5d20ff005abb218da8a6c 
  src/tests/isolator.hpp 89df4c4959c680354b002fa12e3a270a358087af 
  src/tests/isolator_tests.cpp c38f87632cb6984543cb3767dbd656cde7459610 

Diff: https://reviews.apache.org/r/24177/diff/


Testing
---

make check


Thanks,

Ian Downes

Jenkins build is back to normal : Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME #2279

2014-07-31 Thread Apache Jenkins Server

See

Re: A question on deploying mesos framework to the cluster

2014-07-31 Thread John Yang

Thanks, I’ll look into it.

Best,
John

On Aug 1, 2014, at 8:52 AM, Vinod Kone  wrote:

>> I looked at Marathon and Aurora and still couldn’t find a way to register
>> a Framework so that the Scheduler component runs on a Slave node.
>> 
> 
> You would want the scheduler code to be stored in a (shared) storage area
> accessible by the slave. The scheduler could be launched as a task of
> marathon/aurora. See
> http://www.ebaytechblog.com/2014/04/04/delivering-ebays-ci-solution-with-apache-mesos-part-i/#.U9rWN41dU7E
> for an example.
> 
> 
>> 
>> By any chance, is there a native support or an external add-on that I can
>> use?
>> 
> 
> There is no native or add-on support to launch frameworks by Mesos. The
> current solution is to launch a (meta) framework like aurora/marathon
> manually and then use that framework to launch other frameworks.

Re: Review Request 23771: Added a Docker containerizer.

2014-07-31 Thread Jie Yu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23771/#review49320
---

Ship it!


LGTM! Please rebase to the master and I'll get it committed tomorrow! Thanks 
Tim!


src/docker/docker.hpp


I guess we have some mis-communication last time. I want these function to 
be static to the cpp file (not static to this class) so that you don't need to 
declare them in the header file.



src/docker/docker.cpp


Add a comment about this helper:

// Asynchronously read stderr from subprocess.



src/docker/docker.cpp


Add a comment about this helper. Say 'Return a failure if error occurs'.

Also, checkError seems to be a more appropriate name



src/docker/docker.cpp


Add a TODO here saying:

// TODO: Consider returning stdout as well.



src/docker/docker.cpp


The semantics here is that you'll return the result of 'docker.rm' if 
'kill' failed and 'remove=true', right? If yes, please document this somewhere.


- Jie Yu


On Aug. 1, 2014, 12:46 a.m., Timothy Chen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23771/
> ---
> 
> (Updated Aug. 1, 2014, 12:46 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, Ian Downes, and Jie Yu.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Docker implementation.
> This is all the docker code Ben, I and Yifan worked on excluding the 
> composing containerizer patches.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am 45afcd1 
>   src/common/status_utils.hpp c51a8c6 
>   src/docker/docker.hpp PRE-CREATION 
>   src/docker/docker.cpp PRE-CREATION 
>   src/examples/docker_no_executor_framework.cpp PRE-CREATION 
>   src/health-check/main.cpp 10d57a0 
>   src/launcher/executor.cpp 948673e 
>   src/linux/cgroups.hpp decad9d 
>   src/linux/cgroups.cpp af6be22 
>   src/master/master.cpp 77f2536 
>   src/slave/containerizer/containerizer.cpp 1b71f33 
>   src/slave/containerizer/docker.hpp PRE-CREATION 
>   src/slave/containerizer/docker.cpp PRE-CREATION 
>   src/slave/containerizer/external_containerizer.cpp 3f28d85 
>   src/slave/containerizer/isolators/cgroups/cpushare.hpp 780037b 
>   src/slave/containerizer/isolators/cgroups/cpushare.cpp 3265a80 
>   src/slave/containerizer/isolators/cgroups/mem.hpp 8c476c7 
>   src/slave/containerizer/isolators/cgroups/mem.cpp e8d1e35 
>   src/slave/containerizer/isolators/posix.hpp 17bbd10 
>   src/slave/flags.hpp 1fe7b7d 
>   src/slave/slave.cpp f42ab60 
>   src/tests/cgroups_tests.cpp 01cf498 
>   src/tests/docker_containerizer_tests.cpp PRE-CREATION 
>   src/tests/docker_tests.cpp PRE-CREATION 
>   src/tests/environment.cpp 551698f 
>   src/tests/flags.hpp a003e7f 
>   src/tests/script.cpp 0d29c6d 
>   src/usage/usage.hpp 5a76746 
>   src/usage/usage.cpp 29014d1 
> 
> Diff: https://reviews.apache.org/r/23771/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Timothy Chen
> 
>

Build failed in Jenkins: Mesos-Ubuntu-distcheck #245

2014-07-31 Thread Apache Jenkins Server

See 

Changes:

[yujie.jay] Fixed the build error in port mapping tests.

[bmahler] Fixed a flaky slave recovery test.

--
[...truncated 4199 lines...]

[--] 4 tests from Sequence
[ RUN  ] Sequence.Random
[   OK ] Sequence.Random (11 ms)
[ RUN  ] Sequence.Serialize
[   OK ] Sequence.Serialize (11 ms)
[ RUN  ] Sequence.DiscardOne
[   OK ] Sequence.DiscardOne (18 ms)
[ RUN  ] Sequence.DiscardAll
[   OK ] Sequence.DiscardAll (11 ms)
[--] 4 tests from Sequence (51 ms total)

[--] 1 test from Future
[ RUN  ] Future.FromTry
[   OK ] Future.FromTry (0 ms)
[--] 1 test from Future (0 ms total)

[--] 42 tests from Process
[ RUN  ] Process.pid
[   OK ] Process.pid (0 ms)
[ RUN  ] Process.exited
[   OK ] Process.exited (1 ms)
[ RUN  ] Process.associate
[   OK ] Process.associate (0 ms)
[ RUN  ] Process.delay
[   OK ] Process.delay (8 ms)
[ RUN  ] Process.select
[   OK ] Process.select (0 ms)
[ RUN  ] Process.dispatch
[   OK ] Process.dispatch (16 ms)
[ RUN  ] Process.provide
I0801 04:08:51.241659 19049 process.cpp:3335] Handling HTTP event for process 
'(26)' with path: '/(26)'
I0801 04:08:51.241758 19049 process.cpp:1816] Sending file at 
'/tmp/EPmBsR/lorem.txt' with length 446
I0801 04:08:51.249776 19062 process.cpp:1037] Socket closed while receiving
[   OK ] Process.provide (48 ms)
[ RUN  ] Process.order
[   OK ] Process.order (4 ms)
[ RUN  ] Process.defer2
[   OK ] Process.defer2 (40 ms)
[ RUN  ] Process.delegate
I0801 04:08:51.329751 19034 process.cpp:3318] Delegating message 'func' to 
(30)@140.211.11.27:39902
[   OK ] Process.delegate (2 ms)
[ RUN  ] Process.then
[   OK ] Process.then (0 ms)
[ RUN  ] Process.http1
[   OK ] Process.http1 (14 ms)
[ RUN  ] Process.spawn
[   OK ] Process.spawn (28 ms)
[ RUN  ] Process.onAny
[   OK ] Process.onAny (0 ms)
[ RUN  ] Process.action
[   OK ] Process.action (36 ms)
[ RUN  ] Process.expect
[   OK ] Process.expect (20 ms)
[ RUN  ] Process.discard3
[   OK ] Process.discard3 (0 ms)
[ RUN  ] Process.await3
[   OK ] Process.await3 (3 ms)
[ RUN  ] Process.await2
[   OK ] Process.await2 (1 ms)
[ RUN  ] Process.PercentEncodedURLs
I0801 04:08:51.435185 19039 process.cpp:3335] Handling HTTP event for process 
'id(42)' with path: '/id%2842%29/handler2'
I0801 04:08:51.435369 19062 process.cpp:1037] Socket closed while receiving
[   OK ] Process.PercentEncodedURLs (2 ms)
[ RUN  ] Process.event
[   OK ] Process.event (0 ms)
[ RUN  ] Process.after1
[   OK ] Process.after1 (10 ms)
[ RUN  ] Process.donate
[   OK ] Process.donate (0 ms)
[ RUN  ] Process.defer3
[   OK ] Process.defer3 (4 ms)
[ RUN  ] Process.handlers
[   OK ] Process.handlers (0 ms)
[ RUN  ] Process.future
[   OK ] Process.future (0 ms)
[ RUN  ] Process.after2
[   OK ] Process.after2 (10 ms)
[ RUN  ] Process.await1
[   OK ] Process.await1 (1 ms)
[ RUN  ] Process.collect
[   OK ] Process.collect (0 ms)
[ RUN  ] Process.http2
I0801 04:08:51.486529 19062 process.cpp:1037] Socket closed while receiving
I0801 04:08:51.497727 19062 process.cpp:1098] Socket closed while receiving
[   OK ] Process.http2 (37 ms)
[ RUN  ] Process.settle
[   OK ] Process.settle (51 ms)
[ RUN  ] Process.async
[   OK ] Process.async (81 ms)
[ RUN  ] Process.thunk
[   OK ] Process.thunk (8 ms)
[ RUN  ] Process.discard2
[   OK ] Process.discard2 (0 ms)
[ RUN  ] Process.executor
[   OK ] Process.executor (4 ms)
[ RUN  ] Process.remote
I0801 04:08:51.645781 19062 process.cpp:1037] Socket closed while receiving
[   OK ] Process.remote (28 ms)
[ RUN  ] Process.defer1
[   OK ] Process.defer1 (56 ms)
[ RUN  ] Process.listener
[   OK ] Process.listener (1 ms)
[ RUN  ] Process.chain
[   OK ] Process.chain (0 ms)
[ RUN  ] Process.limiter
[   OK ] Process.limiter (39 ms)
[ RUN  ] Process.discard1
[   OK ] Process.discard1 (0 ms)
[ RUN  ] Process.inheritance
[   OK ] Process.inheritance (0 ms)
[--] 42 tests from Process (553 ms total)

[--] Global test environment tear-down
[==] 114 tests from 17 test cases ran. (2303 ms total)
[  PASSED  ] 114 tests.

  YOU HAVE 3 DISABLED TESTS

make[6]: Leaving directory 
`/x1/jenkins/jenkins-slave/workspace/Mesos-Ubuntu-distcheck/build/mesos-0.20.0/_build/3rdparty/libprocess'
make[5]: Leaving directory 
`/x1/jenkins/jenkins-slave/workspace/Mesos-Ubuntu-distcheck/build/mesos-0.20.0/_build/3rdparty/libprocess'
Making check in include
make[5]: Entering directory 
`/x1/jenkins/jenkins-slave/workspace/Mesos-Ubuntu-distcheck/build/mesos-0.20.0/_build/3rdparty/libprocess/include'
make[5]: Nothing to

Re: Review Request 23224: Refactored the python bindings into multiple modules.

2014-07-31 Thread Benjamin Hindman


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23224/#review49319
---

Ship it!


Rad Thomas! Can you draft a note to the dev/user lists giving people a heads up 
that this will hit master and folks that are using/running Python off of master 
will need to upgrade. Thanks!

- Benjamin Hindman


On Aug. 1, 2014, 2:26 a.m., Thomas Rampelberg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23224/
> ---
> 
> (Updated Aug. 1, 2014, 2:26 a.m.)
> 
> 
> Review request for mesos.
> 
> 
> Bugs: MESOS-857
> https://issues.apache.org/jira/browse/MESOS-857
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> The existing module has been split into two separate ones:
> 
> - mesos.interface - This contains the stub implementations for the 
> Executor/Scheduler as well as the protobufs.
> - mesos.native - The old _mesos module.
> 
> There is also a base metapackage `mesos` that allows a potential `pip install 
> mesos` to correctly install everything required. While mesos.interface can 
> now be uploaded to the cheeseshop, mesos.native has not changed and will need 
> some more work first.
> 
> 
> 
> Diffs
> -
> 
>   Makefile.am b91d8cf011832e6e91b16f03a2d80fbb601eba8f 
>   configure.ac e7472081339fc9c773eb2cf2d5f15dc459ac378d 
>   docs/upgrades.md 919f0108655568d91060e6db998094e6dc39127b 
>   mpi/mpiexec-mesos.in da0733fc29f97e67385cab55d60d4e2afd76aba9 
>   mpi/mpiexec-mesos.py 0ab50167eaa43f9d69f37b7c10e26fa7a7d9f250 
>   src/Makefile.am b660d912c92594ef679b71caf508134f20511dae 
>   src/examples/python/test-containerizer.in 
> 569519b3b9755959f9bf931d3c81be9a00b64bc9 
>   src/examples/python/test-executor.in 
> 7e8875f0fd74dc9f9207986864edbce588ec3fb8 
>   src/examples/python/test-framework.in 
> c4683b97b87ba8753e842b0c75cc3d65140a5cf7 
>   src/examples/python/test_containerizer.py 
> c65d891539bcee775741626596997afe8471c930 
>   src/examples/python/test_executor.py 
> 065b50a6146cb39a82024d82c20cf89f940a9e57 
>   src/examples/python/test_framework.py 
> fce090fe542e3863770d7daea3d8764da1d8d5df 
>   src/python/interface/setup.py.in PRE-CREATION 
>   src/python/interface/src/mesos/__init__.py PRE-CREATION 
>   src/python/native/ext_modules.py.in PRE-CREATION 
>   src/python/native/mesos_executor_driver_impl.hpp  
>   src/python/native/mesos_executor_driver_impl.cpp  
>   src/python/native/mesos_scheduler_driver_impl.hpp  
>   src/python/native/mesos_scheduler_driver_impl.cpp  
>   src/python/native/module.hpp  
>   src/python/native/module.cpp f523c1f3a5b62413ec01e5101c680c30980874dc 
>   src/python/native/proxy_executor.hpp  
>   src/python/native/proxy_executor.cpp  
>   src/python/native/proxy_scheduler.hpp  
>   src/python/native/proxy_scheduler.cpp  
>   src/python/native/setup.py.in PRE-CREATION 
>   src/python/native/src/mesos/__init__.py PRE-CREATION 
>   src/python/native/src/mesos/native/__init__.py PRE-CREATION 
>   src/python/protocol/setup.py.in PRE-CREATION 
>   src/python/protocol/src/mesos/__init__.py PRE-CREATION 
>   src/python/protocol/src/mesos/protocol/__init__.py PRE-CREATION 
>   src/python/setup.py.in 60add3b39d09dc5a1b74764b2b6c4b3c461f150a 
>   src/python/src/mesos.py 0152ab456f072f8d4a1c4ab19fe74e181eadbd05 
>   src/python/src/mesos/__init__.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/23224/diff/
> 
> 
> Testing
> ---
> 
> - `make distcheck -j6` (note that this runs the actual python framework tests)
> - `import mesos.native`
> - `python setup.py sdist bdist_egg register upload` && `pip install 
> mesos.interface`
> 
> 
> Thanks,
> 
> Thomas Rampelberg
> 
>

Re: Review Request 23224: Refactored the python bindings into multiple modules.

2014-07-31 Thread Thomas Rampelberg


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23224/
---

(Updated Aug. 1, 2014, 2:26 a.m.)


Review request for mesos.


Bugs: MESOS-857
https://issues.apache.org/jira/browse/MESOS-857


Repository: mesos-git


Description
---

The existing module has been split into two separate ones:

- mesos.interface - This contains the stub implementations for the 
Executor/Scheduler as well as the protobufs.
- mesos.native - The old _mesos module.

There is also a base metapackage `mesos` that allows a potential `pip install 
mesos` to correctly install everything required. While mesos.interface can now 
be uploaded to the cheeseshop, mesos.native has not changed and will need some 
more work first.



Diffs (updated)
-

  Makefile.am b91d8cf011832e6e91b16f03a2d80fbb601eba8f 
  configure.ac e7472081339fc9c773eb2cf2d5f15dc459ac378d 
  docs/upgrades.md 919f0108655568d91060e6db998094e6dc39127b 
  mpi/mpiexec-mesos.in da0733fc29f97e67385cab55d60d4e2afd76aba9 
  mpi/mpiexec-mesos.py 0ab50167eaa43f9d69f37b7c10e26fa7a7d9f250 
  src/Makefile.am b660d912c92594ef679b71caf508134f20511dae 
  src/examples/python/test-containerizer.in 
569519b3b9755959f9bf931d3c81be9a00b64bc9 
  src/examples/python/test-executor.in 7e8875f0fd74dc9f9207986864edbce588ec3fb8 
  src/examples/python/test-framework.in 
c4683b97b87ba8753e842b0c75cc3d65140a5cf7 
  src/examples/python/test_containerizer.py 
c65d891539bcee775741626596997afe8471c930 
  src/examples/python/test_executor.py 065b50a6146cb39a82024d82c20cf89f940a9e57 
  src/examples/python/test_framework.py 
fce090fe542e3863770d7daea3d8764da1d8d5df 
  src/python/interface/setup.py.in PRE-CREATION 
  src/python/interface/src/mesos/__init__.py PRE-CREATION 
  src/python/native/ext_modules.py.in PRE-CREATION 
  src/python/native/mesos_executor_driver_impl.hpp  
  src/python/native/mesos_executor_driver_impl.cpp  
  src/python/native/mesos_scheduler_driver_impl.hpp  
  src/python/native/mesos_scheduler_driver_impl.cpp  
  src/python/native/module.hpp  
  src/python/native/module.cpp f523c1f3a5b62413ec01e5101c680c30980874dc 
  src/python/native/proxy_executor.hpp  
  src/python/native/proxy_executor.cpp  
  src/python/native/proxy_scheduler.hpp  
  src/python/native/proxy_scheduler.cpp  
  src/python/native/setup.py.in PRE-CREATION 
  src/python/native/src/mesos/__init__.py PRE-CREATION 
  src/python/native/src/mesos/native/__init__.py PRE-CREATION 
  src/python/protocol/setup.py.in PRE-CREATION 
  src/python/protocol/src/mesos/__init__.py PRE-CREATION 
  src/python/protocol/src/mesos/protocol/__init__.py PRE-CREATION 
  src/python/setup.py.in 60add3b39d09dc5a1b74764b2b6c4b3c461f150a 
  src/python/src/mesos.py 0152ab456f072f8d4a1c4ab19fe74e181eadbd05 
  src/python/src/mesos/__init__.py PRE-CREATION 

Diff: https://reviews.apache.org/r/23224/diff/


Testing
---

- `make distcheck -j6` (note that this runs the actual python framework tests)
- `import mesos.native`
- `python setup.py sdist bdist_egg register upload` && `pip install 
mesos.interface`


Thanks,

Thomas Rampelberg

Re: Review Request 23224: Refactored the python bindings into multiple modules.

2014-07-31 Thread Thomas Rampelberg


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23224/
---

(Updated Aug. 1, 2014, 1:58 a.m.)


Review request for mesos.


Bugs: MESOS-857
https://issues.apache.org/jira/browse/MESOS-857


Repository: mesos-git


Description
---

The existing module has been split into two separate ones:

- mesos.interface - This contains the stub implementations for the 
Executor/Scheduler as well as the protobufs.
- mesos.native - The old _mesos module.

There is also a base metapackage `mesos` that allows a potential `pip install 
mesos` to correctly install everything required. While mesos.interface can now 
be uploaded to the cheeseshop, mesos.native has not changed and will need some 
more work first.



Diffs (updated)
-

  Makefile.am b91d8cf011832e6e91b16f03a2d80fbb601eba8f 
  configure.ac e7472081339fc9c773eb2cf2d5f15dc459ac378d 
  docs/upgrades.md 919f0108655568d91060e6db998094e6dc39127b 
  mpi/mpiexec-mesos.in da0733fc29f97e67385cab55d60d4e2afd76aba9 
  mpi/mpiexec-mesos.py 0ab50167eaa43f9d69f37b7c10e26fa7a7d9f250 
  src/Makefile.am b660d912c92594ef679b71caf508134f20511dae 
  src/examples/python/test-containerizer.in 
569519b3b9755959f9bf931d3c81be9a00b64bc9 
  src/examples/python/test-executor.in 7e8875f0fd74dc9f9207986864edbce588ec3fb8 
  src/examples/python/test-framework.in 
c4683b97b87ba8753e842b0c75cc3d65140a5cf7 
  src/examples/python/test_containerizer.py 
c65d891539bcee775741626596997afe8471c930 
  src/examples/python/test_executor.py 065b50a6146cb39a82024d82c20cf89f940a9e57 
  src/examples/python/test_framework.py 
fce090fe542e3863770d7daea3d8764da1d8d5df 
  src/python/interface/setup.py.in PRE-CREATION 
  src/python/interface/src/mesos/__init__.py PRE-CREATION 
  src/python/native/ext_modules.py.in PRE-CREATION 
  src/python/native/mesos_executor_driver_impl.hpp  
  src/python/native/mesos_executor_driver_impl.cpp  
  src/python/native/mesos_scheduler_driver_impl.hpp  
  src/python/native/mesos_scheduler_driver_impl.cpp  
  src/python/native/module.hpp  
  src/python/native/module.cpp f523c1f3a5b62413ec01e5101c680c30980874dc 
  src/python/native/proxy_executor.hpp  
  src/python/native/proxy_executor.cpp  
  src/python/native/proxy_scheduler.hpp  
  src/python/native/proxy_scheduler.cpp  
  src/python/native/setup.py.in PRE-CREATION 
  src/python/native/src/mesos/__init__.py PRE-CREATION 
  src/python/native/src/mesos/native/__init__.py PRE-CREATION 
  src/python/protocol/setup.py.in PRE-CREATION 
  src/python/protocol/src/mesos/__init__.py PRE-CREATION 
  src/python/protocol/src/mesos/protocol/__init__.py PRE-CREATION 
  src/python/setup.py.in 60add3b39d09dc5a1b74764b2b6c4b3c461f150a 
  src/python/src/mesos.py 0152ab456f072f8d4a1c4ab19fe74e181eadbd05 
  src/python/src/mesos/__init__.py PRE-CREATION 

Diff: https://reviews.apache.org/r/23224/diff/


Testing
---

- `make distcheck -j6` (note that this runs the actual python framework tests)
- `import mesos.native`
- `python setup.py sdist bdist_egg register upload` && `pip install 
mesos.interface`


Thanks,

Thomas Rampelberg

Re: Review Request 23224: Refactored the python bindings into multiple modules.

2014-07-31 Thread Thomas Rampelberg


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23224/
---

(Updated Aug. 1, 2014, 1:39 a.m.)


Review request for mesos.


Bugs: MESOS-857
https://issues.apache.org/jira/browse/MESOS-857


Repository: mesos-git


Description
---

The existing module has been split into two separate ones:

- mesos.interface - This contains the stub implementations for the 
Executor/Scheduler as well as the protobufs.
- mesos.native - The old _mesos module.

There is also a base metapackage `mesos` that allows a potential `pip install 
mesos` to correctly install everything required. While mesos.interface can now 
be uploaded to the cheeseshop, mesos.native has not changed and will need some 
more work first.



Diffs (updated)
-

  Makefile.am b91d8cf011832e6e91b16f03a2d80fbb601eba8f 
  configure.ac e7472081339fc9c773eb2cf2d5f15dc459ac378d 
  docs/upgrades.md 919f0108655568d91060e6db998094e6dc39127b 
  mpi/mpiexec-mesos.in da0733fc29f97e67385cab55d60d4e2afd76aba9 
  mpi/mpiexec-mesos.py 0ab50167eaa43f9d69f37b7c10e26fa7a7d9f250 
  src/Makefile.am b660d912c92594ef679b71caf508134f20511dae 
  src/examples/python/test-containerizer.in 
569519b3b9755959f9bf931d3c81be9a00b64bc9 
  src/examples/python/test-executor.in 7e8875f0fd74dc9f9207986864edbce588ec3fb8 
  src/examples/python/test-framework.in 
c4683b97b87ba8753e842b0c75cc3d65140a5cf7 
  src/examples/python/test_containerizer.py 
c65d891539bcee775741626596997afe8471c930 
  src/examples/python/test_executor.py 065b50a6146cb39a82024d82c20cf89f940a9e57 
  src/examples/python/test_framework.py 
fce090fe542e3863770d7daea3d8764da1d8d5df 
  src/python/interface/setup.py.in PRE-CREATION 
  src/python/interface/src/mesos/__init__.py PRE-CREATION 
  src/python/native/ext_modules.py.in PRE-CREATION 
  src/python/native/mesos_executor_driver_impl.hpp  
  src/python/native/mesos_executor_driver_impl.cpp  
  src/python/native/mesos_scheduler_driver_impl.hpp  
  src/python/native/mesos_scheduler_driver_impl.cpp  
  src/python/native/module.hpp  
  src/python/native/module.cpp f523c1f3a5b62413ec01e5101c680c30980874dc 
  src/python/native/proxy_executor.hpp  
  src/python/native/proxy_executor.cpp  
  src/python/native/proxy_scheduler.hpp  
  src/python/native/proxy_scheduler.cpp  
  src/python/native/setup.py.in PRE-CREATION 
  src/python/native/src/mesos/__init__.py PRE-CREATION 
  src/python/native/src/mesos/native/__init__.py PRE-CREATION 
  src/python/protocol/setup.py.in PRE-CREATION 
  src/python/protocol/src/mesos/__init__.py PRE-CREATION 
  src/python/protocol/src/mesos/protocol/__init__.py PRE-CREATION 
  src/python/setup.py.in 60add3b39d09dc5a1b74764b2b6c4b3c461f150a 
  src/python/src/mesos.py 0152ab456f072f8d4a1c4ab19fe74e181eadbd05 
  src/python/src/mesos/__init__.py PRE-CREATION 

Diff: https://reviews.apache.org/r/23224/diff/


Testing
---

- `make distcheck -j6` (note that this runs the actual python framework tests)
- `import mesos.native`
- `python setup.py sdist bdist_egg register upload` && `pip install 
mesos.interface`


Thanks,

Thomas Rampelberg

Re: Review Request 23224: Refactored the python bindings into multiple modules.

2014-07-31 Thread Mesos ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23224/#review49316
---


Patch looks great!

Reviews applied: [23224]

All tests passed.

- Mesos ReviewBot


On Aug. 1, 2014, 1:39 a.m., Thomas Rampelberg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23224/
> ---
> 
> (Updated Aug. 1, 2014, 1:39 a.m.)
> 
> 
> Review request for mesos.
> 
> 
> Bugs: MESOS-857
> https://issues.apache.org/jira/browse/MESOS-857
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> The existing module has been split into two separate ones:
> 
> - mesos.interface - This contains the stub implementations for the 
> Executor/Scheduler as well as the protobufs.
> - mesos.native - The old _mesos module.
> 
> There is also a base metapackage `mesos` that allows a potential `pip install 
> mesos` to correctly install everything required. While mesos.interface can 
> now be uploaded to the cheeseshop, mesos.native has not changed and will need 
> some more work first.
> 
> 
> 
> Diffs
> -
> 
>   Makefile.am b91d8cf011832e6e91b16f03a2d80fbb601eba8f 
>   configure.ac e7472081339fc9c773eb2cf2d5f15dc459ac378d 
>   docs/upgrades.md 919f0108655568d91060e6db998094e6dc39127b 
>   mpi/mpiexec-mesos.in da0733fc29f97e67385cab55d60d4e2afd76aba9 
>   mpi/mpiexec-mesos.py 0ab50167eaa43f9d69f37b7c10e26fa7a7d9f250 
>   src/Makefile.am b660d912c92594ef679b71caf508134f20511dae 
>   src/examples/python/test-containerizer.in 
> 569519b3b9755959f9bf931d3c81be9a00b64bc9 
>   src/examples/python/test-executor.in 
> 7e8875f0fd74dc9f9207986864edbce588ec3fb8 
>   src/examples/python/test-framework.in 
> c4683b97b87ba8753e842b0c75cc3d65140a5cf7 
>   src/examples/python/test_containerizer.py 
> c65d891539bcee775741626596997afe8471c930 
>   src/examples/python/test_executor.py 
> 065b50a6146cb39a82024d82c20cf89f940a9e57 
>   src/examples/python/test_framework.py 
> fce090fe542e3863770d7daea3d8764da1d8d5df 
>   src/python/interface/setup.py.in PRE-CREATION 
>   src/python/interface/src/mesos/__init__.py PRE-CREATION 
>   src/python/native/ext_modules.py.in PRE-CREATION 
>   src/python/native/mesos_executor_driver_impl.hpp  
>   src/python/native/mesos_executor_driver_impl.cpp  
>   src/python/native/mesos_scheduler_driver_impl.hpp  
>   src/python/native/mesos_scheduler_driver_impl.cpp  
>   src/python/native/module.hpp  
>   src/python/native/module.cpp f523c1f3a5b62413ec01e5101c680c30980874dc 
>   src/python/native/proxy_executor.hpp  
>   src/python/native/proxy_executor.cpp  
>   src/python/native/proxy_scheduler.hpp  
>   src/python/native/proxy_scheduler.cpp  
>   src/python/native/setup.py.in PRE-CREATION 
>   src/python/native/src/mesos/__init__.py PRE-CREATION 
>   src/python/native/src/mesos/native/__init__.py PRE-CREATION 
>   src/python/protocol/setup.py.in PRE-CREATION 
>   src/python/protocol/src/mesos/__init__.py PRE-CREATION 
>   src/python/protocol/src/mesos/protocol/__init__.py PRE-CREATION 
>   src/python/setup.py.in 60add3b39d09dc5a1b74764b2b6c4b3c461f150a 
>   src/python/src/mesos.py 0152ab456f072f8d4a1c4ab19fe74e181eadbd05 
>   src/python/src/mesos/__init__.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/23224/diff/
> 
> 
> Testing
> ---
> 
> - `make distcheck -j6` (note that this runs the actual python framework tests)
> - `import mesos.native`
> - `python setup.py sdist bdist_egg register upload` && `pip install 
> mesos.interface`
> 
> 
> Thanks,
> 
> Thomas Rampelberg
> 
>

Re: Review Request 23771: Added a Docker containerizer.

2014-07-31 Thread Timothy Chen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23771/#review49313
---



src/health-check/main.cpp


These are changes that got in from rebase. This should be all in master 
already.


- Timothy Chen


On Aug. 1, 2014, 12:46 a.m., Timothy Chen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23771/
> ---
> 
> (Updated Aug. 1, 2014, 12:46 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, Ian Downes, and Jie Yu.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Docker implementation.
> This is all the docker code Ben, I and Yifan worked on excluding the 
> composing containerizer patches.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am 45afcd1 
>   src/common/status_utils.hpp c51a8c6 
>   src/docker/docker.hpp PRE-CREATION 
>   src/docker/docker.cpp PRE-CREATION 
>   src/examples/docker_no_executor_framework.cpp PRE-CREATION 
>   src/health-check/main.cpp 10d57a0 
>   src/launcher/executor.cpp 948673e 
>   src/linux/cgroups.hpp decad9d 
>   src/linux/cgroups.cpp af6be22 
>   src/master/master.cpp 77f2536 
>   src/slave/containerizer/containerizer.cpp 1b71f33 
>   src/slave/containerizer/docker.hpp PRE-CREATION 
>   src/slave/containerizer/docker.cpp PRE-CREATION 
>   src/slave/containerizer/external_containerizer.cpp 3f28d85 
>   src/slave/containerizer/isolators/cgroups/cpushare.hpp 780037b 
>   src/slave/containerizer/isolators/cgroups/cpushare.cpp 3265a80 
>   src/slave/containerizer/isolators/cgroups/mem.hpp 8c476c7 
>   src/slave/containerizer/isolators/cgroups/mem.cpp e8d1e35 
>   src/slave/containerizer/isolators/posix.hpp 17bbd10 
>   src/slave/flags.hpp 1fe7b7d 
>   src/slave/slave.cpp f42ab60 
>   src/tests/cgroups_tests.cpp 01cf498 
>   src/tests/docker_containerizer_tests.cpp PRE-CREATION 
>   src/tests/docker_tests.cpp PRE-CREATION 
>   src/tests/environment.cpp 551698f 
>   src/tests/flags.hpp a003e7f 
>   src/tests/script.cpp 0d29c6d 
>   src/usage/usage.hpp 5a76746 
>   src/usage/usage.cpp 29014d1 
> 
> Diff: https://reviews.apache.org/r/23771/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Timothy Chen
> 
>

Re: Review Request 23771: Added a Docker containerizer.

2014-07-31 Thread Timothy Chen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23771/
---

(Updated Aug. 1, 2014, 12:46 a.m.)


Review request for mesos, Benjamin Hindman, Ian Downes, and Jie Yu.


Repository: mesos-git


Description
---

Docker implementation.
This is all the docker code Ben, I and Yifan worked on excluding the composing 
containerizer patches.


Diffs (updated)
-

  src/Makefile.am 45afcd1 
  src/common/status_utils.hpp c51a8c6 
  src/docker/docker.hpp PRE-CREATION 
  src/docker/docker.cpp PRE-CREATION 
  src/examples/docker_no_executor_framework.cpp PRE-CREATION 
  src/health-check/main.cpp 10d57a0 
  src/launcher/executor.cpp 948673e 
  src/linux/cgroups.hpp decad9d 
  src/linux/cgroups.cpp af6be22 
  src/master/master.cpp 77f2536 
  src/slave/containerizer/containerizer.cpp 1b71f33 
  src/slave/containerizer/docker.hpp PRE-CREATION 
  src/slave/containerizer/docker.cpp PRE-CREATION 
  src/slave/containerizer/external_containerizer.cpp 3f28d85 
  src/slave/containerizer/isolators/cgroups/cpushare.hpp 780037b 
  src/slave/containerizer/isolators/cgroups/cpushare.cpp 3265a80 
  src/slave/containerizer/isolators/cgroups/mem.hpp 8c476c7 
  src/slave/containerizer/isolators/cgroups/mem.cpp e8d1e35 
  src/slave/containerizer/isolators/posix.hpp 17bbd10 
  src/slave/flags.hpp 1fe7b7d 
  src/slave/slave.cpp f42ab60 
  src/tests/cgroups_tests.cpp 01cf498 
  src/tests/docker_containerizer_tests.cpp PRE-CREATION 
  src/tests/docker_tests.cpp PRE-CREATION 
  src/tests/environment.cpp 551698f 
  src/tests/flags.hpp a003e7f 
  src/tests/script.cpp 0d29c6d 
  src/usage/usage.hpp 5a76746 
  src/usage/usage.cpp 29014d1 

Diff: https://reviews.apache.org/r/23771/diff/


Testing
---

make check


Thanks,

Timothy Chen

Re: Review Request 23224: Refactored the python bindings into multiple modules.

2014-07-31 Thread Thomas Rampelberg


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23224/
---

(Updated Aug. 1, 2014, 12:41 a.m.)


Review request for mesos.


Bugs: MESOS-857
https://issues.apache.org/jira/browse/MESOS-857


Repository: mesos-git


Description
---

The existing module has been split into two separate ones:

- mesos.interface - This contains the stub implementations for the 
Executor/Scheduler as well as the protobufs.
- mesos.native - The old _mesos module.

There is also a base metapackage `mesos` that allows a potential `pip install 
mesos` to correctly install everything required. While mesos.interface can now 
be uploaded to the cheeseshop, mesos.native has not changed and will need some 
more work first.



Diffs (updated)
-

  Makefile.am b91d8cf011832e6e91b16f03a2d80fbb601eba8f 
  configure.ac e7472081339fc9c773eb2cf2d5f15dc459ac378d 
  docs/upgrades.md 919f0108655568d91060e6db998094e6dc39127b 
  mpi/mpiexec-mesos.in da0733fc29f97e67385cab55d60d4e2afd76aba9 
  mpi/mpiexec-mesos.py 0ab50167eaa43f9d69f37b7c10e26fa7a7d9f250 
  src/Makefile.am b660d912c92594ef679b71caf508134f20511dae 
  src/examples/python/test-containerizer.in 
569519b3b9755959f9bf931d3c81be9a00b64bc9 
  src/examples/python/test-executor.in 7e8875f0fd74dc9f9207986864edbce588ec3fb8 
  src/examples/python/test-framework.in 
c4683b97b87ba8753e842b0c75cc3d65140a5cf7 
  src/examples/python/test_containerizer.py 
c65d891539bcee775741626596997afe8471c930 
  src/examples/python/test_executor.py 065b50a6146cb39a82024d82c20cf89f940a9e57 
  src/examples/python/test_framework.py 
fce090fe542e3863770d7daea3d8764da1d8d5df 
  src/python/interface/setup.py.in PRE-CREATION 
  src/python/interface/src/mesos/__init__.py PRE-CREATION 
  src/python/native/ext_modules.py.in PRE-CREATION 
  src/python/native/mesos_executor_driver_impl.hpp  
  src/python/native/mesos_executor_driver_impl.cpp  
  src/python/native/mesos_scheduler_driver_impl.hpp  
  src/python/native/mesos_scheduler_driver_impl.cpp  
  src/python/native/module.hpp  
  src/python/native/module.cpp b94712681e6f0e9bf5dfdafa10621d1df82dc367 
  src/python/native/proxy_executor.hpp  
  src/python/native/proxy_executor.cpp  
  src/python/native/proxy_scheduler.hpp  
  src/python/native/proxy_scheduler.cpp  
  src/python/native/setup.py.in PRE-CREATION 
  src/python/native/src/mesos/__init__.py PRE-CREATION 
  src/python/native/src/mesos/native/__init__.py PRE-CREATION 
  src/python/protocol/setup.py.in PRE-CREATION 
  src/python/protocol/src/mesos/__init__.py PRE-CREATION 
  src/python/protocol/src/mesos/protocol/__init__.py PRE-CREATION 
  src/python/setup.py.in 60add3b39d09dc5a1b74764b2b6c4b3c461f150a 
  src/python/src/mesos.py 0152ab456f072f8d4a1c4ab19fe74e181eadbd05 
  src/python/src/mesos/__init__.py PRE-CREATION 

Diff: https://reviews.apache.org/r/23224/diff/


Testing
---

- `make distcheck -j6` (note that this runs the actual python framework tests)
- `import mesos.native`
- `python setup.py sdist bdist_egg register upload` && `pip install 
mesos.interface`


Thanks,

Thomas Rampelberg

Re: Review Request 23224: Refactored the python bindings into multiple modules.

2014-07-31 Thread Thomas Rampelberg


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23224/
---

(Updated Aug. 1, 2014, 12:35 a.m.)


Review request for mesos.


Bugs: MESOS-857
https://issues.apache.org/jira/browse/MESOS-857


Repository: mesos-git


Description
---

The existing module has been split into two separate ones:

- mesos.interface - This contains the stub implementations for the 
Executor/Scheduler as well as the protobufs.
- mesos.native - The old _mesos module.

There is also a base metapackage `mesos` that allows a potential `pip install 
mesos` to correctly install everything required. While mesos.interface can now 
be uploaded to the cheeseshop, mesos.native has not changed and will need some 
more work first.



Diffs (updated)
-

  Makefile.am b91d8cf011832e6e91b16f03a2d80fbb601eba8f 
  configure.ac e7472081339fc9c773eb2cf2d5f15dc459ac378d 
  docs/upgrades.md 919f0108655568d91060e6db998094e6dc39127b 
  mpi/mpiexec-mesos.in da0733fc29f97e67385cab55d60d4e2afd76aba9 
  mpi/mpiexec-mesos.py 0ab50167eaa43f9d69f37b7c10e26fa7a7d9f250 
  src/Makefile.am b660d912c92594ef679b71caf508134f20511dae 
  src/examples/python/test-containerizer.in 
569519b3b9755959f9bf931d3c81be9a00b64bc9 
  src/examples/python/test-executor.in 7e8875f0fd74dc9f9207986864edbce588ec3fb8 
  src/examples/python/test-framework.in 
c4683b97b87ba8753e842b0c75cc3d65140a5cf7 
  src/examples/python/test_containerizer.py 
c65d891539bcee775741626596997afe8471c930 
  src/examples/python/test_executor.py 065b50a6146cb39a82024d82c20cf89f940a9e57 
  src/examples/python/test_framework.py 
fce090fe542e3863770d7daea3d8764da1d8d5df 
  src/python/interface/setup.py.in PRE-CREATION 
  src/python/interface/src/mesos/__init__.py PRE-CREATION 
  src/python/native/mesos_executor_driver_impl.hpp  
  src/python/native/mesos_executor_driver_impl.cpp  
  src/python/native/mesos_scheduler_driver_impl.hpp  
  src/python/native/mesos_scheduler_driver_impl.cpp  
  src/python/native/module.hpp  
  src/python/native/module.cpp b94712681e6f0e9bf5dfdafa10621d1df82dc367 
  src/python/native/proxy_executor.hpp  
  src/python/native/proxy_executor.cpp  
  src/python/native/proxy_scheduler.hpp  
  src/python/native/proxy_scheduler.cpp  
  src/python/native/setup.py.in PRE-CREATION 
  src/python/native/src/mesos/__init__.py PRE-CREATION 
  src/python/native/src/mesos/native/__init__.py PRE-CREATION 
  src/python/protocol/setup.py.in PRE-CREATION 
  src/python/protocol/src/mesos/__init__.py PRE-CREATION 
  src/python/protocol/src/mesos/protocol/__init__.py PRE-CREATION 
  src/python/setup.py.in 60add3b39d09dc5a1b74764b2b6c4b3c461f150a 
  src/python/src/mesos.py 0152ab456f072f8d4a1c4ab19fe74e181eadbd05 
  src/python/src/mesos/__init__.py PRE-CREATION 

Diff: https://reviews.apache.org/r/23224/diff/


Testing
---

- `make distcheck -j6` (note that this runs the actual python framework tests)
- `import mesos.native`
- `python setup.py sdist bdist_egg register upload` && `pip install 
mesos.interface`


Thanks,

Thomas Rampelberg

Re: Review Request 23771: Added a Docker containerizer.

2014-07-31 Thread Timothy Chen



> On July 31, 2014, 6:39 p.m., Jie Yu wrote:
> > src/docker/docker.cpp, line 351
> > 
> >
> > if (status.get() != 0) {
> >   return err(s).then(lambda::bind(&__check, cmd, status.get(), 
> > lambda::_1));
> > }

I actually can't do this since the return type doesn't make __check. 
What I ended up doing is another static method with a template.


> On July 31, 2014, 6:39 p.m., Jie Yu wrote:
> > src/docker/docker.cpp, line 80
> > 
> >
> > This has been gone through a few iterations already. Sorry, this is my 
> > fault that I haven't thought about it in deep. But I hope we can resolve 
> > this before shipping this.
> > 
> > A few issues:
> > 
> > 1) This is a library, we usually don't have LOG() in the library 
> > (otherwise, very likely we'll have double logging). Instead, we usually put 
> > error messages in the return value (either Future or Try).
> > 
> > 2) Returning 'bool' doesn't seem to be necessary? We always return 
> > false on error and return true on success, right? In that case, we can just 
> > use Future?
> > 
> > So here is what I would suggest (s/success/check/ maybe):
> > 
> > Future check(const string& cmd, const Subprocess& s)
> > {
> >   return s.status().then(lambda::bind(&_success, cmd, s);
> > }
> > 
> > Future _check(const string& cmd, const Subprocess& s)
> > {
> >   Option status = s.status().get();
> >   if (status.isNone()) {
> > return Failure("No status found for '" + cmd + "'");
> >   }
> >   
> >   if (status.get() != 0) {
> > return err(s).then(lambda::bind(&__check, cmd, status, lambda::_1));
> >   }
> > 
> >   return Nothing();
> > }
> > 
> > Future __check(const string& cmd, int status, const string& 
> > err)
> > {
> >   return Failure(
> >   "Failed to '" + cmd + "': exit status = " +
> >   WSTRINGIFY(status) + " stderr = " + err);
> > }
> > 
> > // Retrieve the standard error of 's'.
> > Future err(const Subprocess& s)
> > {
> >   CHECK_SOME(s.err());
> > 
> >   Try nonblock = os::nonblock(s.err().get());
> >   if (nonblock.isError()) {
> > return Failure("Cannot set nonblock for stderr");
> >   }
> > 
> >   return io::read(s.err().get());
> > }

I had an compile error where _check conflicts with the _check method from 
process/check.hpp. I renamed all to [_|__]checkStatus


- Timothy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23771/#review49240
---


On July 31, 2014, 8:06 a.m., Timothy Chen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23771/
> ---
> 
> (Updated July 31, 2014, 8:06 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, Ian Downes, and Jie Yu.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Docker implementation.
> This is all the docker code Ben, I and Yifan worked on excluding the 
> composing containerizer patches.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am 45afcd1 
>   src/common/status_utils.hpp 1980551 
>   src/docker/docker.hpp PRE-CREATION 
>   src/docker/docker.cpp PRE-CREATION 
>   src/examples/docker_no_executor_framework.cpp PRE-CREATION 
>   src/health-check/main.cpp 707810a 
>   src/launcher/executor.cpp 9c80848 
>   src/linux/cgroups.hpp decad9d 
>   src/linux/cgroups.cpp 6a73dd7 
>   src/master/master.cpp 251b699 
>   src/slave/containerizer/containerizer.cpp 1b71f33 
>   src/slave/containerizer/docker.hpp PRE-CREATION 
>   src/slave/containerizer/docker.cpp PRE-CREATION 
>   src/slave/containerizer/external_containerizer.cpp 3f28d85 
>   src/slave/containerizer/isolators/cgroups/cpushare.hpp 780037b 
>   src/slave/containerizer/isolators/cgroups/cpushare.cpp 3265a80 
>   src/slave/containerizer/isolators/cgroups/mem.hpp 8c476c7 
>   src/slave/containerizer/isolators/cgroups/mem.cpp e8d1e35 
>   src/slave/containerizer/isolators/posix.hpp 17bbd10 
>   src/slave/flags.hpp 1fe7b7d 
>   src/slave/slave.cpp f42ab60 
>   src/tests/cgroups_tests.cpp 01cf498 
>   src/tests/docker_containerizer_tests.cpp PRE-CREATION 
>   src/tests/docker_tests.cpp PRE-CREATION 
>   src/tests/environment.cpp 434b3f7 
>   src/tests/flags.hpp a003e7f 
>   src/tests/script.cpp 15a6542 
>   src/usage/usage.hpp 5a76746 
>   src/usage/usage.cpp 29014d1 
> 
> Diff: https://reviews.apache.org/r/23771/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Timothy Chen
> 
>

Jenkins build is back to normal : Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME #2022

2014-07-31 Thread Apache Jenkins Server

See

Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME #2278

2014-07-31 Thread Apache Jenkins Server

See 


Changes:

[dhamon] Fix network isolator build

[adam] Fixed 3rdparty stout namespace and todo style

[adam] Fixed 3rdparty libprocess todo and namespace style

[adam] Fixed namespace and todo styles in mesos source

[adam] Enabled todo and namespace rules in style check

[adam] Fixed lingering namespace style error

--
[...truncated 3546 lines...]
byte-compiling build/bdist.linux-x86_64/egg/containerizer_pb2.py to 
containerizer_pb2.pyc
byte-compiling build/bdist.linux-x86_64/egg/mesos_pb2.py to mesos_pb2.pyc
byte-compiling build/bdist.linux-x86_64/egg/mesos.py to mesos.pyc
creating stub loader for _mesos.so
byte-compiling build/bdist.linux-x86_64/egg/_mesos.py to _mesos.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying src/mesos.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/mesos.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/mesos.egg-info/dependency_links.txt -> 
build/bdist.linux-x86_64/egg/EGG-INFO
copying src/mesos.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/mesos.egg-info/top_level.txt -> 
build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/mesos-0.20.0-py2.7-linux-x86_64.egg' and adding 
'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
make[3]: Leaving directory 
`
make[2]: Leaving directory 
`
Making all in ec2
make[2]: Entering directory 
`
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory 
`
make[1]: Leaving directory 
`
make[1]: Entering directory 
`
Making dvi in .
make[2]: Entering directory 
`
make[2]: Nothing to be done for `dvi-am'.
make[2]: Leaving directory 
`
Making dvi in 3rdparty
make[2]: Entering directory 
`
Making dvi in libprocess
make[3]: Entering directory 
`
Making dvi in 3rdparty
make[4]: Entering directory 
`
Making dvi in stout
make[5]: Entering directory 
`
Making dvi in .
make[6]: Entering directory 
`
make[6]: Nothing to be done for `dvi-am'.
make[6]: Leaving directory 
`
Making dvi in include
make[6]: Entering directory 
`
make[6]: Nothing to be done for `dvi'.
make[6]: Leaving directory 
`
make[5]: Leaving directory 
`
make[5]: Entering directory 
`
make[5]: Nothing to be done for `dvi-am'.
make[5]: Leaving directory 
`
make[4

Re: A question on deploying mesos framework to the cluster

2014-07-31 Thread Vinod Kone

> I looked at Marathon and Aurora and still couldn’t find a way to register
> a Framework so that the Scheduler component runs on a Slave node.
>

You would want the scheduler code to be stored in a (shared) storage area
accessible by the slave. The scheduler could be launched as a task of
marathon/aurora. See
http://www.ebaytechblog.com/2014/04/04/delivering-ebays-ci-solution-with-apache-mesos-part-i/#.U9rWN41dU7E
for an example.


>
> By any chance, is there a native support or an external add-on that I can
> use?
>

There is no native or add-on support to launch frameworks by Mesos. The
current solution is to launch a (meta) framework like aurora/marathon
manually and then use that framework to launch other frameworks.

Re: Review Request 23771: Added a Docker containerizer.

2014-07-31 Thread Timothy Chen



> On July 31, 2014, 6:39 p.m., Jie Yu wrote:
> > src/tests/environment.cpp, lines 147-151
> > 
> >
> > Hum, you switch the order. But that doesn't solve the problem. My point 
> > here is that:
> > 
> > This function should never return true early, otherwise, you'll miss 
> > the following checks. So it should be
> > 
> > if (docker.isError()) {
> >   return false;
> > }
> > 
> > #ifdef __linux__
> > if (user.get() != "root") {
> >   return false;
> > }
> > #endif

ah ok I see now! 


- Timothy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23771/#review49240
---


On July 31, 2014, 8:06 a.m., Timothy Chen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23771/
> ---
> 
> (Updated July 31, 2014, 8:06 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, Ian Downes, and Jie Yu.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Docker implementation.
> This is all the docker code Ben, I and Yifan worked on excluding the 
> composing containerizer patches.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am 45afcd1 
>   src/common/status_utils.hpp 1980551 
>   src/docker/docker.hpp PRE-CREATION 
>   src/docker/docker.cpp PRE-CREATION 
>   src/examples/docker_no_executor_framework.cpp PRE-CREATION 
>   src/health-check/main.cpp 707810a 
>   src/launcher/executor.cpp 9c80848 
>   src/linux/cgroups.hpp decad9d 
>   src/linux/cgroups.cpp 6a73dd7 
>   src/master/master.cpp 251b699 
>   src/slave/containerizer/containerizer.cpp 1b71f33 
>   src/slave/containerizer/docker.hpp PRE-CREATION 
>   src/slave/containerizer/docker.cpp PRE-CREATION 
>   src/slave/containerizer/external_containerizer.cpp 3f28d85 
>   src/slave/containerizer/isolators/cgroups/cpushare.hpp 780037b 
>   src/slave/containerizer/isolators/cgroups/cpushare.cpp 3265a80 
>   src/slave/containerizer/isolators/cgroups/mem.hpp 8c476c7 
>   src/slave/containerizer/isolators/cgroups/mem.cpp e8d1e35 
>   src/slave/containerizer/isolators/posix.hpp 17bbd10 
>   src/slave/flags.hpp 1fe7b7d 
>   src/slave/slave.cpp f42ab60 
>   src/tests/cgroups_tests.cpp 01cf498 
>   src/tests/docker_containerizer_tests.cpp PRE-CREATION 
>   src/tests/docker_tests.cpp PRE-CREATION 
>   src/tests/environment.cpp 434b3f7 
>   src/tests/flags.hpp a003e7f 
>   src/tests/script.cpp 15a6542 
>   src/usage/usage.hpp 5a76746 
>   src/usage/usage.cpp 29014d1 
> 
> Diff: https://reviews.apache.org/r/23771/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Timothy Chen
> 
>

Re: Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME #2277

2014-07-31 Thread Benjamin Mahler

Seen this issue in a few of the slave recovery tests, will push a fix.


On Wed, Jul 30, 2014 at 11:21 PM, Apache Jenkins Server <
jenk...@builds.apache.org> wrote:

> See <
> https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/2277/changes
> >
>
> Changes:
>
> [yujie.jay] Adjusted the order in which filters are added/removed to
> tolerate slave
>
> --
> [...truncated 60736 lines...]
> I0731 06:21:18.372212 15017 leveldb.cpp:343] Persisting action (18 bytes)
> to leveldb took 19.936017ms
> I0731 06:21:18.372248 15017 leveldb.cpp:401] Deleting ~2 keys from leveldb
> took 22408ns
> I0731 06:21:18.372258 15017 replica.cpp:676] Persisted action at 4
> I0731 06:21:18.372267 15017 replica.cpp:661] Replica learned TRUNCATE
> action at position 4
> I0731 06:21:18.372515 15017 log.cpp:680] Attempting to append 1092 bytes
> to the log
> I0731 06:21:18.372555 15017 coordinator.cpp:340] Coordinator attempting to
> write APPEND action at position 5
> I0731 06:21:18.372774 15017 replica.cpp:508] Replica received write
> request for position 5
> I0731 06:21:18.372864 15019 sched.cpp:747] Stopping framework
> '20140731-062118-453759884-45430-14840-'
> I0731 06:21:18.372928 15024 master.cpp:1537] Asked to unregister framework
> 20140731-062118-453759884-45430-14840-
> I0731 06:21:18.372938 15024 master.cpp:3826] Removing framework
> 20140731-062118-453759884-45430-14840-
> I0731 06:21:18.373008 15024 master.hpp:826] Removing offer
> 20140731-062118-453759884-45430-14840-0 with resources cpus(*):2;
> mem(*):1024; disk(*):127026; ports(*):[31000-32000] on slave
> 20140731-062118-453759884-45430-14840-0 (hemera.apache.org)
> I0731 06:21:18.373090 15024 hierarchical_allocator_process.hpp:407]
> Deactivated framework 20140731-062118-453759884-45430-14840-
> I0731 06:21:18.373163 15024 hierarchical_allocator_process.hpp:635]
> Recovered cpus(*):2; mem(*):1024; disk(*):127026; ports(*):[31000-32000]
> (total allocatable: cpus(*):2; mem(*):1024; disk(*):127026;
> ports(*):[31000-32000]) on slave 20140731-062118-453759884-45430-14840-0
> from framework 20140731-062118-453759884-45430-14840-
> I0731 06:21:18.373205 15024 hierarchical_allocator_process.hpp:362]
> Removed framework 20140731-062118-453759884-45430-14840-
> I0731 06:21:18.373226 15024 slave.cpp:1407] Asked to shut down framework
> 20140731-062118-453759884-45430-14840- by master@140.211.11.27:45430
> W0731 06:21:18.373236 15024 slave.cpp:1422] Cannot shut down unknown
> framework 20140731-062118-453759884-45430-14840-
> I0731 06:21:18.373348 15024 master.cpp:626] Master terminating
> I0731 06:21:18.385665 15017 leveldb.cpp:343] Persisting action (
> bytes) to leveldb took 12.87577ms
> I0731 06:21:18.385743 15017 replica.cpp:676] Persisted action at 5
> I0731 06:21:18.401689 15024 slave.cpp:2332] master@140.211.11.27:45430
> exited
> W0731 06:21:18.401705 15024 slave.cpp:2335] Master disconnected! Waiting
> for a new master to be elected
> I0731 06:21:18.401717 15024 slave.cpp:2332] master@140.211.11.27:45430
> exited
> W0731 06:21:18.401726 15024 slave.cpp:2335] Master disconnected! Waiting
> for a new master to be elected
> I0731 06:21:18.401737 15024 slave.cpp:2332] master@140.211.11.27:45430
> exited
> W0731 06:21:18.401746 15024 slave.cpp:2335] Master disconnected! Waiting
> for a new master to be elected
> I0731 06:21:18.401757 15024 slave.cpp:2332] master@140.211.11.27:45430
> exited
> W0731 06:21:18.401764 15024 slave.cpp:2335] Master disconnected! Waiting
> for a new master to be elected
> I0731 06:21:18.401777 15024 slave.cpp:2332] master@140.211.11.27:45430
> exited
> W0731 06:21:18.401783 15024 slave.cpp:2335] Master disconnected! Waiting
> for a new master to be elected
> I0731 06:21:18.401795 15024 slave.cpp:2332] master@140.211.11.27:45430
> exited
> W0731 06:21:18.401803 15024 slave.cpp:2335] Master disconnected! Waiting
> for a new master to be elected
> I0731 06:21:18.401815 15024 slave.cpp:2332] master@140.211.11.27:45430
> exited
> W0731 06:21:18.401823 15024 slave.cpp:2335] Master disconnected! Waiting
> for a new master to be elected
> I0731 06:21:18.401834 15024 slave.cpp:2332] master@140.211.11.27:45430
> exited
> W0731 06:21:18.401841 15024 slave.cpp:2335] Master disconnected! Waiting
> for a new master to be elected
> I0731 06:21:18.401854 15024 slave.cpp:2332] master@140.211.11.27:45430
> exited
> W0731 06:21:18.401860 15024 slave.cpp:2335] Master disconnected! Waiting
> for a new master to be elected
> I0731 06:21:18.401872 15024 slave.cpp:2332] master@140.211.11.27:45430
> exited
> W0731 06:21:18.401880 15024 slave.cpp:2335] Master disconnected!

Re: Review Request 23224: Refactored the python bindings into multiple modules.

2014-07-31 Thread Thomas Rampelberg


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23224/
---

(Updated July 31, 2014, 10:05 p.m.)


Review request for mesos.


Changes
---

Of course bsd and gnu find aren't the same. No need for crazy regex, just use 
-o instead.


Bugs: MESOS-857
https://issues.apache.org/jira/browse/MESOS-857


Repository: mesos-git


Description
---

The existing module has been split into two separate ones:

- mesos.interface - This contains the stub implementations for the 
Executor/Scheduler as well as the protobufs.
- mesos.native - The old _mesos module.

There is also a base metapackage `mesos` that allows a potential `pip install 
mesos` to correctly install everything required. While mesos.interface can now 
be uploaded to the cheeseshop, mesos.native has not changed and will need some 
more work first.



Diffs (updated)
-

  Makefile.am b91d8cf011832e6e91b16f03a2d80fbb601eba8f 
  configure.ac e7472081339fc9c773eb2cf2d5f15dc459ac378d 
  docs/upgrades.md 919f0108655568d91060e6db998094e6dc39127b 
  mpi/mpiexec-mesos.in da0733fc29f97e67385cab55d60d4e2afd76aba9 
  mpi/mpiexec-mesos.py 0ab50167eaa43f9d69f37b7c10e26fa7a7d9f250 
  src/Makefile.am b660d912c92594ef679b71caf508134f20511dae 
  src/examples/python/test-containerizer.in 
569519b3b9755959f9bf931d3c81be9a00b64bc9 
  src/examples/python/test-executor.in 7e8875f0fd74dc9f9207986864edbce588ec3fb8 
  src/examples/python/test-framework.in 
c4683b97b87ba8753e842b0c75cc3d65140a5cf7 
  src/examples/python/test_containerizer.py 
c65d891539bcee775741626596997afe8471c930 
  src/examples/python/test_executor.py 065b50a6146cb39a82024d82c20cf89f940a9e57 
  src/examples/python/test_framework.py 
fce090fe542e3863770d7daea3d8764da1d8d5df 
  src/python/interface/setup.py.in PRE-CREATION 
  src/python/interface/src/mesos/__init__.py PRE-CREATION 
  src/python/native/ext_def.py.in PRE-CREATION 
  src/python/native/mesos_executor_driver_impl.hpp  
  src/python/native/mesos_executor_driver_impl.cpp  
  src/python/native/mesos_scheduler_driver_impl.hpp  
  src/python/native/mesos_scheduler_driver_impl.cpp  
  src/python/native/module.hpp  
  src/python/native/module.cpp b94712681e6f0e9bf5dfdafa10621d1df82dc367 
  src/python/native/proxy_executor.hpp  
  src/python/native/proxy_executor.cpp  
  src/python/native/proxy_scheduler.hpp  
  src/python/native/proxy_scheduler.cpp  
  src/python/native/setup.py.in PRE-CREATION 
  src/python/native/src/mesos/__init__.py PRE-CREATION 
  src/python/native/src/mesos/native/__init__.py PRE-CREATION 
  src/python/protocol/setup.py.in PRE-CREATION 
  src/python/protocol/src/mesos/__init__.py PRE-CREATION 
  src/python/protocol/src/mesos/protocol/__init__.py PRE-CREATION 
  src/python/setup.py.in 60add3b39d09dc5a1b74764b2b6c4b3c461f150a 
  src/python/src/mesos.py 0152ab456f072f8d4a1c4ab19fe74e181eadbd05 
  src/python/src/mesos/__init__.py PRE-CREATION 

Diff: https://reviews.apache.org/r/23224/diff/


Testing
---

- `make distcheck -j6` (note that this runs the actual python framework tests)
- `import mesos.native`
- `python setup.py sdist bdist_egg register upload` && `pip install 
mesos.interface`


Thanks,

Thomas Rampelberg

Re: Review Request 23771: Added a Docker containerizer.

2014-07-31 Thread Timothy Chen



> On July 31, 2014, 6:39 p.m., Jie Yu wrote:
> > src/docker/docker.cpp, lines 288-294
> > 
> >
> > You should be able to chain remove with check:
> > 
> > return check(cmd, s).then(lambda::bind(&__kill, *this, container, 
> > lambda::_1));
> > 
> > __kill(docker, container, force)
> > {
> >   docker.rm(container, force);
> > }

I actually can't since I want to call remove regardless of status.


- Timothy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23771/#review49240
---


On July 31, 2014, 8:06 a.m., Timothy Chen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23771/
> ---
> 
> (Updated July 31, 2014, 8:06 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, Ian Downes, and Jie Yu.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Docker implementation.
> This is all the docker code Ben, I and Yifan worked on excluding the 
> composing containerizer patches.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am 45afcd1 
>   src/common/status_utils.hpp 1980551 
>   src/docker/docker.hpp PRE-CREATION 
>   src/docker/docker.cpp PRE-CREATION 
>   src/examples/docker_no_executor_framework.cpp PRE-CREATION 
>   src/health-check/main.cpp 707810a 
>   src/launcher/executor.cpp 9c80848 
>   src/linux/cgroups.hpp decad9d 
>   src/linux/cgroups.cpp 6a73dd7 
>   src/master/master.cpp 251b699 
>   src/slave/containerizer/containerizer.cpp 1b71f33 
>   src/slave/containerizer/docker.hpp PRE-CREATION 
>   src/slave/containerizer/docker.cpp PRE-CREATION 
>   src/slave/containerizer/external_containerizer.cpp 3f28d85 
>   src/slave/containerizer/isolators/cgroups/cpushare.hpp 780037b 
>   src/slave/containerizer/isolators/cgroups/cpushare.cpp 3265a80 
>   src/slave/containerizer/isolators/cgroups/mem.hpp 8c476c7 
>   src/slave/containerizer/isolators/cgroups/mem.cpp e8d1e35 
>   src/slave/containerizer/isolators/posix.hpp 17bbd10 
>   src/slave/flags.hpp 1fe7b7d 
>   src/slave/slave.cpp f42ab60 
>   src/tests/cgroups_tests.cpp 01cf498 
>   src/tests/docker_containerizer_tests.cpp PRE-CREATION 
>   src/tests/docker_tests.cpp PRE-CREATION 
>   src/tests/environment.cpp 434b3f7 
>   src/tests/flags.hpp a003e7f 
>   src/tests/script.cpp 15a6542 
>   src/usage/usage.hpp 5a76746 
>   src/usage/usage.cpp 29014d1 
> 
> Diff: https://reviews.apache.org/r/23771/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Timothy Chen
> 
>

Re: Review Request 24128: Add 'max_num_executors' flag for use with network isolation

2014-07-31 Thread Mesos ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24128/#review49288
---


Patch looks great!

Reviews applied: [24128]

All tests passed.

- Mesos ReviewBot


On July 31, 2014, 6:54 p.m., Dominic Hamon wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24128/
> ---
> 
> (Updated July 31, 2014, 6:54 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Jie Yu.
> 
> 
> Bugs: MESOS-1654
> https://issues.apache.org/jira/browse/MESOS-1654
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> when network isolation is enabled, we should keep track of how many executors 
> have launched on a given host to avoid sending offers for those that have hit 
> the container limit.
> 
> this is a short-term fix for the issue to enable broader testing of network 
> monitoring and isolation.
> 
> 
> Diffs
> -
> 
>   src/master/flags.hpp 848d28b2da2df4981d5fceee07a014069d938394 
>   src/master/master.cpp 273a516a964f586183557e270d34b55c5a34036b 
> 
> Diff: https://reviews.apache.org/r/24128/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Dominic Hamon
> 
>

Re: Review Request 24128: Add 'max_num_executors' flag for use with network isolation

2014-07-31 Thread Dominic Hamon


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24128/
---

(Updated July 31, 2014, 11:54 a.m.)


Review request for mesos, Benjamin Hindman and Jie Yu.


Changes
---

rebased to master


Bugs: MESOS-1654
https://issues.apache.org/jira/browse/MESOS-1654


Repository: mesos-git


Description
---

when network isolation is enabled, we should keep track of how many executors 
have launched on a given host to avoid sending offers for those that have hit 
the container limit.

this is a short-term fix for the issue to enable broader testing of network 
monitoring and isolation.


Diffs (updated)
-

  src/master/flags.hpp 848d28b2da2df4981d5fceee07a014069d938394 
  src/master/master.cpp 273a516a964f586183557e270d34b55c5a34036b 

Diff: https://reviews.apache.org/r/24128/diff/


Testing
---

make check


Thanks,

Dominic Hamon

Re: Review Request 24129: Fix network isolator build

2014-07-31 Thread Jie Yu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24129/#review49268
---

Ship it!



src/tests/port_mapping_tests.cpp


ASSERT_SOME(entries);



src/tests/port_mapping_tests.cpp


ASSERT_SOME(files);


- Jie Yu


On July 31, 2014, 4:50 p.m., Dominic Hamon wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24129/
> ---
> 
> (Updated July 31, 2014, 4:50 p.m.)
> 
> 
> Review request for mesos and Jie Yu.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Changes to stout were not propagated to the code in the WITH_NETWORK_ISOLATOR 
> define blocks.
> 
> 
> Diffs
> -
> 
>   src/tests/port_mapping_tests.cpp 657463a32f788a219ec4cae95e7cd8d605828d60 
> 
> Diff: https://reviews.apache.org/r/24129/diff/
> 
> 
> Testing
> ---
> 
> ./configure --with-network-isolator
> make check
> 
> 
> Thanks,
> 
> Dominic Hamon
> 
>

Re: Review Request 23771: Added a Docker containerizer.

2014-07-31 Thread Jie Yu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23771/#review49240
---

src/docker/docker.cpp

CHECK_READY(s.status());
CHECK_SOME(s.status().get());

src/docker/docker.cpp

See my comments below.

src/docker/docker.cpp

This has been gone through a few iterations already. Sorry, this is my 
fault that I haven't thought about it in deep. But I hope we can resolve this 
before shipping this.

A few issues:

1) This is a library, we usually don't have LOG() in the library 
(otherwise, very likely we'll have double logging). Instead, we usually put 
error messages in the return value (either Future or Try).

2) Returning 'bool' doesn't seem to be necessary? We always return false on 
error and return true on success, right? In that case, we can just use 
Future?

So here is what I would suggest (s/success/check/ maybe):

Future check(const string& cmd, const Subprocess& s)
{
  return s.status().then(lambda::bind(&_success, cmd, s);
}

Future _check(const string& cmd, const Subprocess& s)
{
  Option status = s.status().get();
  if (status.isNone()) {
return Failure("No status found for '" + cmd + "'");
  }

  if (status.get() != 0) {
return err(s).then(lambda::bind(&__check, cmd, status, lambda::_1));
  }

  return Nothing();
}

Future __check(const string& cmd, int status, const string& err)
{
  return Failure(
  "Failed to '" + cmd + "': exit status = " +
  WSTRINGIFY(status) + " stderr = " + err);
}

// Retrieve the standard error of 's'.
Future err(const Subprocess& s)
{
  CHECK_SOME(s.err());

  Try nonblock = os::nonblock(s.err().get());
  if (nonblock.isError()) {
return Failure("Cannot set nonblock for stderr");
  }

  return io::read(s.err().get());
}

src/docker/docker.cpp

Add a CHECK_READY(s.status());

src/docker/docker.cpp

Here, you can just do:

return check(cmd, s);

src/docker/docker.cpp

You should be able to chain remove with check:

return check(cmd, s).then(lambda::bind(&__kill, *this, container, 
lambda::_1));

__kill(docker, container, force)
{
  docker.rm(container, force);
}

src/docker/docker.cpp

This should fit in the above line?

src/docker/docker.cpp

pass in cmd here

src/docker/docker.cpp

if (status.isNone()) {
  return Failure(...);
}

src/docker/docker.cpp

if (status.get() != 0) {
  return err(s).then(lambda::bind(&__check, cmd, status.get(), lambda::_1));
}

src/docker/docker.cpp

Ditto. Passing cmd in.

src/docker/docker.cpp

Ditto.

src/slave/containerizer/docker.cpp

If 'docker.run' returns Nothing, you won't have a chance to execute this 
code, but I think it's ok because Self::destroy will do 'docker.kill' in anyway.

src/slave/containerizer/docker.cpp

Logging format please.

src/slave/containerizer/docker.cpp

Logging format.

src/slave/containerizer/docker.cpp

Ditto.

src/tests/environment.cpp

I think this has been fixed. YOu need to do a rebase.

src/tests/environment.cpp

Hum, you switch the order. But that doesn't solve the problem. My point 
here is that:

This function should never return true early, otherwise, you'll miss the 
following checks. So it should be

if (docker.isError()) {
  return false;
}

#ifdef __linux__
if (user.get() != "root") {
  return false;
}
#endif

src/tests/environment.cpp

This is buggy too (not your fault). Do a rebase, it should be fixed. If 
not, fix it.

- Jie Yu

On July 31, 2014, 8:06 a.m., Timothy Chen wrote:
> 
> ---
> This is an automatical

Re: Review Request 23771: Added a Docker containerizer.

2014-07-31 Thread Mesos ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23771/#review49252
---


Bad patch!

Reviews applied: [23771]

Failed command: git apply --index 23771.patch

Error:
 error: patch failed: src/Makefile.am:223
error: src/Makefile.am: patch does not apply
error: patch failed: src/common/status_utils.hpp:22
error: src/common/status_utils.hpp: patch does not apply
error: patch failed: src/slave/containerizer/containerizer.cpp:36
error: src/slave/containerizer/containerizer.cpp: patch does not apply
error: patch failed: src/slave/flags.hpp:339
error: src/slave/flags.hpp: patch does not apply
error: patch failed: src/slave/slave.cpp:2426
error: src/slave/slave.cpp: patch does not apply


- Mesos ReviewBot


On July 31, 2014, 8:06 a.m., Timothy Chen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23771/
> ---
> 
> (Updated July 31, 2014, 8:06 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman, Ian Downes, and Jie Yu.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Docker implementation.
> This is all the docker code Ben, I and Yifan worked on excluding the 
> composing containerizer patches.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am 45afcd1 
>   src/common/status_utils.hpp 1980551 
>   src/docker/docker.hpp PRE-CREATION 
>   src/docker/docker.cpp PRE-CREATION 
>   src/examples/docker_no_executor_framework.cpp PRE-CREATION 
>   src/health-check/main.cpp 707810a 
>   src/launcher/executor.cpp 9c80848 
>   src/linux/cgroups.hpp decad9d 
>   src/linux/cgroups.cpp 6a73dd7 
>   src/master/master.cpp 251b699 
>   src/slave/containerizer/containerizer.cpp 1b71f33 
>   src/slave/containerizer/docker.hpp PRE-CREATION 
>   src/slave/containerizer/docker.cpp PRE-CREATION 
>   src/slave/containerizer/external_containerizer.cpp 3f28d85 
>   src/slave/containerizer/isolators/cgroups/cpushare.hpp 780037b 
>   src/slave/containerizer/isolators/cgroups/cpushare.cpp 3265a80 
>   src/slave/containerizer/isolators/cgroups/mem.hpp 8c476c7 
>   src/slave/containerizer/isolators/cgroups/mem.cpp e8d1e35 
>   src/slave/containerizer/isolators/posix.hpp 17bbd10 
>   src/slave/flags.hpp 1fe7b7d 
>   src/slave/slave.cpp f42ab60 
>   src/tests/cgroups_tests.cpp 01cf498 
>   src/tests/docker_containerizer_tests.cpp PRE-CREATION 
>   src/tests/docker_tests.cpp PRE-CREATION 
>   src/tests/environment.cpp 434b3f7 
>   src/tests/flags.hpp a003e7f 
>   src/tests/script.cpp 15a6542 
>   src/usage/usage.hpp 5a76746 
>   src/usage/usage.cpp 29014d1 
> 
> Diff: https://reviews.apache.org/r/23771/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Timothy Chen
> 
>

Re: Review Request 24128: Add 'max_num_executors' flag for use with network isolation

2014-07-31 Thread Dominic Hamon


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24128/
---

(Updated July 31, 2014, 10:09 a.m.)


Review request for mesos, Benjamin Hindman and Jie Yu.


Changes
---

rename flag. now depends on the fix for the network isolator build.


Bugs: MESOS-1654
https://issues.apache.org/jira/browse/MESOS-1654


Repository: mesos-git


Description
---

when network isolation is enabled, we should keep track of how many executors 
have launched on a given host to avoid sending offers for those that have hit 
the container limit.

this is a short-term fix for the issue to enable broader testing of network 
monitoring and isolation.


Diffs (updated)
-

  src/master/flags.hpp 848d28b2da2df4981d5fceee07a014069d938394 
  src/master/master.cpp 273a516a964f586183557e270d34b55c5a34036b 

Diff: https://reviews.apache.org/r/24128/diff/


Testing
---

make check


Thanks,

Dominic Hamon

Re: Review Request 24129: Fix network isolator build

2014-07-31 Thread Dominic Hamon


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24129/
---

(Updated July 31, 2014, 9:50 a.m.)


Review request for mesos and Jie Yu.


Repository: mesos-git


Description
---

Changes to stout were not propagated to the code in the WITH_NETWORK_ISOLATOR 
define blocks.


Diffs (updated)
-

  src/tests/port_mapping_tests.cpp 657463a32f788a219ec4cae95e7cd8d605828d60 

Diff: https://reviews.apache.org/r/24129/diff/


Testing
---

./configure --with-network-isolator
make check


Thanks,

Dominic Hamon

Re: Review Request 24129: Fix network isolator build

2014-07-31 Thread Mesos ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24129/#review49226
---


Patch looks great!

Reviews applied: [24129]

All tests passed.

- Mesos ReviewBot


On July 31, 2014, 12:07 a.m., Dominic Hamon wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24129/
> ---
> 
> (Updated July 31, 2014, 12:07 a.m.)
> 
> 
> Review request for mesos and Jie Yu.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Changes to stout were not propagated to the code in the WITH_NETWORK_ISOLATOR 
> define blocks.
> 
> 
> Diffs
> -
> 
>   src/tests/port_mapping_tests.cpp 657463a32f788a219ec4cae95e7cd8d605828d60 
> 
> Diff: https://reviews.apache.org/r/24129/diff/
> 
> 
> Testing
> ---
> 
> ./configure --with-network-isolator
> make check
> 
> 
> Thanks,
> 
> Dominic Hamon
> 
>

Re: Review Request 24128: Add 'max_num_executors' flag for use with network isolation

2014-07-31 Thread Mesos ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24128/#review49218
---


Bad patch!

Reviews applied: [24128]

Failed command: ./support/mesos-style.py

Error:
 Checking 494 files using filter 
--filter=-,+build/class,+build/deprecated,+build/endif_comment,+runtime/vlog,+whitespace/blank_line,+whitespace/comma,+whitespace/ending_newline,+whitespace/forcolon,+whitespace/indent,+whitespace/line_length,+whitespace/tab,+whitespace/todo
src/master/flags.hpp:293:  Redundant blank line at the end of a code block 
should be deleted.  [whitespace/blank_line] [3]
Total errors found: 1


- Mesos ReviewBot


On July 31, 2014, 12:05 a.m., Dominic Hamon wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24128/
> ---
> 
> (Updated July 31, 2014, 12:05 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Jie Yu.
> 
> 
> Bugs: MESOS-1654
> https://issues.apache.org/jira/browse/MESOS-1654
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> when network isolation is enabled, we should keep track of how many executors 
> have launched on a given host to avoid sending offers for those that have hit 
> the container limit.
> 
> this is a short-term fix for the issue to enable broader testing of network 
> monitoring and isolation.
> 
> 
> Diffs
> -
> 
>   src/master/flags.hpp 848d28b2da2df4981d5fceee07a014069d938394 
>   src/master/master.cpp 273a516a964f586183557e270d34b55c5a34036b 
> 
> Diff: https://reviews.apache.org/r/24128/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Dominic Hamon
> 
>

Re: Review Request 24124: Added a slave metric for exposing the number of active containers.

2014-07-31 Thread Mesos ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24124/#review49216
---


Bad patch!

Reviews applied: [24020]

Failed command: git apply --index 24020.patch

Error:
 error: patch failed: 
src/slave/containerizer/isolators/network/port_mapping.hpp:239
error: src/slave/containerizer/isolators/network/port_mapping.hpp: patch does 
not apply
error: patch failed: 
src/slave/containerizer/isolators/network/port_mapping.cpp:1060
error: src/slave/containerizer/isolators/network/port_mapping.cpp: patch does 
not apply


- Mesos ReviewBot


On July 30, 2014, 10:55 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24124/
> ---
> 
> (Updated July 30, 2014, 10:55 p.m.)
> 
> 
> Review request for mesos, Ben Mahler, Dominic Hamon, and Ian Downes.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> See summary.
> 
> 
> Diffs
> -
> 
>   src/slave/slave.hpp a896bb6 
>   src/slave/slave.cpp 1d56918 
>   src/tests/slave_tests.cpp e45255a 
> 
> Diff: https://reviews.apache.org/r/24124/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 23771: Added a Docker containerizer.

2014-07-31 Thread Timothy Chen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23771/
---

(Updated July 31, 2014, 8:06 a.m.)


Review request for mesos, Benjamin Hindman, Ian Downes, and Jie Yu.


Repository: mesos-git


Description
---

Docker implementation.
This is all the docker code Ben, I and Yifan worked on excluding the composing 
containerizer patches.


Diffs (updated)
-

  src/Makefile.am 45afcd1 
  src/common/status_utils.hpp 1980551 
  src/docker/docker.hpp PRE-CREATION 
  src/docker/docker.cpp PRE-CREATION 
  src/examples/docker_no_executor_framework.cpp PRE-CREATION 
  src/health-check/main.cpp 707810a 
  src/launcher/executor.cpp 9c80848 
  src/linux/cgroups.hpp decad9d 
  src/linux/cgroups.cpp 6a73dd7 
  src/master/master.cpp 251b699 
  src/slave/containerizer/containerizer.cpp 1b71f33 
  src/slave/containerizer/docker.hpp PRE-CREATION 
  src/slave/containerizer/docker.cpp PRE-CREATION 
  src/slave/containerizer/external_containerizer.cpp 3f28d85 
  src/slave/containerizer/isolators/cgroups/cpushare.hpp 780037b 
  src/slave/containerizer/isolators/cgroups/cpushare.cpp 3265a80 
  src/slave/containerizer/isolators/cgroups/mem.hpp 8c476c7 
  src/slave/containerizer/isolators/cgroups/mem.cpp e8d1e35 
  src/slave/containerizer/isolators/posix.hpp 17bbd10 
  src/slave/flags.hpp 1fe7b7d 
  src/slave/slave.cpp f42ab60 
  src/tests/cgroups_tests.cpp 01cf498 
  src/tests/docker_containerizer_tests.cpp PRE-CREATION 
  src/tests/docker_tests.cpp PRE-CREATION 
  src/tests/environment.cpp 434b3f7 
  src/tests/flags.hpp a003e7f 
  src/tests/script.cpp 15a6542 
  src/usage/usage.hpp 5a76746 
  src/usage/usage.cpp 29014d1 

Diff: https://reviews.apache.org/r/23771/diff/


Testing
---

make check


Thanks,

Timothy Chen

Re: Review Request 24123: Fixed a flaky test: ZooKeeperTest.LeaderDetectorTimeoutHandling

Review Request 24178: Basic filesystem isolator for Linux.

Review Request 24177: Pass executor directory to Isolator::prepare().

Jenkins build is back to normal : Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME #2279

Re: A question on deploying mesos framework to the cluster

Re: Review Request 23771: Added a Docker containerizer.

Build failed in Jenkins: Mesos-Ubuntu-distcheck #245

Re: Review Request 23224: Refactored the python bindings into multiple modules.

Re: Review Request 23224: Refactored the python bindings into multiple modules.

Re: Review Request 23224: Refactored the python bindings into multiple modules.

Re: Review Request 23224: Refactored the python bindings into multiple modules.

Re: Review Request 23224: Refactored the python bindings into multiple modules.

Re: Review Request 23771: Added a Docker containerizer.

Re: Review Request 23771: Added a Docker containerizer.

Re: Review Request 23224: Refactored the python bindings into multiple modules.

Re: Review Request 23224: Refactored the python bindings into multiple modules.

Re: Review Request 23771: Added a Docker containerizer.

Jenkins build is back to normal : Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME #2022

Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME #2278

Re: A question on deploying mesos framework to the cluster

Re: Review Request 23771: Added a Docker containerizer.

Re: Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME #2277

Re: Review Request 23224: Refactored the python bindings into multiple modules.

Re: Review Request 23771: Added a Docker containerizer.

Re: Review Request 24128: Add 'max_num_executors' flag for use with network isolation

Re: Review Request 24128: Add 'max_num_executors' flag for use with network isolation

Re: Review Request 24129: Fix network isolator build

Re: Review Request 23771: Added a Docker containerizer.

Re: Review Request 23771: Added a Docker containerizer.

Re: Review Request 24128: Add 'max_num_executors' flag for use with network isolation

Re: Review Request 24129: Fix network isolator build

Re: Review Request 24129: Fix network isolator build

Re: Review Request 24128: Add 'max_num_executors' flag for use with network isolation

Re: Review Request 24124: Added a slave metric for exposing the number of active containers.

Re: Review Request 23771: Added a Docker containerizer.

35 matches

Site Navigation

Mail list logo

Footer information