Re: Review Request 71865: Moved creating authorization Object out of `Master::authorizeSlave`.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71865/#review218906
---



Bad review!

Reviews applied: [71865, 71864, 71863, 71862, 71861, 71860, 71859]

Error:
2019-12-04 07:50:02 URL:https://reviews.apache.org/r/71861/diff/raw/ 
[10266/10266] -> "71861.patch" [1]
error: patch failed: src/master/http.cpp:154
error: src/master/http.cpp: patch does not apply

- Mesos Reviewbot


On Dec. 3, 2019, 5:55 p.m., Andrei Sekretenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71865/
> ---
> 
> (Updated Dec. 3, 2019, 5:55 p.m.)
> 
> 
> Review request for mesos, Benjamin Bannier and Benjamin Mahler.
> 
> 
> Bugs: MESOS-10023 and MESOS-10056
> https://issues.apache.org/jira/browse/MESOS-10023
> https://issues.apache.org/jira/browse/MESOS-10056
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Moved creating authorization Object out of `Master::authorizeSlave`.
> 
> 
> Diffs
> -
> 
>   src/master/authorization.hpp PRE-CREATION 
>   src/master/authorization.cpp PRE-CREATION 
>   src/master/master.hpp 93630421d58e6fd26566e81a23cd910957795665 
>   src/master/master.cpp 14b90a5e276df055bb8a570331f27cab200c9869 
> 
> 
> Diff: https://reviews.apache.org/r/71865/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Andrei Sekretenko
> 
>

Re: Review Request 71858: WIP: Set resource limits when launching executor container.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71858/#review218904
---



Patch looks great!

Reviews applied: [71855, 71856, 71858]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers 
--disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker-build.sh

- Mesos Reviewbot


On Dec. 3, 2019, 2:07 p.m., Qian Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71858/
> ---
> 
> (Updated Dec. 3, 2019, 2:07 p.m.)
> 
> 
> Review request for mesos, Andrei Budnik and Greg Mann.
> 
> 
> Bugs: MESOS-10046
> https://issues.apache.org/jira/browse/MESOS-10046
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Set resource limits when launching executor container.
> 
> 
> Diffs
> -
> 
>   src/slave/slave.cpp 3839a120446339fea8aa857f431a2dba28ed4002 
> 
> 
> Diff: https://reviews.apache.org/r/71858/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Qian Zhang
> 
>

Re: Review Request 71852: Used CMake's automatic parallelization in mesos-tidy setup.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71852/#review218899
---



Patch looks great!

Reviews applied: [71852]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers 
--disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker-build.sh

- Mesos Reviewbot


On Dec. 3, 2019, 1:12 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71852/
> ---
> 
> (Updated Dec. 3, 2019, 1:12 p.m.)
> 
> 
> Review request for mesos and Benno Evers.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Since the `--parallel` flag for `cmake build` is only supported in more
> recent CMake versions we bump the installed version to the latest
> release.
> 
> 
> Diffs
> -
> 
>   support/mesos-tidy/Dockerfile 7d17a82c023c614016e771c305e3a114731eb832 
>   support/mesos-tidy/entrypoint.sh 7d2225d8f75998ee13acf0bd57c9483dfc7acd98 
> 
> 
> Diff: https://reviews.apache.org/r/71852/diff/1/
> 
> 
> Testing
> ---
> 
> Built a local image and tested it with a modified `support/mesos-tidy.sh`.
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 71664: SSL Wrapper: Implemented BIO for SSL socket wrapper.

[Joseph Wu]

> On Nov. 26, 2019, 1:59 p.m., Greg Mann wrote:
> > 3rdparty/libprocess/src/ssl/socket_wrapper.cpp
> > Lines 212-215 (patched)
> > 
> >
> > I wonder if openssl's internal calls to `BIO_flush()` will retry if the 
> > call fails... perhaps we could synchronously return zero here if there are 
> > pending writes, and return 1 if everything has been written?

No.  If this call returns 0 (or anything less than 1), the SSL library will 
consider this a `SSL_ERROR_SYSCALL` and will refuse to make any further 
progress, thereby shutting down the socket.  I haven't found any way around 
this, but I also have not seen any consequences from lying about the flush 
either.


- Joseph


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71664/#review218804
---


On Nov. 19, 2019, 12:16 p.m., Joseph Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71664/
> ---
> 
> (Updated Nov. 19, 2019, 12:16 p.m.)
> 
> 
> Review request for mesos, Benno Evers, Benjamin Mahler, Greg Mann, and Till 
> Toenshoff.
> 
> 
> Bugs: MESOS-10009
> https://issues.apache.org/jira/browse/MESOS-10009
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This implements the OpenSSL basic I/O abstraction based on the
> libprocess event loop.  This BIO wraps a socket and handles the
> reading/writing, using io::read and io::write.
> 
> This BIO can be passed into an SSL context to enable usage of
> SSL translation functions like SSL_read and SSL_write.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/src/ssl/socket_wrapper.cpp PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71664/diff/3/
> 
> 
> Testing
> ---
> 
> cmake --build . --target process
> 
> A tiny bit of testing next patch.
> 
> 
> Thanks,
> 
> Joseph Wu
> 
>

Re: Review Request 71827: Improved operator api subscribe initial payload performance.

[Andrei Sekretenko]

> On Dec. 2, 2019, 1:23 p.m., Andrei Sekretenko wrote:
> > src/master/http.cpp
> > Lines 444 (patched)
> > 
> >
> > Shouldn't we delete `_getState`, `_getFrameworks` and so on altogether? 
> > 
> > They are dead code now: not used anywhere, not covered by tests and, 
> > essentialy, not maintained. I don't think their correctness can be relied 
> > upon after this patch.
> > 
> > Probably they should be removed in a separate next patch so that if we 
> > need them back someday, the first step to getting them back will be 
> > reverting that patch.
> > 
> > And the comments will need to be adjusted to not refer to them as well.
> 
> Benjamin Mahler wrote:
> Good observation!
> 
> When writing the code, these were useful for debugging to compare the 
> serialized bytes against the expected serialized bytes (turns out the only 
> bug was my lack of calling Trim() which left trailing garbage bytes at the 
> end of the string).
> 
> We should consider the case where we have a serialization bug, and we 
> need to investigate it. Having the expected vs actual will be helpful, but we 
> won't have that if we remove the object creation functions. If it's just as 
> simple as a field being missing, then that's pretty easy (probably just 
> forgot to write it). But if it's more like the data has some garbage in it, 
> that may be harder to debug especially without the actual vs expected bytes 
> handy. Thoughts on this?
> 
> Another thing I was considering (but couldn't see a clean way to do) was 
> to have our test suite somehow assert that the two paths produce identical 
> results, e.g.:
> 
>  - If running within tests, have the endpoint handler run both paths and 
> CHECK they produce the same serialized bytes.
>  - Or, introduce some test visible function in the master that could get 
> called at some point in any test lifecycle (e.g. cleanup, or periodically) to 
> assert that both produce equal results.
> 
> They both seemed pretty heavy to leave around in the code, and I became 
> less paranoid about serialization bugs, so I decided not to do these.

Hmm.. if we want to make use of old `_getState`, I would propose two options:

1. Add a flag to the GET_STATE call which would switch it to the old 
serialization path, and add a local test that will compare results of two 
GET_STATEs, old path vs new.
This might be a good aid in case someone needs to debug protobuf serialization 
again. Can imagine this happending when updating the bundled protobuf. 
Also, this will somewhat alleviate the concern about the unmaintained 
`_getState` eventually becoming buggy and non-usable.
The coverage of such test will be rather poor, though.

2. Add a master flag that, as you suggested, will make it run both paths and 
crash the master if the results are different. 
Executed on a testing cluster, this will give a much better coverage. The price 
will be introducing more complexity into GET_STATE.


- Andrei


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71827/#review218878
---


On Nov. 26, 2019, 9:31 p.m., Benjamin Mahler wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71827/
> ---
> 
> (Updated Nov. 26, 2019, 9:31 p.m.)
> 
> 
> Review request for mesos, Andrei Sekretenko and Greg Mann.
> 
> 
> Bugs: MESOS-10026
> https://issues.apache.org/jira/browse/MESOS-10026
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This uses the same approach for other GET_ calls in MESOS-10026
> of directly serializing from in-memory state, rather than building
> up the temporary object and evolving it.
> 
> There is currently no benchmark but the improvement should closely
> resemble that of the GET_STATE call, for example:
> 
> Before:
> v0 '/state' response took 6.55 secs
> v1 'GetState' application/x-protobuf response took 24.08 secs
> v1 'GetState' application/json response took 22.76 secs
> 
> After:
> v0 '/state' response took 8.00 secs
> v1 'GetState' application/x-protobuf response took 5.73 secs
> v1 'GetState' application/json response took 9.62 secs
> 
> 
> Diffs
> -
> 
>   src/master/http.cpp e03655863ea2d4e4464b3d14b359de3d7f059778 
>   src/master/master.hpp 93630421d58e6fd26566e81a23cd910957795665 
> 
> 
> Diff: https://reviews.apache.org/r/71827/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>

Review Request 71867: WIP: Support for docker containerizer.

[Benno Evers]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71867/
---

Review request for mesos and Benjamin Bannier.


Repository: mesos


Description
---

Note that currently we cannot test the code, because that
would require launching an executor inside a docker container,
which seems to be unprecedented in unit tests.


Diffs
-

  src/slave/containerizer/docker.hpp 0349f537cef9651427e0e3ed33fd693107e07aa0 
  src/slave/containerizer/docker.cpp 2a9b2ffcbd01ae916839ae43c8342285ac3e14a2 
  src/tests/containerizer/docker_containerizer_tests.cpp 
689a7220a09f2a58dffdf0dc9fd9f0548600be0e 


Diff: https://reviews.apache.org/r/71867/diff/1/


Testing
---


Thanks,

Benno Evers

Re: Review Request 71836: WIP: Bind-mount domain socket into executor containers.

[Benno Evers]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71836/
---

(Updated Dec. 3, 2019, 6:33 p.m.)


Review request for mesos and Benjamin Bannier.


Changes
---

Address some linter issues, update rootfs logic.


Bugs: MESOS-10037
https://issues.apache.org/jira/browse/MESOS-10037


Repository: mesos


Description
---

WIP: Bind-mount domain socket into executor containers.


Diffs (updated)
-

  src/slave/containerizer/mesos/containerizer.cpp 
dd940162b1e44dbdab1773894b2b33a260fdc8da 
  src/tests/containerizer/linux_filesystem_isolator_tests.cpp 
2390902ce4dc6fb341ca836af95984a040e8e658 


Diff: https://reviews.apache.org/r/71836/diff/2/

Changes: https://reviews.apache.org/r/71836/diff/1-2/


Testing
---


Thanks,

Benno Evers

Review Request 71866: Added new chmod() function to stout.

[Benno Evers]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71866/
---

Review request for mesos, Benjamin Bannier and Benjamin Mahler.


Repository: mesos


Description
---

Added new functions `os::chmod()` and `os::posix::chmod()` that
are wrapping the POSIX `::chmod()` function.


Diffs
-

  3rdparty/stout/include/stout/os/chmod.hpp PRE-CREATION 
  3rdparty/stout/include/stout/os/posix/chmod.hpp PRE-CREATION 
  3rdparty/stout/include/stout/os/windows/chmod.hpp PRE-CREATION 


Diff: https://reviews.apache.org/r/71866/diff/1/


Testing
---


Thanks,

Benno Evers

Re: Review Request 71833: Created unix domain socket on agent startup.

[Benno Evers]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71833/
---

(Updated Dec. 3, 2019, 6:30 p.m.)


Review request for mesos and Benjamin Bannier.


Changes
---

Cleanup some todo's and linter issues.


Summary (updated)
-

Created unix domain socket on agent startup.


Bugs: MESOS-10036
https://issues.apache.org/jira/browse/MESOS-10036


Repository: mesos


Description (updated)
---

Added agent code to check whether domain socket support is
enabled, and if so to create or open the socket at
MESOS_DOMAIN_SOCKET_LOCATION.


Diffs (updated)
-

  src/local/local.cpp 8ff43618f06463b4ba86b64b25e5de692e406448 
  src/slave/main.cpp fd58637cd680291e6794bcdb0655603bb97744c7 
  src/slave/slave.hpp 77b5bc0082c6bb73fbd48a2ebe812629921645cb 
  src/slave/slave.cpp 3839a120446339fea8aa857f431a2dba28ed4002 
  src/tests/cluster.cpp f7bc882a644ec65710ada3d15507e1d4c5ba06f7 
  src/tests/mock_slave.cpp 71be957884ea88258ef37e60649e3947e89b12d0 


Diff: https://reviews.apache.org/r/71833/diff/4/

Changes: https://reviews.apache.org/r/71833/diff/3-4/


Testing
---


Thanks,

Benno Evers

Re: Review Request 71816: WIP: Added domain-socket related flags to Mesos agent.

[Benno Evers]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71816/
---

(Updated Dec. 3, 2019, 6:30 p.m.)


Review request for mesos and Benjamin Bannier.


Changes
---

Fix linter issues.


Bugs: MESOS-10035
https://issues.apache.org/jira/browse/MESOS-10035


Repository: mesos


Description (updated)
---

Added two new flags 'http_executor_domain_sockets' and
'domain_socket_location' to the agent, along with minimal
logic to pass the relevant values on to executors.

They are used to control whether the agent should
provide a domain socket to HTTP executors and the
location of the socket on the host filesystem, respectively.


Diffs (updated)
-

  src/slave/flags.hpp 3c5ffca62454c340656cfc89c53a198757815794 
  src/slave/flags.cpp 50b09cf3b0cf89568a45afd3fe89fa16c6a79222 
  src/slave/slave.cpp 3839a120446339fea8aa857f431a2dba28ed4002 


Diff: https://reviews.apache.org/r/71816/diff/2/

Changes: https://reviews.apache.org/r/71816/diff/1-2/


Testing
---


Thanks,

Benno Evers

Re: Review Request 71815: Made the default executors connect via domain sockets if available.

[Benno Evers]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71815/
---

(Updated Dec. 3, 2019, 6:29 p.m.)


Review request for mesos and Benjamin Bannier.


Changes
---

Add command executor test.


Summary (updated)
-

Made the default executors connect via domain sockets if available.


Bugs: MESOS-10039
https://issues.apache.org/jira/browse/MESOS-10039


Repository: mesos


Description
---

Added support for the `MESOS_DOMAIN_SOCKET` environment variable
to the default executor. If this variable is present, the executor
will use the specified domain socket to connect to the agent,
as opposed to the IP address encoded in the agent PID.


Diffs (updated)
-

  src/executor/executor.cpp b4126037ed3e14a93d4acb9d02b62115ba4ef690 
  src/launcher/default_executor.cpp 521494a2770192e39b5159624f057e01f2690c8a 
  src/tests/command_executor_tests.cpp c7f7711a7b085bdacd0f16b16a6c85cc47e2045c 
  src/tests/default_executor_tests.cpp 49c4e3b37bde848dc7a4fd02a1458c650a84a16f 


Diff: https://reviews.apache.org/r/71815/diff/3/

Changes: https://reviews.apache.org/r/71815/diff/2-3/


Testing
---

Ran the new test.


Thanks,

Benno Evers

Re: Review Request 71814: Added support for new 'http+unix' URL scheme in libprocess.

[Benno Evers]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71814/
---

(Updated Dec. 3, 2019, 6:27 p.m.)


Review request for mesos, Benjamin Bannier and Benjamin Mahler.


Changes
---

Some cleanup; experimentally removing windows ifdefs.


Summary (updated)
-

Added support for new 'http+unix' URL scheme in libprocess.


Bugs: MESOS-10039
https://issues.apache.org/jira/browse/MESOS-10039


Repository: mesos


Description
---

Added support for a new 'http+unix' URL scheme, to denote
resources accessible by HTTP connections over unix domain
sockets.

The encoding looks like

http+unix:///

optionally followed by query and fragment sections.


Diffs (updated)
-

  3rdparty/libprocess/include/process/http.hpp 
0013850f07110d9b9dbdb9b0c8ac7001a82420d2 
  3rdparty/libprocess/src/http.cpp b487ce214128193a3443f4e62a5af24205cbd399 


Diff: https://reviews.apache.org/r/71814/diff/2/

Changes: https://reviews.apache.org/r/71814/diff/1-2/


Testing
---


Thanks,

Benno Evers

Re: Review Request 71832: Added new issocket() function to stout.

[Benno Evers]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71832/
---

(Updated Dec. 3, 2019, 6:27 p.m.)


Review request for mesos, Benjamin Bannier and Benjamin Mahler.


Changes
---

Changed `strcpy()` -> `strncpy()`.


Summary (updated)
-

Added new issocket() function to stout.


Bugs: MESOS-10036
https://issues.apache.org/jira/browse/MESOS-10036


Repository: mesos


Description (updated)
---

Added a new function 'os::stat::issocket()' that tells whether a given
file is a socket or not.


Diffs (updated)
-

  3rdparty/stout/include/stout/os/posix/stat.hpp 
74e0c934baa29ec936e8011221a3da29dd67a465 
  3rdparty/stout/tests/os/socket_tests.cpp 
9ca236cb15525004d8c165ccd19f63902729b04d 


Diff: https://reviews.apache.org/r/71832/diff/3/

Changes: https://reviews.apache.org/r/71832/diff/2-3/


Testing
---

* Manual testing by using it in the subsequent patch.
* `./stout-tests`


Thanks,

Benno Evers

Review Request 71865: Moved creating authorization Object out of `Master::authorizeSlave`.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71865/
---

Review request for mesos, Benjamin Bannier and Benjamin Mahler.


Bugs: MESOS-10023 and MESOS-10056
https://issues.apache.org/jira/browse/MESOS-10023
https://issues.apache.org/jira/browse/MESOS-10056


Repository: mesos


Description
---

Moved creating authorization Object out of `Master::authorizeSlave`.


Diffs
-

  src/master/authorization.hpp PRE-CREATION 
  src/master/authorization.cpp PRE-CREATION 
  src/master/master.hpp 93630421d58e6fd26566e81a23cd910957795665 
  src/master/master.cpp 14b90a5e276df055bb8a570331f27cab200c9869 


Diff: https://reviews.apache.org/r/71865/diff/1/


Testing
---

make check


Thanks,

Andrei Sekretenko

Review Request 71864: Moved creating authorization Object out of `Master::authorize*reserve`.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71864/
---

Review request for mesos, Benjamin Bannier and Benjamin Mahler.


Bugs: MESOS-10023 and MESOS-10056
https://issues.apache.org/jira/browse/MESOS-10023
https://issues.apache.org/jira/browse/MESOS-10056


Repository: mesos


Description
---

Moved creating authorization Object out of `Master::authorize*reserve`.


Diffs
-

  src/master/authorization.hpp PRE-CREATION 
  src/master/authorization.cpp PRE-CREATION 
  src/master/http.cpp e03655863ea2d4e4464b3d14b359de3d7f059778 
  src/master/master.hpp 93630421d58e6fd26566e81a23cd910957795665 
  src/master/master.cpp 14b90a5e276df055bb8a570331f27cab200c9869 


Diff: https://reviews.apache.org/r/71864/diff/1/


Testing
---


Thanks,

Andrei Sekretenko

Review Request 71863: Moved creating authorization Object out of `Master::authorize*Volume`.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71863/
---

Review request for mesos, Benjamin Bannier and Benjamin Mahler.


Bugs: MESOS-10023 and MESOS-10056
https://issues.apache.org/jira/browse/MESOS-10023
https://issues.apache.org/jira/browse/MESOS-10056


Repository: mesos


Description
---

Moved creating authorization Object out of `Master::authorize*Volume`.


Diffs
-

  src/master/authorization.hpp PRE-CREATION 
  src/master/authorization.cpp PRE-CREATION 
  src/master/http.cpp e03655863ea2d4e4464b3d14b359de3d7f059778 
  src/master/master.hpp 93630421d58e6fd26566e81a23cd910957795665 
  src/master/master.cpp 14b90a5e276df055bb8a570331f27cab200c9869 


Diff: https://reviews.apache.org/r/71863/diff/1/


Testing
---


Thanks,

Andrei Sekretenko

Review Request 71862: Moved creating authorization Object out of `Master::authorize*Disk`.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71862/
---

Review request for mesos, Benjamin Bannier and Benjamin Mahler.


Bugs: MESOS-10023 and MESOS-10056
https://issues.apache.org/jira/browse/MESOS-10023
https://issues.apache.org/jira/browse/MESOS-10056


Repository: mesos


Description
---

Moved creating authorization Object out of `Master::authorize*Disk`.


Diffs
-

  src/master/authorization.hpp PRE-CREATION 
  src/master/authorization.cpp PRE-CREATION 
  src/master/master.hpp 93630421d58e6fd26566e81a23cd910957795665 
  src/master/master.cpp 14b90a5e276df055bb8a570331f27cab200c9869 


Diff: https://reviews.apache.org/r/71862/diff/1/


Testing
---


Thanks,

Andrei Sekretenko

Review Request 71861: Moved creating authz Object out of `Master::authorizeResizeVolume`.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71861/
---

Review request for mesos, Benjamin Bannier and Benjamin Mahler.


Bugs: MESOS-10023 and MESOS-10056
https://issues.apache.org/jira/browse/MESOS-10023
https://issues.apache.org/jira/browse/MESOS-10056


Repository: mesos


Description
---

Moved creating authz Object out of `Master::authorizeResizeVolume`.


Diffs
-

  src/master/authorization.hpp PRE-CREATION 
  src/master/authorization.cpp PRE-CREATION 
  src/master/http.cpp e03655863ea2d4e4464b3d14b359de3d7f059778 
  src/master/master.hpp 93630421d58e6fd26566e81a23cd910957795665 
  src/master/master.cpp 14b90a5e276df055bb8a570331f27cab200c9869 


Diff: https://reviews.apache.org/r/71861/diff/1/


Testing
---


Thanks,

Andrei Sekretenko

Review Request 71860: Moved creating authorization Object out of `Master::authorizeFramework`.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71860/
---

Review request for mesos, Benjamin Bannier and Benjamin Mahler.


Bugs: MESOS-10023 and MESOS-10056
https://issues.apache.org/jira/browse/MESOS-10023
https://issues.apache.org/jira/browse/MESOS-10056


Repository: mesos


Description
---

Moved creating authorization Object out of `Master::authorizeFramework`.


Diffs
-

  src/master/authorization.hpp PRE-CREATION 
  src/master/authorization.cpp PRE-CREATION 
  src/master/master.cpp 14b90a5e276df055bb8a570331f27cab200c9869 


Diff: https://reviews.apache.org/r/71860/diff/1/


Testing
---


Thanks,

Andrei Sekretenko

Review Request 71859: Moved creating authorization Object out of `Master::authorizeTask`.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71859/
---

Review request for mesos, Benjamin Bannier and Benjamin Mahler.


Bugs: MESOS-10023 and MESOS-10056
https://issues.apache.org/jira/browse/MESOS-10023
https://issues.apache.org/jira/browse/MESOS-10056


Repository: mesos


Description
---

This is the first patch in the chain that extract Master code
generating Action-Object pairs into a dedicated ActionObject class,
thus seperating authz Object creation from feeding them into authorizer.

This is a prerequisite to using ObjectApprover interface for
synchronous authorization of Scheduler API calls.


Diffs
-

  src/CMakeLists.txt ef9382dc77d37fed344b7267119f3251acd3088a 
  src/Makefile.am 111c156c8a7abc5ece04779e8ac8879a30c22dbf 
  src/common/authorization.hpp 565d5ca6620442803fa80be652ab7382102347f5 
  src/common/authorization.cpp fa71b0e8e8b9541376a9fd199f4d7b9db56a3f0f 
  src/master/authorization.hpp PRE-CREATION 
  src/master/authorization.cpp PRE-CREATION 
  src/master/master.hpp 93630421d58e6fd26566e81a23cd910957795665 
  src/master/master.cpp 14b90a5e276df055bb8a570331f27cab200c9869 


Diff: https://reviews.apache.org/r/71859/diff/1/


Testing
---


Thanks,

Andrei Sekretenko

Review Request 71858: WIP: Set resource limits when launching executor container.

[Qian Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71858/
---

Review request for mesos, Andrei Budnik and Greg Mann.


Bugs: MESOS-10046
https://issues.apache.org/jira/browse/MESOS-10046


Repository: mesos


Description
---

Set resource limits when launching executor container.


Diffs
-

  src/slave/slave.cpp 3839a120446339fea8aa857f431a2dba28ed4002 


Diff: https://reviews.apache.org/r/71858/diff/1/


Testing
---


Thanks,

Qian Zhang

Review Request 71856: Added resource limits into `ContainerConfig`.

[Qian Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71856/
---

Review request for mesos, Andrei Budnik and Greg Mann.


Bugs: MESOS-10046
https://issues.apache.org/jira/browse/MESOS-10046


Repository: mesos


Description
---

Added resource limits into `ContainerConfig`.


Diffs
-

  include/mesos/slave/containerizer.proto 
8b1a268b5ddf1b791c47219355143da8dbd03789 


Diff: https://reviews.apache.org/r/71856/diff/1/


Testing
---


Thanks,

Qian Zhang

Review Request 71855: Added resource limits into `TaskInfo` and `ExecutorInfo`.

[Qian Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71855/
---

Review request for mesos, Andrei Budnik and Greg Mann.


Bugs: MESOS-10043
https://issues.apache.org/jira/browse/MESOS-10043


Repository: mesos


Description
---

Added resource limits into `TaskInfo` and `ExecutorInfo`.


Diffs
-

  include/mesos/mesos.proto 7d77ddd8021eb7cc3919f3151594dd38606c15c5 
  include/mesos/v1/mesos.proto 7abb6ba60a1b1861338c47717a49c72a5dffa68c 


Diff: https://reviews.apache.org/r/71855/diff/1/


Testing
---


Thanks,

Qian Zhang

Review Request 71854: Handled `/api/v1` and /api/v1/executor` over agent executor socket.

[Benjamin Bannier]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71854/
---

Review request for mesos and Benno Evers.


Bugs: MESOS-10038
https://issues.apache.org/jira/browse/MESOS-10038


Repository: mesos


Description
---

This patch wires up the agent executor socket to handle calls for
`/api/v1` and `/api/v1/executor` so executors can use domain sockets to
communicate with the agent; executors use the latter to subscribe with
the agent and the former to e.g., launch containers.

Note that with this patch we now expose the full operator API over the
agent's domain socket.


Diffs
-

  src/slave/slave.hpp 77b5bc0082c6bb73fbd48a2ebe812629921645cb 
  src/slave/slave.cpp 3839a120446339fea8aa857f431a2dba28ed4002 


Diff: https://reviews.apache.org/r/71854/diff/1/


Testing
---

In both an SSL and non-SSL setup `make check` and also with tested with 
`LinuxFilesystemIsolatorMesosTest.ROOT_DomainSocketVariable` added earlier in 
this chain. The Mesos test harness with authenticate and authorize in SSL 
builds and permit all requests in non-SSL builds.


Thanks,

Benjamin Bannier

Review Request 71853: Constructed `HttpEvent` from owning instead of raw pointers.

[Benjamin Bannier]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71853/
---

Review request for mesos and Benno Evers.


Repository: mesos


Description
---

Constructed `HttpEvent` from owning instead of raw pointers.


Diffs
-

  3rdparty/libprocess/include/process/event.hpp 
6d7f8127cfe19e77ee35bcea7d881a4e5180a64e 
  3rdparty/libprocess/src/process.cpp 1de6da2d77f75fa228fbd20d4e0f46e7dcf91719 


Diff: https://reviews.apache.org/r/71853/diff/1/


Testing
---

`make check`


Thanks,

Benjamin Bannier

Review Request 71852: Used CMake's automatic parallelization in mesos-tidy setup.

[Benjamin Bannier]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71852/
---

Review request for mesos and Benno Evers.


Repository: mesos


Description
---

Since the `--parallel` flag for `cmake build` is only supported in more
recent CMake versions we bump the installed version to the latest
release.


Diffs
-

  support/mesos-tidy/Dockerfile 7d17a82c023c614016e771c305e3a114731eb832 
  support/mesos-tidy/entrypoint.sh 7d2225d8f75998ee13acf0bd57c9483dfc7acd98 


Diff: https://reviews.apache.org/r/71852/diff/1/


Testing
---

Built a local image and tested it with a modified `support/mesos-tidy.sh`.


Thanks,

Benjamin Bannier

Re: Review Request 71825: Updated mesos code to work against recordio encoder/decoder changes.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71825/#review218889
---


Ship it!




Ship It!

- Andrei Sekretenko


On Dec. 3, 2019, 5:26 a.m., Benjamin Mahler wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71825/
> ---
> 
> (Updated Dec. 3, 2019, 5:26 a.m.)
> 
> 
> Review request for mesos, Andrei Sekretenko and Greg Mann.
> 
> 
> Bugs: MESOS-10026
> https://issues.apache.org/jira/browse/MESOS-10026
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The recordio encoder and decoder were updated to operate on records
> as bytes instead of typed T records.
> 
> 
> Diffs
> -
> 
>   src/checks/checker_process.cpp c214bd1af24051b504e86948180c947c550ff3fd 
>   src/common/http.hpp b9ab561e11d52b776a283687b3a19aa1b7942522 
>   src/common/recordio.hpp 8cb2e73f16c753a85074fbf2ae9fd92ae0d2aeea 
>   src/executor/executor.cpp b4126037ed3e14a93d4acb9d02b62115ba4ef690 
>   src/resource_provider/http_connection.hpp 
> 05863aa9871d73c34993901017f53eb0f22e7886 
>   src/resource_provider/manager.cpp 427ce70a1fd746c77ef2093e82b39df9d4caff70 
>   src/scheduler/scheduler.cpp 674483aa80bc74b654343c97892a96f49d5c7ed4 
>   src/slave/containerizer/mesos/io/switchboard.cpp 
> 8e02e511db6ee049dde80d9f28fdcaeaafb21db8 
>   src/slave/http.cpp 4d68ce746a5ddb349c2cd0830f172a536ce4e551 
>   src/tests/api_tests.cpp 393f9a27afb4d329505c9e583913d2988500b195 
>   src/tests/common/recordio_tests.cpp 
> 5dd68802ed385c3f7df39e2060bd1f087dbe8ef6 
>   src/tests/containerizer/io_switchboard_tests.cpp 
> e4431450d7d97e551e0a0a757c5250816a9f9f0c 
>   src/tests/executor_http_api_tests.cpp 
> 99bcafb65b5056ce6e57fb9e69aaff5dab14584d 
>   src/tests/master/mock_master_api_subscriber.cpp 
> a0808e85ac51da360c236e86646c661c26eeb270 
>   src/tests/resource_provider_manager_tests.cpp 
> 84ec70ba76ec869f69dbb1349ad83b655fe4616e 
>   src/tests/scheduler_http_api_tests.cpp 
> d5b5eec0df0c01c4706c9bb4438c9daa305bd376 
> 
> 
> Diff: https://reviews.apache.org/r/71825/diff/2/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>

Re: Review Request 71824: Updated stout recordio encoder/decoder to be lower-level.

[Andrei Sekretenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71824/#review21
---


Ship it!




Ship It!

- Andrei Sekretenko


On Dec. 3, 2019, 5:25 a.m., Benjamin Mahler wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71824/
> ---
> 
> (Updated Dec. 3, 2019, 5:25 a.m.)
> 
> 
> Review request for mesos, Andrei Sekretenko and Greg Mann.
> 
> 
> Bugs: MESOS-10026
> https://issues.apache.org/jira/browse/MESOS-10026
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> These previously were templated to operate against typed records,
> which imposes a few limitations. Namely, the caller cannot encode
> records of different types without using some wrapper type.
> Similarly, on the decoding side if the caller expects different
> types of records based on some higher level protocol, it needs to
> use a wrapper type that just stores the record's bytes for the
> caller to decode.
> 
> The actual motivation for this patch came from a case where the
> caller already has the record in bytes when encoding. We could
> use Encoder but it imposes an extra copy of each record,
> which is not so straightforward to eliminate while keeping the
> interface simple and functional for generic T records.
> 
> So, this patch makes the recordio encoder/decoder operate on
> records as bytes, and callers can do whatever they wish with
> the bytes.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/recordio.hpp 
> 9d226c21ed4ba7f31acf8db817470085792aca45 
>   3rdparty/stout/tests/recordio_tests.cpp 
> 177939427a7ef5f15cac318a98e40cf77b0b0ada 
> 
> 
> Diff: https://reviews.apache.org/r/71824/diff/2/
> 
> 
> Testing
> ---
> 
> Needs the following patch to compile mesos.
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>