Re: Review Request 70892: Added `/containerizer/debug` endpoint.

2019-07-18 Thread Gilbert Song 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70892/#review216729
---


Ship it!




Ship It!

- Gilbert Song


On June 19, 2019, 7:49 a.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70892/
> ---
> 
> (Updated June 19, 2019, 7:49 a.m.)
> 
> 
> Review request for mesos, Gilbert Song, James Peach, and Qian Zhang.
> 
> 
> Bugs: MESOS-9829
> https://issues.apache.org/jira/browse/MESOS-9829
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch introduces an agent's `/containerizer/debug` endpoint,
> which exposes the debug info related to Mesos containerizer.
> Currently, this endpoint returns a list of pending futures related to
> isolators or containerizer launcher. This endpoint is experimental,
> and the format of its output may change over time.
> 
> 
> Diffs
> -
> 
>   docs/authorization.md 91dc03be66bdcf135610bff50e5abcfe8d95c5db 
>   src/common/authorization.cpp 89487219ef87619084663e1a0a17e75924394d38 
>   src/local/local.cpp 6ac6b027f66ef555440ca86803198bfe3227f8d7 
>   src/slave/http.hpp b8c83f1db95c2175bb94f6cd76cc2c58aa118c4e 
>   src/slave/http.cpp 321dca786f106169f88df086422f5ef817d2f098 
>   src/slave/main.cpp ef5ea02f169427d9913b807664bf3073d72cd9b6 
>   src/slave/slave.hpp 58a5608ceb53d7f75a085f1d3a4f757e8c3ddd3b 
>   src/slave/slave.cpp 0e7e4d450b77f48a42debf47cb6fc12fdde5858b 
>   src/tests/cluster.cpp b43f806eab096b7d4e86e2b5d4b81d6baecb0ee5 
>   src/tests/mock_slave.hpp c057b40d6db433f52d1dfe10ae02a3b0bc936564 
>   src/tests/mock_slave.cpp dd458e3d09e212d6cde514d86989878954f79fc1 
> 
> 
> Diff: https://reviews.apache.org/r/70892/diff/5/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 1. Add `::sleep(5)` to the beginning of the 
> `LinuxSeccompIsolatorProcess::prepare` method.
> 2. Rebuild Mesos and then spin up a local Mesos cluster.
> 3. Run simple task, e.g.:
> ```
> ./src/mesos-execute --master="`hostname`:5050" --name="test" 
> --containerizer=mesos --command="sleep 1"
> ```
> 4. In the parallel terminal session:
> ```
> curl `hostname`:5051/containerizer/debug
> # will return: 
> {"pending":[["linux/seccomp::prepare",{"containerId":""}]]}
> ```
> 5. After 5 seconds this endpoint returns an empty list of pending futures: 
> `{"pending":[]}`
> 
> Unit tests: TBD in the following patches.
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 70892: Added `/containerizer/debug` endpoint.

2019-07-17 Thread Gilbert Song 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70892/#review216684
---


Ship it!




Ship It!

- Gilbert Song


On June 19, 2019, 7:49 a.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70892/
> ---
> 
> (Updated June 19, 2019, 7:49 a.m.)
> 
> 
> Review request for mesos, Gilbert Song, James Peach, and Qian Zhang.
> 
> 
> Bugs: MESOS-9829
> https://issues.apache.org/jira/browse/MESOS-9829
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch introduces an agent's `/containerizer/debug` endpoint,
> which exposes the debug info related to Mesos containerizer.
> Currently, this endpoint returns a list of pending futures related to
> isolators or containerizer launcher. This endpoint is experimental,
> and the format of its output may change over time.
> 
> 
> Diffs
> -
> 
>   docs/authorization.md 91dc03be66bdcf135610bff50e5abcfe8d95c5db 
>   src/common/authorization.cpp 89487219ef87619084663e1a0a17e75924394d38 
>   src/local/local.cpp 6ac6b027f66ef555440ca86803198bfe3227f8d7 
>   src/slave/http.hpp b8c83f1db95c2175bb94f6cd76cc2c58aa118c4e 
>   src/slave/http.cpp 321dca786f106169f88df086422f5ef817d2f098 
>   src/slave/main.cpp ef5ea02f169427d9913b807664bf3073d72cd9b6 
>   src/slave/slave.hpp 58a5608ceb53d7f75a085f1d3a4f757e8c3ddd3b 
>   src/slave/slave.cpp 247797543d9608fb212b5f09865f744052a460e3 
>   src/tests/cluster.cpp b43f806eab096b7d4e86e2b5d4b81d6baecb0ee5 
>   src/tests/mock_slave.hpp c057b40d6db433f52d1dfe10ae02a3b0bc936564 
>   src/tests/mock_slave.cpp dd458e3d09e212d6cde514d86989878954f79fc1 
> 
> 
> Diff: https://reviews.apache.org/r/70892/diff/4/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 1. Add `::sleep(5)` to the beginning of the 
> `LinuxSeccompIsolatorProcess::prepare` method.
> 2. Rebuild Mesos and then spin up a local Mesos cluster.
> 3. Run simple task, e.g.:
> ```
> ./src/mesos-execute --master="`hostname`:5050" --name="test" 
> --containerizer=mesos --command="sleep 1"
> ```
> 4. In the parallel terminal session:
> ```
> curl `hostname`:5051/containerizer/debug
> # will return: 
> {"pending":[["linux/seccomp::prepare",{"containerId":""}]]}
> ```
> 5. After 5 seconds this endpoint returns an empty list of pending futures: 
> `{"pending":[]}`
> 
> Unit tests: TBD in the following patches.
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 70892: Added `/containerizer/debug` endpoint.

2019-07-16 Thread Gilbert Song 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70892/#review216645
---



this patch lgtm! thx!

- Gilbert Song


On June 19, 2019, 7:49 a.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70892/
> ---
> 
> (Updated June 19, 2019, 7:49 a.m.)
> 
> 
> Review request for mesos, Gilbert Song, James Peach, and Qian Zhang.
> 
> 
> Bugs: MESOS-9829
> https://issues.apache.org/jira/browse/MESOS-9829
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch introduces an agent's `/containerizer/debug` endpoint,
> which exposes the debug info related to Mesos containerizer.
> Currently, this endpoint returns a list of pending futures related to
> isolators or containerizer launcher. This endpoint is experimental,
> and the format of its output may change over time.
> 
> 
> Diffs
> -
> 
>   src/local/local.cpp 6ac6b027f66ef555440ca86803198bfe3227f8d7 
>   src/slave/http.hpp b8c83f1db95c2175bb94f6cd76cc2c58aa118c4e 
>   src/slave/http.cpp 69e6d74e8b113cc6c937f47df8984ff9a63e5bb4 
>   src/slave/main.cpp ef5ea02f169427d9913b807664bf3073d72cd9b6 
>   src/slave/slave.hpp 6954f53ff1531b9fcb688ef76acddf6a3d849a41 
>   src/slave/slave.cpp 30039b0857a4d85b4b96fa95d7f8724d57cdec6e 
>   src/tests/cluster.hpp c04ee14beafe350568a295c8258566aa3704028a 
>   src/tests/cluster.cpp b43f806eab096b7d4e86e2b5d4b81d6baecb0ee5 
>   src/tests/mock_slave.hpp c057b40d6db433f52d1dfe10ae02a3b0bc936564 
>   src/tests/mock_slave.cpp dd458e3d09e212d6cde514d86989878954f79fc1 
> 
> 
> Diff: https://reviews.apache.org/r/70892/diff/3/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 1. Add `::sleep(5)` to the beginning of the 
> `LinuxSeccompIsolatorProcess::prepare` method.
> 2. Rebuild Mesos and then spin up a local Mesos cluster.
> 3. Run simple task, e.g.:
> ```
> ./src/mesos-execute --master="`hostname`:5050" --name="test" 
> --containerizer=mesos --command="sleep 1"
> ```
> 4. In the parallel terminal session:
> ```
> curl `hostname`:5051/containerizer/debug
> # will return: 
> {"pending":[["linux/seccomp::prepare",{"containerId":""}]]}
> ```
> 5. After 5 seconds this endpoint returns an empty list of pending futures: 
> `{"pending":[]}`
> 
> Unit tests: TBD in the following patches.
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 70892: Added `/containerizer/debug` endpoint.

2019-07-12 Thread Andrei Budnik 


> On Июль 1, 2019, 8:28 д.п., Qian Zhang wrote:
> > src/slave/http.cpp
> > Lines 2431 (patched)
> > 
> >
> > If this endpoint will only return pending operations, do we really need 
> > the `pending` here? I think we can just return a `JSON::ARRAY` instead of 
> > `JSON::OBJECT` and each element in the array is a pending operations.
> > 
> > Or we may consider to add other info (other than pending operations) 
> > into this endpoint?

We may want to add more debug info related to containerizer. Also, the data is 
self-descriptive, which is better for operators.


> On Июль 1, 2019, 8:28 д.п., Qian Zhang wrote:
> > src/slave/slave.cpp
> > Lines 837 (patched)
> > 
> >
> > Do we plan to return pending operations for other components rather 
> > than just containerizer in future? If yes, then I think we'd better give it 
> > a generic name (like `/debug`) and return component name (currently it is 
> > just `containerizer`) along with operation name and args in the response.

Currently, we do not plan to introduce generic `/debug` endpoint. We don't know 
all the requirements for the generic `/debug` endpoint. In the case of 
`/containerizer/debug` endpoint, the only requirement is that it responses even 
if containerizer hangs.

Also, `/containerizer/debug` endpoint is marked as expiremental and is supposed 
to be used only by operators (DevOps), not by automatic tools.


- Andrei


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70892/#review216264
---


On Июнь 19, 2019, 2:49 п.п., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70892/
> ---
> 
> (Updated Июнь 19, 2019, 2:49 п.п.)
> 
> 
> Review request for mesos, Gilbert Song, James Peach, and Qian Zhang.
> 
> 
> Bugs: MESOS-9829
> https://issues.apache.org/jira/browse/MESOS-9829
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch introduces an agent's `/containerizer/debug` endpoint,
> which exposes the debug info related to Mesos containerizer.
> Currently, this endpoint returns a list of pending futures related to
> isolators or containerizer launcher. This endpoint is experimental,
> and the format of its output may change over time.
> 
> 
> Diffs
> -
> 
>   src/local/local.cpp 6ac6b027f66ef555440ca86803198bfe3227f8d7 
>   src/slave/http.hpp b8c83f1db95c2175bb94f6cd76cc2c58aa118c4e 
>   src/slave/http.cpp 69e6d74e8b113cc6c937f47df8984ff9a63e5bb4 
>   src/slave/main.cpp ef5ea02f169427d9913b807664bf3073d72cd9b6 
>   src/slave/slave.hpp 6954f53ff1531b9fcb688ef76acddf6a3d849a41 
>   src/slave/slave.cpp 30039b0857a4d85b4b96fa95d7f8724d57cdec6e 
>   src/tests/cluster.cpp b43f806eab096b7d4e86e2b5d4b81d6baecb0ee5 
>   src/tests/mock_slave.hpp c057b40d6db433f52d1dfe10ae02a3b0bc936564 
>   src/tests/mock_slave.cpp dd458e3d09e212d6cde514d86989878954f79fc1 
> 
> 
> Diff: https://reviews.apache.org/r/70892/diff/2/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 1. Add `::sleep(5)` to the beginning of the 
> `LinuxSeccompIsolatorProcess::prepare` method.
> 2. Rebuild Mesos and then spin up a local Mesos cluster.
> 3. Run simple task, e.g.:
> ```
> ./src/mesos-execute --master="`hostname`:5050" --name="test" 
> --containerizer=mesos --command="sleep 1"
> ```
> 4. In the parallel terminal session:
> ```
> curl `hostname`:5051/containerizer/debug
> # will return: 
> {"pending":[["linux/seccomp::prepare",{"containerId":""}]]}
> ```
> 5. After 5 seconds this endpoint returns an empty list of pending futures: 
> `{"pending":[]}`
> 
> Unit tests: TBD in the following patches.
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 70892: Added `/containerizer/debug` endpoint.

2019-07-01 Thread Qian Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70892/#review216264
---




src/slave/http.hpp
Lines 84 (patched)


s/containerizer_debug/containerizerDebug/



src/slave/http.hpp
Lines 130 (patched)


s/_containerizer_debug/_containerizerDebug/



src/slave/http.cpp
Lines 2350-2352 (patched)


Can we merge them into a single paragraph?



src/slave/http.cpp
Lines 2428-2429 (patched)


Why do we put an array into an array (`futures`)? I think we should put 
`JSON::Object` into the `futures` array, and in the `JSON::Object` there should 
be two fields: name and args.



src/slave/http.cpp
Lines 2431 (patched)


If this endpoint will only return pending operations, do we really need the 
`pending` here? I think we can just return a `JSON::ARRAY` instead of 
`JSON::OBJECT` and each element in the array is a pending operations.

Or we may consider to add other info (other than pending operations) into 
this endpoint?



src/slave/slave.cpp
Lines 837 (patched)


Do we plan to return pending operations for other components rather than 
just containerizer in future? If yes, then I think we'd better give it a 
generic name (like `/debug`) and return component name (currently it is just 
`containerizer`) along with operation name and args in the response.


- Qian Zhang


On June 19, 2019, 10:49 p.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70892/
> ---
> 
> (Updated June 19, 2019, 10:49 p.m.)
> 
> 
> Review request for mesos, Gilbert Song, James Peach, and Qian Zhang.
> 
> 
> Bugs: MESOS-9829
> https://issues.apache.org/jira/browse/MESOS-9829
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch introduces an agent's `/containerizer/debug` endpoint,
> which exposes the debug info related to Mesos containerizer.
> Currently, this endpoint returns a list of pending futures related to
> isolators or containerizer launcher. This endpoint is experimental,
> and the format of its output may change over time.
> 
> 
> Diffs
> -
> 
>   src/slave/http.hpp b8c83f1db95c2175bb94f6cd76cc2c58aa118c4e 
>   src/slave/http.cpp 69e6d74e8b113cc6c937f47df8984ff9a63e5bb4 
>   src/slave/slave.cpp 30039b0857a4d85b4b96fa95d7f8724d57cdec6e 
> 
> 
> Diff: https://reviews.apache.org/r/70892/diff/1/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 1. Add `::sleep(5)` to the beginning of the 
> `LinuxSeccompIsolatorProcess::prepare` method.
> 2. Rebuild Mesos and then spin up a local Mesos cluster.
> 3. Run simple task, e.g.:
> ```
> ./src/mesos-execute --master="`hostname`:5050" --name="test" 
> --containerizer=mesos --command="sleep 1"
> ```
> 4. In the parallel terminal session:
> ```
> curl `hostname`:5051/containerizer/debug
> # will return: 
> {"pending":[["linux/seccomp::prepare",{"containerId":""}]]}
> ```
> 5. After 5 seconds this endpoint returns an empty list of pending futures: 
> `{"pending":[]}`
> 
> Unit tests: TBD in the following patches.
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Re: Review Request 70892: Added `/containerizer/debug` endpoint.

2019-06-20 Thread Mesos Reviewbot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70892/#review216004
---



Patch looks great!

Reviews applied: [70887, 70888, 70889, 70890, 70891, 70892]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers 
--disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker-build.sh

- Mesos Reviewbot


On June 19, 2019, 2:49 p.m., Andrei Budnik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70892/
> ---
> 
> (Updated June 19, 2019, 2:49 p.m.)
> 
> 
> Review request for mesos, Gilbert Song, James Peach, and Qian Zhang.
> 
> 
> Bugs: MESOS-9829
> https://issues.apache.org/jira/browse/MESOS-9829
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch introduces an agent's `/containerizer/debug` endpoint,
> which exposes the debug info related to Mesos containerizer.
> Currently, this endpoint returns a list of pending futures related to
> isolators or containerizer launcher. This endpoint is experimental,
> and the format of its output may change over time.
> 
> 
> Diffs
> -
> 
>   src/slave/http.hpp b8c83f1db95c2175bb94f6cd76cc2c58aa118c4e 
>   src/slave/http.cpp 69e6d74e8b113cc6c937f47df8984ff9a63e5bb4 
>   src/slave/slave.cpp 30039b0857a4d85b4b96fa95d7f8724d57cdec6e 
> 
> 
> Diff: https://reviews.apache.org/r/70892/diff/1/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 1. Add `::sleep(5)` to the beginning of the 
> `LinuxSeccompIsolatorProcess::prepare` method.
> 2. Rebuild Mesos and then spin up a local Mesos cluster.
> 3. Run simple task, e.g.:
> ```
> ./src/mesos-execute --master="`hostname`:5050" --name="test" 
> --containerizer=mesos --command="sleep 1"
> ```
> 4. In the parallel terminal session:
> ```
> curl `hostname`:5051/containerizer/debug
> # will return: 
> {"pending":[["linux/seccomp::prepare",{"containerId":""}]]}
> ```
> 5. After 5 seconds this endpoint returns an empty list of pending futures: 
> `{"pending":[]}`
> 
> Unit tests: TBD in the following patches.
> 
> 
> Thanks,
> 
> Andrei Budnik
> 
>

Review Request 70892: Added `/containerizer/debug` endpoint.

2019-06-19 Thread Andrei Budnik 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70892/
---

Review request for mesos, Gilbert Song, James Peach, and Qian Zhang.


Bugs: MESOS-9841
https://issues.apache.org/jira/browse/MESOS-9841


Repository: mesos


Description
---

This patch introduces an agent's `/containerizer/debug` endpoint,
which exposes the debug info related to Mesos containerizer.
Currently, this endpoint returns a list of pending futures related to
isolators or containerizer launcher. This endpoint is experimental,
and the format of its output may change over time.


Diffs
-

  src/slave/http.hpp b8c83f1db95c2175bb94f6cd76cc2c58aa118c4e 
  src/slave/http.cpp 69e6d74e8b113cc6c937f47df8984ff9a63e5bb4 
  src/slave/slave.cpp 30039b0857a4d85b4b96fa95d7f8724d57cdec6e 


Diff: https://reviews.apache.org/r/70892/diff/1/


Testing
---

Manual testing.
1. Add `::sleep(5)` to the beginning of the 
`LinuxSeccompIsolatorProcess::prepare` method.
2. Rebuild Mesos and then spin up a local Mesos cluster.
3. Run simple task, e.g.:
```
./src/mesos-execute --master="`hostname`:5050" --name="test" 
--containerizer=mesos --command="sleep 1"
```
4. In the parallel terminal session:
```
curl `hostname`:5051/containerizer/debug
# will return: {"pending":[["linux/seccomp::prepare",{"containerId":""}]]}
```
5. After 5 seconds this endpoint returns an empty list of pending futures: 
`{"pending":[]}`

Unit tests: TBD in the following patches.


Thanks,

Andrei Budnik