Re: Review Request 65812: Added an overloaded version of `os::realpath` to stout for evaluating real path within a scoped root directory

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65812/#review198317
---



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['65811', '65812']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/65812

Relevant logs:

- 
[stout-tests-cmake-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/65812/logs/stout-tests-cmake-stdout.log):

```
 CUSTOMBUILD : CMake warning :  
[D:\DCOS\mesos\3rdparty\zlib-1.2.8.vcxproj]


   "D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj" (default 
target) (1) ->
   "D:\DCOS\mesos\3rdparty\curl-7.57.0.vcxproj" (default target) (6) ->
 CUSTOMBUILD : CMake warning :  
[D:\DCOS\mesos\3rdparty\curl-7.57.0.vcxproj]


   "D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj" (default 
target) (1) ->
   (Link target) -> 
 ip_tests.obj : error LNK2005: "class std::basic_string,class std::allocator > __cdecl path::clean(class 
std::basic_string,class std::allocator 
> const &,char)" 
(?clean@path@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV23@D@Z)
 already defined in flags_tests.obj 
[D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj]
 mac_tests.obj : error LNK2005: "class std::basic_string,class std::allocator > __cdecl path::clean(class 
std::basic_string,class std::allocator 
> const &,char)" 
(?clean@path@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV23@D@Z)
 already defined in flags_tests.obj 
[D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj]
 main.obj : error LNK2005: "class std::basic_string,class std::allocator > __cdecl path::clean(class 
std::basic_string,class std::allocator 
> const &,char)" 
(?clean@path@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV23@D@Z)
 already defined in flags_tests.obj 
[D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj]
 os_tests.obj : error LNK2005: "class std::basic_string,class std::allocator > __cdecl path::clean(class 
std::basic_string,class std::allocator 
> const &,char)" 
(?clean@path@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV23@D@Z)
 already defined in flags_tests.obj 
[D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj]
 path_tests.obj : error LNK2005: "class std::basic_string,class std::allocator > __cdecl path::clean(class 
std::basic_string,class std::allocator 
> const &,char)" 
(?clean@path@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV23@D@Z)
 already defined in flags_tests.obj 
[D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj]
 subcommand_tests.obj : error LNK2005: "class 
std::basic_string,class std::allocator 
> __cdecl path::clean(class std::basic_string,class std::allocator > const &,char)" 
(?clean@path@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV23@D@Z)
 already defined in flags_tests.obj 
[D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj]
 copyfile_tests.obj : error LNK2005: "class 
std::basic_string,class std::allocator 
> __cdecl path::clean(class std::basic_string,class std::allocator > const &,char)" 
(?clean@path@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV23@D@Z)
 already defined in flags_tests.obj 
[D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj]
 env_tests.obj : error LNK2005: "class std::basic_string,class std::allocator > __cdecl path::clean(class 
std::basic_string,class std::allocator 
> const &,char)" 
(?clean@path@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV23@D@Z)
 already defined in flags_tests.obj 
[D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj]
 filesystem_tests.obj : error LNK2005: "class 
std::basic_string,class std::allocator 
> __cdecl path::clean(class std::basic_string,class std::allocator > const &,char)" 
(?clean@path@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV23@D@Z)
 already defined in flags_tests.obj 
[D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj]
 process_tests.obj : error LNK2005: "class 
std::basic_string,class std::allocator 
> __cdecl path::clean(class std::basic_string,class std::allocator > const &,char)" 
(?clean@path@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV23@D@Z)
 already defined in flags_tests.obj 
[D:\DCOS\mesos\3rdparty\stout\tests\stout-tests.vcxproj]
 rmdir_tests.obj : error LNK2005: "class std::basic_string,class std::allocator > __cdecl path::clean(class 
std::basic_string,class std::allocator 
> const &,char)" 
(?clean@path@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV23@D@Z)
 already defined in flag

Review Request 65812: Added an overloaded version of `os::realpath` to stout for evaluating real path within a scoped root directory

[Jason Lai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65812/
---

Review request for mesos, Anish Gupta, Eric Chung, Gilbert Song, Jie Yu, James 
Peach, and Zhitao Li.


Bugs: MESOS-8257
https://issues.apache.org/jira/browse/MESOS-8257


Repository: mesos


Description
---

Added an overloaded version of `os::realpath` to stout for evaluating real path 
within a scoped root directory


Diffs
-

  3rdparty/stout/include/stout/os/posix/realpath.hpp 
31352cefc5b8d0ccd9af8f6dabdec4a959fded32 


Diff: https://reviews.apache.org/r/65812/diff/1/


Testing
---


Thanks,

Jason Lai

Review Request 65811: Add `path::clean` to stout for normalizing path (for POSIX only now)

[Jason Lai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65811/
---

Review request for mesos, Anish Gupta, Eric Chung, Gilbert Song, Jie Yu, James 
Peach, and Zhitao Li.


Bugs: MESOS-8257
https://issues.apache.org/jira/browse/MESOS-8257


Repository: mesos


Description
---

Add `path::clean` to stout for normalizing path (for POSIX only now)


Diffs
-

  3rdparty/stout/include/stout/path.hpp 
27438d31617b3b78bf3d4deffd25c93340610e8d 


Diff: https://reviews.apache.org/r/65811/diff/1/


Testing
---


Thanks,

Jason Lai

Re: Review Request 65661: Updated comment and variables to decouple quota limit and guarantee.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65661/#review198309
---



Patch looks great!

Reviews applied: [65661]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On Feb. 15, 2018, 10:42 p.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65661/
> ---
> 
> (Updated Feb. 15, 2018, 10:42 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler, Kapil Arya, Joseph Wu, Michael 
> Park, and Till Toenshoff.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> We currently do not differentiate between quota guarantee and limit
> i.e. quota currently is set for both guarantee and limit. As a
> first step to introduce quota limit, comments and related variable
> names are updated to reflect the difference between quota guarantee
> and limit.
> 
> Also fixed a few typos and removed some outdated comments.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.hpp 
> 37d2df58d374f118697fb5848c36f78b3e65128d 
>   src/master/allocator/mesos/hierarchical.cpp 
> 80e0a347d6641c922cf7c62bef220ff5c8864597 
> 
> 
> Diff: https://reviews.apache.org/r/65661/diff/2/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Re: Review Request 65807: Added an allocator test for quota with ancestor reservation.

[Benjamin Mahler]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65807/#review198308
---


Fix it, then Ship it!





src/tests/hierarchical_allocator_tests.cpp
Lines 5592-5593 (patched)


maybe parens around (total reservations - allocated reservations) or just 
showing the unallocated reservations would be clearer?

```
  // Required headroom: quota(cpus:1;mem:1024)
  //
  // Available headroom = total resources (cpus:3;mem:3072) -
  //  allocated resources (cpus:1;mem:1024) -
  //  unallocated reservations (cpus:0;mem:0)
  //= (cpus:2;mem:2048)
```

Note some ':' characters were missing here as well.



src/tests/hierarchical_allocator_tests.cpp
Lines 5597 (patched)


This would produce a negative? I think you mean available - required?



src/tests/hierarchical_allocator_tests.cpp
Lines 5608-5614 (patched)


s/.get()./->/ here



src/tests/hierarchical_allocator_tests.cpp
Lines 5613-5614 (patched)


```
EXPECT_EQ(agentResources,
  Resources::sum(allocation->resources.at(NO_QUOTA_ROLE)));
```

Or:

```
EXPECT_EQ(
agentResources,
Resources::sum(allocation->resources.at(NO_QUOTA_ROLE)));
```


- Benjamin Mahler


On Feb. 27, 2018, 12:45 a.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65807/
> ---
> 
> (Updated Feb. 27, 2018, 12:45 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Michael Park.
> 
> 
> Bugs: MESOS-8604
> https://issues.apache.org/jira/browse/MESOS-8604
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test ensures that resources can be correctly allocated
> in the presence of quota when ancestor reservations are
> allocated to child. Specifically, it checks that ancestor
> reservations allocated to child are counted by the quota
> headroom calculation. See MESOS-8604.
> 
> 
> Diffs
> -
> 
>   src/tests/hierarchical_allocator_tests.cpp 
> 42dc6ac5e281bb0cdb938106fd3f12324e9e6330 
> 
> 
> Diff: https://reviews.apache.org/r/65807/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Re: Review Request 65806: Fixed a bug where quota headroom is miscalculated.

[Benjamin Mahler]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65806/#review198307
---


Ship it!





src/master/allocator/mesos/hierarchical.cpp
Lines 1677-1678 (original), 1677-1679 (patched)


Do we need the new variable? Looks like we just need to do the following in 
the equation below:

```
availableHeadroom -=
  Resources::sum(reservationScalarQuantities) -
  totalAllocatedReservationScalarQuantities;
```



src/master/allocator/mesos/hierarchical.cpp
Lines 1681-1688 (patched)


How about:

  // Calculate total allocated reservations. Note that we need to ensure
  // we count a reservation for "a" being allocated to "a/b", therefore
  // we cannot simply loop over the reservations' roles.



src/master/allocator/mesos/hierarchical.cpp
Lines 1697-1698 (patched)


We could inline this to make it a little more succinct:

```
  continue; // This role has no allocation.
```


- Benjamin Mahler


On Feb. 27, 2018, 2:46 a.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65806/
> ---
> 
> (Updated Feb. 27, 2018, 2:46 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Michael Park.
> 
> 
> Bugs: MESOS-8604
> https://issues.apache.org/jira/browse/MESOS-8604
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> When calculating the quota headroom, we failed to consider
> ancestor's reservation allocated to the child. This leads
> to under-estimation of available headroom and excessive
> resources being set aside for headroom. See MESOS-8604.
> 
> This patches fixes this issue by counting ancestor's reservation
> allocated to the child as allocated-reservation even though
> the child itself has no reservation.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.cpp 
> 80e0a347d6641c922cf7c62bef220ff5c8864597 
> 
> 
> Diff: https://reviews.apache.org/r/65806/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> Dedicated test in #65807
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Re: Review Request 65806: Fixed a bug where quota headroom is miscalculated.

[Meng Zhu]

> On Feb. 26, 2018, 6:41 p.m., Benjamin Mahler wrote:
> > Does this need to depend on the previous renaming patch? Makes 
> > cherry-picking a little messier

Nope. Removed the dependency.


- Meng


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65806/#review198304
---


On Feb. 26, 2018, 6:46 p.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65806/
> ---
> 
> (Updated Feb. 26, 2018, 6:46 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Michael Park.
> 
> 
> Bugs: MESOS-8604
> https://issues.apache.org/jira/browse/MESOS-8604
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> When calculating the quota headroom, we failed to consider
> ancestor's reservation allocated to the child. This leads
> to under-estimation of available headroom and excessive
> resources being set aside for headroom. See MESOS-8604.
> 
> This patches fixes this issue by counting ancestor's reservation
> allocated to the child as allocated-reservation even though
> the child itself has no reservation.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.cpp 
> 80e0a347d6641c922cf7c62bef220ff5c8864597 
> 
> 
> Diff: https://reviews.apache.org/r/65806/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> Dedicated test in #65807
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Re: Review Request 65806: Fixed a bug where quota headroom is miscalculated.

[Meng Zhu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65806/
---

(Updated Feb. 26, 2018, 6:46 p.m.)


Review request for mesos, Benjamin Mahler and Michael Park.


Changes
---

Removed the unnecessary patch dependency.


Bugs: MESOS-8604
https://issues.apache.org/jira/browse/MESOS-8604


Repository: mesos


Description
---

When calculating the quota headroom, we failed to consider
ancestor's reservation allocated to the child. This leads
to under-estimation of available headroom and excessive
resources being set aside for headroom. See MESOS-8604.

This patches fixes this issue by counting ancestor's reservation
allocated to the child as allocated-reservation even though
the child itself has no reservation.


Diffs
-

  src/master/allocator/mesos/hierarchical.cpp 
80e0a347d6641c922cf7c62bef220ff5c8864597 


Diff: https://reviews.apache.org/r/65806/diff/1/


Testing
---

make check
Dedicated test in #65807


Thanks,

Meng Zhu

Re: Review Request 65806: Fixed a bug where quota headroom is miscalculated.

[Benjamin Mahler]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65806/#review198304
---



Does this need to depend on the previous renaming patch? Makes cherry-picking a 
little messier

- Benjamin Mahler


On Feb. 27, 2018, 12:45 a.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65806/
> ---
> 
> (Updated Feb. 27, 2018, 12:45 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Michael Park.
> 
> 
> Bugs: MESOS-8604
> https://issues.apache.org/jira/browse/MESOS-8604
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> When calculating the quota headroom, we failed to consider
> ancestor's reservation allocated to the child. This leads
> to under-estimation of available headroom and excessive
> resources being set aside for headroom. See MESOS-8604.
> 
> This patches fixes this issue by counting ancestor's reservation
> allocated to the child as allocated-reservation even though
> the child itself has no reservation.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.cpp 
> 80e0a347d6641c922cf7c62bef220ff5c8864597 
> 
> 
> Diff: https://reviews.apache.org/r/65806/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> Dedicated test in #65807
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Re: Review Request 65807: Added an allocator test for quota with ancestor reservation.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65807/#review198303
---



PASS: Mesos patch 65807 was successfully built and tested.

Reviews applied: `['65661', '65806', '65807']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/65807

- Mesos Reviewbot Windows


On Feb. 27, 2018, 12:45 a.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65807/
> ---
> 
> (Updated Feb. 27, 2018, 12:45 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Michael Park.
> 
> 
> Bugs: MESOS-8604
> https://issues.apache.org/jira/browse/MESOS-8604
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test ensures that resources can be correctly allocated
> in the presence of quota when ancestor reservations are
> allocated to child. Specifically, it checks that ancestor
> reservations allocated to child are counted by the quota
> headroom calculation. See MESOS-8604.
> 
> 
> Diffs
> -
> 
>   src/tests/hierarchical_allocator_tests.cpp 
> 42dc6ac5e281bb0cdb938106fd3f12324e9e6330 
> 
> 
> Diff: https://reviews.apache.org/r/65807/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Re: Review Request 65791: Removed stale generated HTTP endpoint documentation.

[Benjamin Mahler]

> On Feb. 24, 2018, 11:08 a.m., Benjamin Bannier wrote:
> > I think a sticky issue which would have been great to resolve here is to 
> > make sure that we enable building of endpoint documentation generation for 
> > non-dockerized site builds as well (i.e., `cd site/ && bundle install && 
> > bundle exec rake` should produce output similar to what the dockerized 
> > version running in CI produces; this probably requires adding a dedicated 
> > rake targe). Granted, this was already slightly broken, and we now make 
> > that more apparent.

Yeah sounds great for a follow up (perhaps we need instructions on how to 
generate the help endpoints when using that method in the short term rather 
than automating it?). We had discussed it but treated it as an orthogonal issue 
that was already broken (as you said, might as well make it clear that the 
docs/ folder has 2 year old stale http endpoint docs).


- Benjamin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65791/#review198252
---


On Feb. 24, 2018, 12:29 a.m., Benjamin Mahler wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65791/
> ---
> 
> (Updated Feb. 24, 2018, 12:29 a.m.)
> 
> 
> Review request for mesos, Benjamin Bannier and Vinod Kone.
> 
> 
> Bugs: MESOS-8442
> https://issues.apache.org/jira/browse/MESOS-8442
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This is now generated by the website CI bot, rather than having
> to be checked in and kept up to date within the docs/ folder.
> 
> 
> Diffs
> -
> 
>   docs/endpoints/files/browse.json.md 
> 6b21e50bb0c0098babedcd612a5acfd0e6ae0aab 
>   docs/endpoints/files/browse.md 0fa204a85fc62aa4b43dacc07aec6951b618a015 
>   docs/endpoints/files/debug.json.md 2806a5844f49a3c051e58db0723879c9627e18e1 
>   docs/endpoints/files/debug.md 33380663eab59c704c610bd392676aee99528f77 
>   docs/endpoints/files/download.json.md 
> 31d442a7e1e3d4c6e32986ec85e42f3364f37de9 
>   docs/endpoints/files/download.md 202d015484b22e63c9d24610808a43d091786844 
>   docs/endpoints/files/read.json.md 64f348e76d61d3d0ad364f4712d3c1d90818d6d1 
>   docs/endpoints/files/read.md ef95c80739eeb09f52658470429c6ffb9b41fb8c 
>   docs/endpoints/index.md 871153774c4b21cdca4afec9bc2ce2bd955b17c4 
>   docs/endpoints/logging/toggle.md 014a1969a7efb709e06c0c90ca023f2fa747414e 
>   docs/endpoints/master/api/v1.md 2d778f33c830caccb0254cf15adc46a86b5db48a 
>   docs/endpoints/master/api/v1/scheduler.md 
> f029d8e6e90bc7d974d60ad4a71d63807f0e901a 
>   docs/endpoints/master/create-volumes.md 
> c9ce39ead8041f249ee6f85d1afdd8f90b38ce9b 
>   docs/endpoints/master/destroy-volumes.md 
> ff82e96a6ee1e9f6e1bbc565ab40c98b758c1227 
>   docs/endpoints/master/flags.md 6b3de41d45278cc34a4f4f0e52b76407ab96b790 
>   docs/endpoints/master/frameworks.md 
> 7aa22130dfa8702d8b93690cb75f1323ea3c2223 
>   docs/endpoints/master/health.md 3752a1bb4a9a7c2cc5cbf25b021592280c174a80 
>   docs/endpoints/master/machine/down.md 
> 05abd155e693163eb50f675d2724534a6f87dfba 
>   docs/endpoints/master/machine/up.md 
> 8a3488bb3f96bc7470aaea5114559cb593cbb3a8 
>   docs/endpoints/master/maintenance/schedule.md 
> 86ae30b06118f13593e91b70dbec97945b1d9bd8 
>   docs/endpoints/master/maintenance/status.md 
> f8e1c7eed44e14a8324b582a2db91e523b2fbc15 
>   docs/endpoints/master/quota.md 6ce7811136ac1c4ae00a3011dca17cbde948f14e 
>   docs/endpoints/master/redirect.md 3c568e3eef658f9a489008f4f84745827f8002f0 
>   docs/endpoints/master/reserve.md 8174027f0e66dc4e9ae7bca951ed9ca38033008d 
>   docs/endpoints/master/roles.json.md 
> 9acba2cfed120918b2571d7336651d125cbe 
>   docs/endpoints/master/roles.md de58120172c390ef1f084031077fd9b865b817db 
>   docs/endpoints/master/slaves.md 9365cdb9e5b17cdf26ed38a0ab1fff8afef38f00 
>   docs/endpoints/master/state-summary.md 
> 21b8703bb011daf5cc560550e6c2ce36ba615bba 
>   docs/endpoints/master/state.json.md 
> 62a723051270ab24591b956805aee5455848201b 
>   docs/endpoints/master/state.md 09a478263d4cd015040f47619b0aa94341c3bba2 
>   docs/endpoints/master/tasks.json.md 
> f9553d7bde7de33b02795ba9088a88e68cb2a0cc 
>   docs/endpoints/master/tasks.md 019a14bb6c8d8acd3aab4a1f7ba6893f4bad835b 
>   docs/endpoints/master/teardown.md b017084e3d195363615a59011671fdbd76182a1a 
>   docs/endpoints/master/unreserve.md 324dcded2013aeb6c167a695bbad1da69340cd57 
>   docs/endpoints/master/weights.md 3c9019e74867cbd931dc5c68a0d8de3ea269e5b2 
>   docs/endpoints/metrics/snapshot.md 0912fa939b38fd1d8335e165c7aeddf1f076 
>   docs/endpoints/profiler/start.md a25f0d0468f2ad9f8d1ff41f6c50ea0bbe87d316 
>   docs/endpoints/profiler/stop.md 5028d13f8a3f9aa3d41b60959a9cc3d6b03fb7d9 
>   docs/endpoints/registrar/registry.md 
> e347db269

Re: Review Request 65807: Added an allocator test for quota with ancestor reservation.

[Mesos Reviewbot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65807/#review198301
---



Patch looks great!

Reviews applied: [65661, 65806, 65807]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 
MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On Feb. 27, 2018, 12:45 a.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65807/
> ---
> 
> (Updated Feb. 27, 2018, 12:45 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Michael Park.
> 
> 
> Bugs: MESOS-8604
> https://issues.apache.org/jira/browse/MESOS-8604
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test ensures that resources can be correctly allocated
> in the presence of quota when ancestor reservations are
> allocated to child. Specifically, it checks that ancestor
> reservations allocated to child are counted by the quota
> headroom calculation. See MESOS-8604.
> 
> 
> Diffs
> -
> 
>   src/tests/hierarchical_allocator_tests.cpp 
> 42dc6ac5e281bb0cdb938106fd3f12324e9e6330 
> 
> 
> Diff: https://reviews.apache.org/r/65807/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Re: Review Request 65661: Updated comment and variables to decouple quota limit and guarantee.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65661/#review198300
---



PASS: Mesos patch 65661 was successfully built and tested.

Reviews applied: `['65661']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/65661

- Mesos Reviewbot Windows


On Feb. 15, 2018, 10:42 p.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65661/
> ---
> 
> (Updated Feb. 15, 2018, 10:42 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler, Kapil Arya, Joseph Wu, Michael 
> Park, and Till Toenshoff.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> We currently do not differentiate between quota guarantee and limit
> i.e. quota currently is set for both guarantee and limit. As a
> first step to introduce quota limit, comments and related variable
> names are updated to reflect the difference between quota guarantee
> and limit.
> 
> Also fixed a few typos and removed some outdated comments.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.hpp 
> 37d2df58d374f118697fb5848c36f78b3e65128d 
>   src/master/allocator/mesos/hierarchical.cpp 
> 80e0a347d6641c922cf7c62bef220ff5c8864597 
> 
> 
> Diff: https://reviews.apache.org/r/65661/diff/2/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Review Request 65807: Added an allocator test for quota with ancestor reservation.

[Meng Zhu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65807/
---

Review request for mesos, Benjamin Mahler and Michael Park.


Bugs: MESOS-8604
https://issues.apache.org/jira/browse/MESOS-8604


Repository: mesos


Description
---

This test ensures that resources can be correctly allocated
in the presence of quota when ancestor reservations are
allocated to child. Specifically, it checks that ancestor
reservations allocated to child are counted by the quota
headroom calculation. See MESOS-8604.


Diffs
-

  src/tests/hierarchical_allocator_tests.cpp 
42dc6ac5e281bb0cdb938106fd3f12324e9e6330 


Diff: https://reviews.apache.org/r/65807/diff/1/


Testing
---


Thanks,

Meng Zhu

Review Request 65806: Fixed a bug where quota headroom is miscalculated.

[Meng Zhu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65806/
---

Review request for mesos, Benjamin Mahler and Michael Park.


Bugs: MESOS-8604
https://issues.apache.org/jira/browse/MESOS-8604


Repository: mesos


Description
---

When calculating the quota headroom, we failed to consider
ancestor's reservation allocated to the child. This leads
to under-estimation of available headroom and excessive
resources being set aside for headroom. See MESOS-8604.

This patches fixes this issue by counting ancestor's reservation
allocated to the child as allocated-reservation even though
the child itself has no reservation.


Diffs
-

  src/master/allocator/mesos/hierarchical.cpp 
80e0a347d6641c922cf7c62bef220ff5c8864597 


Diff: https://reviews.apache.org/r/65806/diff/1/


Testing
---


Thanks,

Meng Zhu

Re: Review Request 59987: Added protobuf map support.

[Chun-Hung Hsiao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59987/#review198292
---


Ship it!




Ship It!

- Chun-Hung Hsiao


On Feb. 26, 2018, 9:05 a.m., Qian Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59987/
> ---
> 
> (Updated Feb. 26, 2018, 9:05 a.m.)
> 
> 
> Review request for mesos, Anand Mazumdar, Chun-Hung Hsiao, and Zhitao Li.
> 
> 
> Bugs: MESOS-7656
> https://issues.apache.org/jira/browse/MESOS-7656
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Map is a feature of proto2 syntax, but it can only be compiled
> with proto3 compiler, see the following discussion for details:
> https://groups.google.com/forum/#!topic/protobuf/p4WxcplrlA4
> We have already upgraded the compiler from 2.6.1 to 3.3.0 in
> MESOS-7228, however, to use protobuf map in Mesos code, we also
> need to add the protobuf map support to the code in Mesos for
> converting protobuf message to JSON object and parsing JSON
> object as protobuf message, that is what I have done in this patch.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/protobuf.hpp 
> 4a1605e5130dbf7e8286dbb43d0d04ab4394e79a 
> 
> 
> Diff: https://reviews.apache.org/r/59987/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Qian Zhang
> 
>

Re: Review Request 52064: Support for multiple versions of docs.

[Benjamin Mahler]

> On Feb. 24, 2018, 12:50 a.m., Benjamin Mahler wrote:
> > Vinod and I went over this, a couple of issues:
> > 
> > * The [endpoint 
> > documentation](http://mesos.apache.org/documentation/latest/endpoints/) is 
> > only being generated for latest (see 
> > https://github.com/apache/mesos/blob/master/support/mesos-website/build.sh),
> >  this needs to be generated for each version.
> > * The C++ and Java API documentation is only being generated for latest 
> > (not sure how easy this is to fix, maybe if you also tackle this do so in a 
> > seperate patch). Also note that home.md linked directly to 
> > "/api/latest/c++" and "/api/latest/java" which means that the old versions 
> > would send users to latest c++ and java docs.
> > * We had a hard time figuring why some of the code changes were needed or 
> > what they were doing (e.g. some of the new regexes, or why the version 
> > selector drop-down has an empty value for the first entry), some comments 
> > would be helpful for the reader.
> > 
> > We were thinking the following approach would be good alternative:
> > 
> > * In order to ensure that we generate the HTTP help endpoint documentation 
> > correctly for each version, the 
> > [build.sh](https://github.com/apache/mesos/blob/master/support/mesos-website/build.sh)
> >  script would loop over each 1.x.y tag (excluding -rc#'s) and checkout the 
> > tag into a sub-directory. Within this sub-directory, we would build mesos 
> > and generate the help endpoints documentation. This sub-directory would 
> > probably be named directly as the version (e.g. "1.0.0", "1.4.1", etc).
> > * Note that for efficiency reasons (it's very expensive to build mesos for 
> > so many tags), the build.sh script would skip a tag if it finds that the 
> > publish directory already exists for the version in the mesos-site 
> > repository.
> > * The Rakefile would be updated to look at each version checkout 
> > subdirectory, and would generate the docs using the documentation folder 
> > within that specific version subdirectory.
> > * Not sure if the releases YAML still needs the docs tag, it seems so since 
> > it's used in the javascript. This would at least need to be updated so that 
> > everything before 1.0.0 is "false".
> > * Is it possible to put the version selector on the right side of the blue 
> > banner?
> 
> Tim Anderegg wrote:
> Sounds good.  I had initially avoided the C++ and Java API documentation 
> since it would take a very long time to generate, as you mention, but I'm 
> happy to tackle that as well. Limiting to 1.0.0+ will alleviate that a bit.
> 
> For the regex, the first change was just a simplification since the two 
> lines were redundant once I updated the code to use "{{ version }}" instead 
> of "latest" (I will verify that however, in case I am remembering 
> incorrectly). The second change was to account for older versions of the 
> Mesos code where "latest" was hardcoded into the HTML templates themselves, 
> which would mess up the links.  That one is already commented, but I will go 
> through and add comments wherever there may be ambiguity.  Since we are now 
> doing only 1.0.0+, this work around can probably be removed.
> 
> For the version selector, happy to move it and remove the empty option, 
> since the value should always be the currently displayed version of the 
> documentation.  I'm also not sure why I did that, it was a while ago...
> 
> For the docs tag, we could either keep it or if we want the logic to just 
> be "all versions from 1.0.0 onwards", that can be hard coded into the 
> build.sh and the HTML template, and we could remove the docs tag.  Let me 
> know what you prefer.
> 
> For the updated approach, just to make sure I'm clear:
> 
> 1) In build.sh, iterate over all tags and checkout the code into a folder 
> with the tag name, (perhaps in $WORKSPACE/mesos/_build/x.y.z/) if the primary 
> $WORKSPACE/mesos/site/publish/documentation/x.y.z folder doesn't already 
> exist.
> 2) Then the Rakefile (from $WORKSPACE/mesos, i.e. the HEAD version) is 
> used to build the docs for each version folder into the primary 
> $WORKSPACE/mesos/site/publish/documentation/x.y.z folder, as before, but only 
> if it doesn't already exist.
> 
> A few questions:
> 
> 1) I'm assuming that this build occurs in a Jenkins job that retains its 
> workspace between runs, which is why we are checking whether or not the 
> folder exists?  Or is the mesos-website publish folder the only thing that 
> persists between runs?  Just want to make sure my conception of the build 
> environment is correct, if the later is true question 2 can be ignored.
> 2) Should the script remove the checked-out code after the build runs for 
> each version that needed it?  Otherwise over time many copies of the Mesos 
> code would persist (one for each tagged version), which would be GB's in size.
> 
> Thanks!  I should be able to fi

Re: Review Request 65804: Fixed a typo in the NsTest.SupportedNamespaces test.

[Mesos Reviewbot Windows]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65804/#review198285
---



PASS: Mesos patch 65804 was successfully built and tested.

Reviews applied: `['65804']`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/65804

- Mesos Reviewbot Windows


On Feb. 26, 2018, 4:03 p.m., James Peach wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65804/
> ---
> 
> (Updated Feb. 26, 2018, 4:03 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov and Jie Yu.
> 
> 
> Bugs: MESOS-8610
> https://issues.apache.org/jira/browse/MESOS-8610
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test tries to verify consistency across the APIs in the `ns`
> namespace. We have a special case for user namespaces (that may be
> available but not supported due to the kernel version), but we were
> testing `CLONE_NEWNS` rather than `CLONE_NEWUSER`.
> 
> 
> Diffs
> -
> 
>   src/tests/containerizer/ns_tests.cpp 
> 904b9338b293e9bc736ed36fe59a1b72c8b025e2 
> 
> 
> Diff: https://reviews.apache.org/r/65804/diff/1/
> 
> 
> Testing
> ---
> 
> make check (Fedora 27)
> 
> 
> Thanks,
> 
> James Peach
> 
>

Re: Review Request 65804: Fixed a typo in the NsTest.SupportedNamespaces test.

[Alexander Rukletsov]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65804/#review198283
---


Ship it!




Ship It!

- Alexander Rukletsov


On Feb. 26, 2018, 4:03 p.m., James Peach wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65804/
> ---
> 
> (Updated Feb. 26, 2018, 4:03 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov and Jie Yu.
> 
> 
> Bugs: MESOS-8610
> https://issues.apache.org/jira/browse/MESOS-8610
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test tries to verify consistency across the APIs in the `ns`
> namespace. We have a special case for user namespaces (that may be
> available but not supported due to the kernel version), but we were
> testing `CLONE_NEWNS` rather than `CLONE_NEWUSER`.
> 
> 
> Diffs
> -
> 
>   src/tests/containerizer/ns_tests.cpp 
> 904b9338b293e9bc736ed36fe59a1b72c8b025e2 
> 
> 
> Diff: https://reviews.apache.org/r/65804/diff/1/
> 
> 
> Testing
> ---
> 
> make check (Fedora 27)
> 
> 
> Thanks,
> 
> James Peach
> 
>

Review Request 65804: Fixed a typo in the NsTest.SupportedNamespaces test.

[James Peach]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65804/
---

Review request for mesos, Alexander Rukletsov and Jie Yu.


Bugs: MESOS-8610
https://issues.apache.org/jira/browse/MESOS-8610


Repository: mesos


Description
---

This test tries to verify consistency across the APIs in the `ns`
namespace. We have a special case for user namespaces (that may be
available but not supported due to the kernel version), but we were
testing `CLONE_NEWNS` rather than `CLONE_NEWUSER`.


Diffs
-

  src/tests/containerizer/ns_tests.cpp 904b9338b293e9bc736ed36fe59a1b72c8b025e2 


Diff: https://reviews.apache.org/r/65804/diff/1/


Testing
---

make check (Fedora 27)


Thanks,

James Peach

Re: Review Request 59987: Added protobuf map support.

[Qian Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59987/
---

(Updated Feb. 26, 2018, 5:05 p.m.)


Review request for mesos, Anand Mazumdar, Chun-Hung Hsiao, and Zhitao Li.


Changes
---

Addressed review comments.


Bugs: MESOS-7656
https://issues.apache.org/jira/browse/MESOS-7656


Repository: mesos


Description
---

Map is a feature of proto2 syntax, but it can only be compiled
with proto3 compiler, see the following discussion for details:
https://groups.google.com/forum/#!topic/protobuf/p4WxcplrlA4
We have already upgraded the compiler from 2.6.1 to 3.3.0 in
MESOS-7228, however, to use protobuf map in Mesos code, we also
need to add the protobuf map support to the code in Mesos for
converting protobuf message to JSON object and parsing JSON
object as protobuf message, that is what I have done in this patch.


Diffs (updated)
-

  3rdparty/stout/include/stout/protobuf.hpp 
4a1605e5130dbf7e8286dbb43d0d04ab4394e79a 


Diff: https://reviews.apache.org/r/59987/diff/5/

Changes: https://reviews.apache.org/r/59987/diff/4-5/


Testing
---


Thanks,

Qian Zhang

Re: Review Request 59987: Added protobuf map support.

[Qian Zhang]

> On Feb. 9, 2018, 4:26 a.m., Chun-Hung Hsiao wrote:
> > 3rdparty/stout/include/stout/protobuf.hpp
> > Lines 499-500 (patched)
> > 
> >
> > How about using `UNREACHABLE` instead?

With `ABORT`, we can output `key_field->type()` which will be helpful for 
debugging purpose, but we cannot do it with `UNREACHABLE`. So I would still 
prefer `ABORT`.


> On Feb. 9, 2018, 4:26 a.m., Chun-Hung Hsiao wrote:
> > 3rdparty/stout/include/stout/protobuf.hpp
> > Lines 502-503 (patched)
> > 
> >
> > Deprecated or not, it cannot be the type of a key, so let's merge this 
> > case with the above cases.

Agree!


> On Feb. 9, 2018, 4:26 a.m., Chun-Hung Hsiao wrote:
> > 3rdparty/stout/include/stout/protobuf.hpp
> > Lines 1045-1046 (patched)
> > 
> >
> > Ditto.

With `ABORT`, we can output `key_field->type()` which will be helpful for 
debugging purpose, but we cannot do it with `UNREACHABLE`. So I would still 
prefer `ABORT`.


> On Feb. 9, 2018, 4:26 a.m., Chun-Hung Hsiao wrote:
> > 3rdparty/stout/include/stout/protobuf.hpp
> > Lines 1113-1114 (patched)
> > 
> >
> > Since stout is positioned as a standalone library, we should return an 
> > error indicating that this is not supported instead of arbitrarily 
> > terminate the program.

I think the reason we do not return error is this code should never be called, 
otherwise that means something really bad happens which is not a normal error 
that we should return. Actually in stout, `ABORT` and `UNREACHABLE` are widely 
used when an unrecoverable issue occurs.


- Qian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59987/#review197100
---


On Dec. 21, 2017, 10:07 a.m., Qian Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59987/
> ---
> 
> (Updated Dec. 21, 2017, 10:07 a.m.)
> 
> 
> Review request for mesos, Anand Mazumdar, Chun-Hung Hsiao, and Zhitao Li.
> 
> 
> Bugs: MESOS-7656
> https://issues.apache.org/jira/browse/MESOS-7656
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Map is a feature of proto2 syntax, but it can only be compiled
> with proto3 compiler, see the following discussion for details:
> https://groups.google.com/forum/#!topic/protobuf/p4WxcplrlA4
> We have already upgraded the compiler from 2.6.1 to 3.3.0 in
> MESOS-7228, however, to use protobuf map in Mesos code, we also
> need to add the protobuf map support to the code in Mesos for
> converting protobuf message to JSON object and parsing JSON
> object as protobuf message, that is what I have done in this patch.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/protobuf.hpp 
> baad12648dd78ab72ea4277f4c7f99da16696a40 
> 
> 
> Diff: https://reviews.apache.org/r/59987/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Qian Zhang
> 
>