Review Request 41829: WIP: Added test cases for allocator.

2015-12-31 Thread Guangya Liu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41829/
---

Review request for mesos and Klaus Ma.


Repository: mesos


Description
---

WIP: Added test cases for allocator.


Diffs
-

  src/tests/hierarchical_allocator_tests.cpp 
e044f832c2c16e53e663c6ced5452649bb0dcb59 

Diff: https://reviews.apache.org/r/41829/diff/


Testing
---


Thanks,

Guangya Liu

Re: Review Request 40632: WIP: Enabled oversubscribed resources for reservations in allocator.

2015-12-31 Thread Guangya Liu 


> On 十二月 30, 2015, 12:42 p.m., Klaus Ma wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1321
> > 
> >
> > As framework can not use ALLOCATION_SLACK mixed with other resources, 
> > allocator need to check seperatly.

Yes, currently the validation logic in master is 
https://github.com/apache/mesos/blob/master/src/master/validation.cpp#L479-L488 
which can make sure a task and its executor can either use non-revocable 
resources or revocable resources of a given name but not both. After we 
introduced the allocation slack, we may want to guard that a task and its 
executor can either use allocation slack or usage slack or non-revocable 
resources but not all. I will upload a patch to handle this part.


- Guangya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40632/#review112309
---


On 十二月 30, 2015, 7:04 a.m., Guangya Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40632/
> ---
> 
> (Updated 十二月 30, 2015, 7:04 a.m.)
> 
> 
> Review request for mesos, Klaus Ma and Jian Qiu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Enabled oversubscribed resources for reservations in allocator.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.cpp 
> 7f900c4e024485704d79e57ae22407557598fe6c 
> 
> Diff: https://reviews.apache.org/r/40632/diff/
> 
> 
> Testing
> ---
> 
> make
> make check
> 
> 
> Thanks,
> 
> Guangya Liu
> 
>

Re: Review Request 41714: Added tests for HDFS URI fetcher plugin.

2015-12-31 Thread Timothy Chen 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41714/#review112378
---

Ship it!


Ship It!

- Timothy Chen


On Dec. 31, 2015, 5:36 a.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41714/
> ---
> 
> (Updated Dec. 31, 2015, 5:36 a.m.)
> 
> 
> Review request for mesos, Adam B, Ben Mahler, Ian Downes, Timothy Chen, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-3925
> https://issues.apache.org/jira/browse/MESOS-3925
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added tests for HDFS URI fetcher plugin.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am bf71fe6ee65233bbc47a41f610d68ff9760e19f6 
>   src/tests/uri_fetcher_tests.cpp d54a7e7b08a741b1d5d371c11747681ae4ddef98 
>   src/uri/schemes/hdfs.hpp PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/41714/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 41681: Introduce HTTP endpoint /weights for updating weight.

2015-12-31 Thread Yongqiao Wang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41681/
---

(Updated Dec. 31, 2015, 11:54 a.m.)


Review request for mesos, Adam B, Neil Conway, and Qian Zhang.


Changes
---

Addressed the comments of Jian.


Bugs: MESOS-4214
https://issues.apache.org/jira/browse/MESOS-4214


Repository: mesos


Description
---

Introduce HTTP endpoint /weights for updating weight.


Diffs (updated)
-

  include/mesos/authorizer/authorizer.hpp 
19f6e1a2d025bf6ff07f515b10d41e8a48d7d0b4 
  include/mesos/authorizer/authorizer.proto 
7b729e19484d92be48bbde4dff2c833a4109936e 
  src/CMakeLists.txt 8169fe4f1a0eda5c2e2b30e22a7346478420a9a0 
  src/Makefile.am bf71fe6ee65233bbc47a41f610d68ff9760e19f6 
  src/authorizer/local/authorizer.hpp 1563c11709c2612350354690b50ceb33d2720f98 
  src/authorizer/local/authorizer.cpp 1d135fb6906c7050a788cbac9ca2d8164ff064ef 
  src/master/http.cpp deb0c8f9852dc0eec1e8c0ff35c684f982e8b110 
  src/master/master.hpp 1cc5531de70bdc0bdce9835c4930dc774406fac9 
  src/master/master.cpp 40ce3e17fca88da689128bcf5d35fdddc396c011 
  src/master/registry.proto 9958f9c2bdb785390fca2f292b65d5a9310434d5 
  src/master/weights_handler.cpp PRE-CREATION 
  src/tests/mesos.hpp a4811b0d8dee33ff2ca4968f532ce64b7ea95249 
  src/tests/mesos.cpp f4b0f82449c4b6a2b4b7b7f14518714485d5a13a 

Diff: https://reviews.apache.org/r/41681/diff/


Testing
---

Make & Make check successfully!

$ (./mesos-master.sh --ip=127.0.0.1 --work_dir=/Users/yqwyq/tmp/mesos-master  
>> /tmp/mesos-master.log 2>&1 &)
$ curl -d 
weights="[{\"weight\":1.0,\"role\":\"role1\"},{\"weight\":8.0,\"role\":\"role2\"}]"
 -X PUT http://localhost:5050/weights
$ curl http://localhost:5050/roles
{
"roles": [
{
"frameworks": [ ], 
"name": "*", 
"resources": {
"cpus": 0, 
"disk": 0, 
"mem": 0
}, 
"weight": 1
}, 
{
"frameworks": [ ], 
"name": "role1", 
"resources": {
"cpus": 0, 
"disk": 0, 
"mem": 0
}, 
"weight": 1
}, 
{
"frameworks": [ ], 
"name": "role2", 
"resources": {
"cpus": 0, 
"disk": 0, 
"mem": 0
}, 
"weight": 8
}
]
}


Thanks,

Yongqiao Wang

Re: Review Request 41597: Extending allocator interface to support dynamic weights

2015-12-31 Thread Yongqiao Wang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41597/
---

(Updated Dec. 31, 2015, 12:07 p.m.)


Review request for mesos, Adam B, Neil Conway, and Qian Zhang.


Changes
---

If the weight is updated to the default value (1.0), then remove from weights 
hashmap.


Bugs: MESOS-3943
https://issues.apache.org/jira/browse/MESOS-3943


Repository: mesos


Description
---

Add the interface in allocator to support updating weight
at runtime, and the allocator is invoked to allocate the
resources based on the updated weights later.


Diffs (updated)
-

  include/mesos/master/allocator.hpp f7ada68d7111486d264284990996413bb3d6 
  include/mesos/mesos.proto 2431fdd6b84625c6140a2b3913736bffada4e7f6 
  include/mesos/v1/mesos.proto 4aed0980b28dc1000aa2821f35303b736bc5bff8 
  src/master/allocator/mesos/allocator.hpp 
50ef3b20f34bc6d87cbeccabcebec9a5031a6554 
  src/master/allocator/mesos/hierarchical.hpp 
86ea5a402ed67f8f22f11d5730147cd907d66a08 
  src/master/allocator/mesos/hierarchical.cpp 
7f900c4e024485704d79e57ae22407557598fe6c 
  src/master/allocator/sorter/drf/sorter.hpp 
050896e8b12cd4097ccd137d5284d6b39b0f06ab 
  src/master/allocator/sorter/drf/sorter.cpp 
3a442f121f3a1505513877a5c78458a4b8d0a824 
  src/master/allocator/sorter/sorter.hpp 
7be6b44a762fd62c2cd7f28b4dc4865a4587ed26 
  src/tests/allocator.hpp 9bdfaecf1a148f113ad52956b50ed7cabe0902ef 

Diff: https://reviews.apache.org/r/41597/diff/


Testing
---

Make & Make check successfully!

Test case: https://reviews.apache.org/r/41672/


Thanks,

Yongqiao Wang

Re: Review Request 41597: Extending allocator interface to support dynamic weights

2015-12-31 Thread Yongqiao Wang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41597/
---

(Updated Dec. 31, 2015, 11:53 a.m.)


Review request for mesos, Adam B, Neil Conway, and Qian Zhang.


Changes
---

Rebase.


Bugs: MESOS-3943
https://issues.apache.org/jira/browse/MESOS-3943


Repository: mesos


Description
---

Add the interface in allocator to support updating weight
at runtime, and the allocator is invoked to allocate the
resources based on the updated weights later.


Diffs (updated)
-

  include/mesos/master/allocator.hpp f7ada68d7111486d264284990996413bb3d6 
  include/mesos/mesos.proto 2431fdd6b84625c6140a2b3913736bffada4e7f6 
  include/mesos/v1/mesos.proto 4aed0980b28dc1000aa2821f35303b736bc5bff8 
  src/master/allocator/mesos/allocator.hpp 
50ef3b20f34bc6d87cbeccabcebec9a5031a6554 
  src/master/allocator/mesos/hierarchical.hpp 
86ea5a402ed67f8f22f11d5730147cd907d66a08 
  src/master/allocator/mesos/hierarchical.cpp 
7f900c4e024485704d79e57ae22407557598fe6c 
  src/master/allocator/sorter/drf/sorter.hpp 
050896e8b12cd4097ccd137d5284d6b39b0f06ab 
  src/master/allocator/sorter/drf/sorter.cpp 
3a442f121f3a1505513877a5c78458a4b8d0a824 
  src/master/allocator/sorter/sorter.hpp 
7be6b44a762fd62c2cd7f28b4dc4865a4587ed26 
  src/tests/allocator.hpp 9bdfaecf1a148f113ad52956b50ed7cabe0902ef 

Diff: https://reviews.apache.org/r/41597/diff/


Testing
---

Make & Make check successfully!

Test case: https://reviews.apache.org/r/41672/


Thanks,

Yongqiao Wang

Re: Review Request 41491: Unified Container: Implemented passing entrypoint in runtime config.

2015-12-31 Thread Gilbert Song 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41491/
---

(Updated Dec. 31, 2015, 3:06 p.m.)


Review request for mesos, Artem Harutyunyan, Jie Yu, and Timothy Chen.


Bugs: MESOS-4225
https://issues.apache.org/jira/browse/MESOS-4225


Repository: mesos


Description
---

Unified Container: Implemented passing entrypoint in runtime config.


Diffs (updated)
-

  src/slave/containerizer/mesos/provisioner/docker/store.cpp 
e951c96a7f9c5ff72f6993981e2e3744e3b837f6 
  src/slave/containerizer/mesos/provisioner/provisioner.hpp 
b2e23d82ea255e1c91d7537a58f86632763d9a56 
  src/slave/containerizer/mesos/provisioner/provisioner.cpp 
89d06e94bd2255bb4f2dad3fae75c1b3d789611d 
  src/slave/containerizer/mesos/provisioner/store.hpp 
aec725f789f7aeb92abfcc6718c2e6e2f1f37981 
  src/tests/containerizer/provisioner_docker_tests.cpp 
bb142f5ea99e8ea9b20a896f95ae37aa1d8d3f98 

Diff: https://reviews.apache.org/r/41491/diff/


Testing
---

make check


Thanks,

Gilbert Song

Re: Review Request 41491: Unified Container: Exposed docker/appc image manifest to mesos containerizer.

2015-12-31 Thread Gilbert Song 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41491/
---

(Updated Dec. 31, 2015, 3:19 p.m.)


Review request for mesos, Artem Harutyunyan, Jie Yu, and Timothy Chen.


Summary (updated)
-

Unified Container: Exposed docker/appc image manifest to mesos containerizer.


Bugs: MESOS-4225
https://issues.apache.org/jira/browse/MESOS-4225


Repository: mesos


Description (updated)
---

Unified Container: Exposed docker/appc image manifest to mesos containerizer.


Diffs
-

  src/slave/containerizer/mesos/provisioner/docker/store.cpp 
e951c96a7f9c5ff72f6993981e2e3744e3b837f6 
  src/slave/containerizer/mesos/provisioner/provisioner.hpp 
b2e23d82ea255e1c91d7537a58f86632763d9a56 
  src/slave/containerizer/mesos/provisioner/provisioner.cpp 
89d06e94bd2255bb4f2dad3fae75c1b3d789611d 
  src/slave/containerizer/mesos/provisioner/store.hpp 
aec725f789f7aeb92abfcc6718c2e6e2f1f37981 
  src/tests/containerizer/provisioner_docker_tests.cpp 
bb142f5ea99e8ea9b20a896f95ae37aa1d8d3f98 

Diff: https://reviews.apache.org/r/41491/diff/


Testing
---

make check


Thanks,

Gilbert Song

Re: Review Request 41769: Made allocator traverse all roles for quota allocation.

2015-12-31 Thread Guangya Liu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41769/
---

(Updated 十二月 31, 2015, 11:34 p.m.)


Review request for mesos, Alexander Rukletsov, Klaus Ma, and Neil Conway.


Repository: mesos


Description
---

Made allocator traverse all roles for quota allocation.


Diffs (updated)
-

  src/master/allocator/mesos/hierarchical.cpp 
7f900c4e024485704d79e57ae22407557598fe6c 

Diff: https://reviews.apache.org/r/41769/diff/


Testing
---

make
make check


Thanks,

Guangya Liu

Re: Review Request 41829: WIP: Added test cases for allocator.

2015-12-31 Thread Guangya Liu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41829/
---

(Updated 十二月 31, 2015, 10:44 p.m.)


Review request for mesos and Klaus Ma.


Repository: mesos


Description
---

WIP: Added test cases for allocator.


Diffs (updated)
-

  src/tests/hierarchical_allocator_tests.cpp 
e044f832c2c16e53e663c6ced5452649bb0dcb59 

Diff: https://reviews.apache.org/r/41829/diff/


Testing (updated)
---

make
make check


Thanks,

Guangya Liu

Re: Review Request 41491: Unified Container: Implemented passing entrypoint in runtime config.

2015-12-31 Thread Gilbert Song 


> On Dec. 30, 2015, 6:41 p.m., Jie Yu wrote:
> > src/tests/containerizer/provisioner_docker_tests.cpp, line 1614
> > 
> >
> > Why change this test?

Because we need to os::write manifest to disk and move it to store directory. 
We have to make sure `layId` is identical as "456"(originally we only need 
`layerid - path` pair for MockPuller), and two `layerPath` are redundant. We 
only use the last one.


- Gilbert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41491/#review112369
---


On Dec. 31, 2015, 3:06 p.m., Gilbert Song wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41491/
> ---
> 
> (Updated Dec. 31, 2015, 3:06 p.m.)
> 
> 
> Review request for mesos, Artem Harutyunyan, Jie Yu, and Timothy Chen.
> 
> 
> Bugs: MESOS-4225
> https://issues.apache.org/jira/browse/MESOS-4225
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Unified Container: Implemented passing entrypoint in runtime config.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/provisioner/docker/store.cpp 
> e951c96a7f9c5ff72f6993981e2e3744e3b837f6 
>   src/slave/containerizer/mesos/provisioner/provisioner.hpp 
> b2e23d82ea255e1c91d7537a58f86632763d9a56 
>   src/slave/containerizer/mesos/provisioner/provisioner.cpp 
> 89d06e94bd2255bb4f2dad3fae75c1b3d789611d 
>   src/slave/containerizer/mesos/provisioner/store.hpp 
> aec725f789f7aeb92abfcc6718c2e6e2f1f37981 
>   src/tests/containerizer/provisioner_docker_tests.cpp 
> bb142f5ea99e8ea9b20a896f95ae37aa1d8d3f98 
> 
> Diff: https://reviews.apache.org/r/41491/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Gilbert Song
> 
>

Re: Review Request 41334: Added helper functions to filter allocation slack resources.

2015-12-31 Thread Guangya Liu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41334/
---

(Updated 一月 1, 2016, 12:08 a.m.)


Review request for mesos, Ben Mahler, Artem Harutyunyan, Joris Van Remoortere, 
Joseph Wu, and Klaus Ma.


Summary (updated)
-

Added helper functions to filter allocation slack resources.


Bugs: MESOS-4146
https://issues.apache.org/jira/browse/MESOS-4146


Repository: mesos


Description (updated)
---

This helper function is used to filter out allocation slack resources.


Diffs
-

  include/mesos/resources.hpp 9c8bfb0e7e1e40ca99b4cc75079e5b950630243d 
  include/mesos/v1/resources.hpp db87c992e2ec466cb1d4d17775bf6423a2fb15aa 
  src/common/resources.cpp b9d31aef8babee212374e352c57fadbff02167f3 
  src/tests/resources_tests.cpp e4a3435adc14f3b6b278b32348a6991543d5a320 
  src/v1/resources.cpp 0c0eee6b3b7ae3254d7dc7b06187855c9b873764 

Diff: https://reviews.apache.org/r/41334/diff/


Testing
---

make
make check


Thanks,

Guangya Liu

Re: Review Request 41334: Added helper functions to filter allocation slack resources.

2015-12-31 Thread Guangya Liu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41334/
---

(Updated 一月 1, 2016, 12:51 a.m.)


Review request for mesos, Ben Mahler, Artem Harutyunyan, Joris Van Remoortere, 
Joseph Wu, and Klaus Ma.


Bugs: MESOS-4146
https://issues.apache.org/jira/browse/MESOS-4146


Repository: mesos


Description
---

This helper function is used to filter out allocation slack resources.


Diffs (updated)
-

  include/mesos/resources.hpp 9c8bfb0e7e1e40ca99b4cc75079e5b950630243d 
  include/mesos/v1/resources.hpp db87c992e2ec466cb1d4d17775bf6423a2fb15aa 
  src/common/resources.cpp b9d31aef8babee212374e352c57fadbff02167f3 
  src/tests/resources_tests.cpp e4a3435adc14f3b6b278b32348a6991543d5a320 
  src/v1/resources.cpp 0c0eee6b3b7ae3254d7dc7b06187855c9b873764 

Diff: https://reviews.apache.org/r/41334/diff/


Testing
---

make
make check


Thanks,

Guangya Liu