----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44251/#review121757 -----------------------------------------------------------
Ship it! Ship It! - Joris Van Remoortere On March 2, 2016, 7:17 p.m., Neil Conway wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44251/ > ----------------------------------------------------------- > > (Updated March 2, 2016, 7:17 p.m.) > > > Review request for mesos and Joris Van Remoortere. > > > Bugs: MESOS-4833 > https://issues.apache.org/jira/browse/MESOS-4833 > > > Repository: mesos > > > Description > ------- > > When the cluster contains many resources that have either labeled > reservations or persistent volumes, allocator performance can decrease > substantially because such metadata prevents `Resource` objects from > being merged together inside the allocator. As a result, the allocator > must manipulate `Resources` vectors that consist of many small > individual `Resource` values; since many `Resources` operations take > linear-time in the number of `Resource` values they contain, this can > cause very significant slowdowns. > > As a short-term solution, this commit strips dynamic reservation and > persistent volume information from the `Resources` objects used > internally by the allocator, because they are not needed when > aggregating resource quantities together. > > A long-term solution for this problem will be addressed as work on > refactoring the allocator more generally. > > > Diffs > ----- > > src/master/allocator/mesos/hierarchical.cpp > 24fa50f62dec10ed43089297473cc386d6ba2f78 > src/master/allocator/sorter/drf/sorter.hpp > 46b2a9caf13b028a3aee6c1590679f885be90fd6 > src/master/allocator/sorter/drf/sorter.cpp > 9e863dd0ca5e2f2f0d517cb833687b757bed2c52 > src/master/allocator/sorter/sorter.hpp > ba91a38e47065718af87c9b3b7c5b74d25a258df > > Diff: https://reviews.apache.org/r/44251/diff/ > > > Testing > ------- > > make check > > Perf: > > ``` > [ RUN ] HierarchicalAllocator_BENCHMARK_Test.ResourceLabels > Using 2000 slaves and 200 frameworks > round 0 allocate took 2.458462secs to make 200 offers > round 1 allocate took 2.427941secs to make 200 offers > round 2 allocate took 2.460724secs to make 200 offers > round 3 allocate took 2.443408secs to make 200 offers > round 4 allocate took 2.464784secs to make 200 offers > round 5 allocate took 2.501429secs to make 200 offers > round 6 allocate took 2.468777secs to make 200 offers > round 7 allocate took 2.482268secs to make 200 offers > round 8 allocate took 2.479014secs to make 200 offers > round 9 allocate took 2.529951secs to make 200 offers > round 10 allocate took 2.460059secs to make 200 offers > ``` > > Performance of `DeclineOffers` without labels is about ~2.1 seconds. > > > Thanks, > > Neil Conway > >
