[jira] [Commented] (MESOS-3161) Document using the gold linker for faster development on linux.
[ https://issues.apache.org/jira/browse/MESOS-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125885#comment-15125885 ] Deshi Xiao commented on MESOS-3161: --- +1 on documents. > Document using the gold linker for faster development on linux. > --- > > Key: MESOS-3161 > URL: https://issues.apache.org/jira/browse/MESOS-3161 > Project: Mesos > Issue Type: Improvement > Components: build, documentation >Reporter: Benjamin Mahler > Labels: newbie > > The [gold linker|https://en.wikipedia.org/wiki/Gold_(linker)] seems to > provide a decent speedup (about ~20%) on a parallel build. From a quick test: > {noformat: title=timings for make check -j24 GTEST_FILTER="" w/ 24 > hyperthreaded cores} > gold: > real 7m18.526s > user 81m21.213s > sys 5m17.224s > default ld: > real 9m7.908s > user 85m13.466s > sys 5m52.199s > {noformat} > On CentOS 5 w/ devtoolset-2: > {noformat} > sudo /usr/sbin/alternatives --altdir > /opt/rh/devtoolset-2/root/etc/alternatives --admindir > /opt/rh/devtoolset-2/root/var/lib/alternatives --set ld > /opt/rh/devtoolset-2/root/usr/bin/ld.gold > {noformat} > On Ubuntu: > {noformat} > sudo update-alternatives --install /usr/bin/ld ld /usr/bin/gold 1 > {noformat} > Ideally we could this out on the website, with instructions for each OS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4435) Update `Master::Http::stateSummary` to use `jsonify`.
[ https://issues.apache.org/jira/browse/MESOS-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Park updated MESOS-4435: Shepherd: Joris Van Remoortere (was: Benjamin Hindman) > Update `Master::Http::stateSummary` to use `jsonify`. > - > > Key: MESOS-4435 > URL: https://issues.apache.org/jira/browse/MESOS-4435 > Project: Mesos > Issue Type: Task > Components: master >Reporter: Michael Park >Assignee: Michael Park > Fix For: 0.28.0 > > > Update {{state-summary}} to use {{jsonify}} to stay consistent with {{state}} > HTTP endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4566) Avoid unnecessary temporary `std::string` constructions and copies in `jsonify`.
[ https://issues.apache.org/jira/browse/MESOS-4566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125777#comment-15125777 ] Michael Park commented on MESOS-4566: - {noformat} commit 5e064503eb519aea3329f65f7ac93370b2910cc7 Author: Michael Park Date: Sun Jan 31 20:38:34 2016 -0800 Avoid unnecessary string copies in `json` for protobuf messages. Review: https://reviews.apache.org/r/43024/ {noformat} {noformat} commit 8daa9a5c842a90b7b824a053c5f45897044de5fa Author: Michael Park Date: Sun Jan 31 20:37:42 2016 -0800 Avoid construction of temporary strings in `NumberWriter` for doubles. With this + https://reviews.apache.org/r/43024/, the number of calls to `operator new` and `operator delete` were reduced by roughly 1/3. Review: https://reviews.apache.org/r/43023/ {noformat} > Avoid unnecessary temporary `std::string` constructions and copies in > `jsonify`. > > > Key: MESOS-4566 > URL: https://issues.apache.org/jira/browse/MESOS-4566 > Project: Mesos > Issue Type: Improvement > Components: stout >Reporter: Michael Park >Assignee: Michael Park > Labels: mesosphere > Fix For: 0.28.0 > > > A few of the critical code paths in {{jsonify}} involve unnecessary temporary > string construction and copies (inherited from the {{JSON::*}}). For example, > {{strings::trim}} is used to remove trailing 0s from printing {{double}}s. We > print {{double}}s a lot, and therefore constructing a temporary > {{std::string}} on printing of every double is extremely costly. This ticket > captures the work involved in avoiding them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4566) Avoid unnecessary temporary `std::string` constructions and copies in `jsonify`.
[ https://issues.apache.org/jira/browse/MESOS-4566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Park updated MESOS-4566: Shepherd: Joris Van Remoortere (was: Benjamin Hindman) > Avoid unnecessary temporary `std::string` constructions and copies in > `jsonify`. > > > Key: MESOS-4566 > URL: https://issues.apache.org/jira/browse/MESOS-4566 > Project: Mesos > Issue Type: Improvement > Components: stout >Reporter: Michael Park >Assignee: Michael Park > Labels: mesosphere > Fix For: 0.28.0 > > > A few of the critical code paths in {{jsonify}} involve unnecessary temporary > string construction and copies (inherited from the {{JSON::*}}). For example, > {{strings::trim}} is used to remove trailing 0s from printing {{double}}s. We > print {{double}}s a lot, and therefore constructing a temporary > {{std::string}} on printing of every double is extremely costly. This ticket > captures the work involved in avoiding them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4145) Update allocator to get allocation slack resources
[ https://issues.apache.org/jira/browse/MESOS-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125602#comment-15125602 ] Guangya Liu commented on MESOS-4145: A high level design was uploaded here https://docs.google.com/document/d/1B_v52zCOFcwCpqCPhgYi9h630a0NE-QM9Br0nCOZUR4/edit > Update allocator to get allocation slack resources > -- > > Key: MESOS-4145 > URL: https://issues.apache.org/jira/browse/MESOS-4145 > Project: Mesos > Issue Type: Bug >Reporter: Guangya Liu >Assignee: Guangya Liu > > The allocator should be updated to > 1) Report allocation slack resources when add a new agent. > 2) Allocate allocation slack resources when sending offers. > 3) Update allocation slack resources when update agent, update dynamic > reservations with both framework and http endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4039) PerfEventIsolatorTest.ROOT_CGROUPS_Sample fails
[ https://issues.apache.org/jira/browse/MESOS-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125422#comment-15125422 ] haosdent commented on MESOS-4039: - According my test, it have two problem in CentOS 7. One is the perf format detect in CentOS 7 is incorrect, the anther one is it would remain process when exit. > PerfEventIsolatorTest.ROOT_CGROUPS_Sample fails > --- > > Key: MESOS-4039 > URL: https://issues.apache.org/jira/browse/MESOS-4039 > Project: Mesos > Issue Type: Bug >Reporter: Greg Mann >Assignee: Jan Schlicht > Labels: mesosphere, test-fail > > PerfEventIsolatorTest.ROOT_CGROUPS_Sample fails on CentOS 6.6: > {code} > [--] 1 test from PerfEventIsolatorTest > [ RUN ] PerfEventIsolatorTest.ROOT_CGROUPS_Sample > ../../src/tests/containerizer/isolator_tests.cpp:848: Failure > isolator: Perf is not supported > [ FAILED ] PerfEventIsolatorTest.ROOT_CGROUPS_Sample (79 ms) > [--] 1 test from PerfEventIsolatorTest (79 ms total) > [--] Global test environment tear-down > [==] 1 test from 1 test case ran. (86 ms total) > [ PASSED ] 0 tests. > [ FAILED ] 1 test, listed below: > [ FAILED ] PerfEventIsolatorTest.ROOT_CGROUPS_Sample > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-4446) Set Docker labels based on TaskInfo labels.
[ https://issues.apache.org/jira/browse/MESOS-4446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Dasgupta updated MESOS-4446: - Summary: Set Docker labels based on TaskInfo labels. (was: Set Docker labels based on TaskStatus labels.) > Set Docker labels based on TaskInfo labels. > --- > > Key: MESOS-4446 > URL: https://issues.apache.org/jira/browse/MESOS-4446 > Project: Mesos > Issue Type: Story > Components: docker >Reporter: Gennady Feldman >Assignee: Abhishek Dasgupta > > So looks like MESOS-3076 added support for Labels to TaskStatus. Would it be > possible to pass those onto the docker container? > This would really help with doing "docker inspect" on the mesos-slave nodes > as well as allow us to better collect docker metrics about the > tasks/containers that are currently running on the slave. > docker supports labels out of the box. See here: > https://docs.docker.com/engine/userguide/labels-custom-metadata/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4561) Investigate `std::string` to C string comparison.
[ https://issues.apache.org/jira/browse/MESOS-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125337#comment-15125337 ] Klaus Ma commented on MESOS-4561: - Just checked the {{operator==}} in Mac, it's using {{string::compare}}. And it seems no performance impact of 5e618ac, is there any test result about it? > Investigate `std::string` to C string comparison. > - > > Key: MESOS-4561 > URL: https://issues.apache.org/jira/browse/MESOS-4561 > Project: Mesos > Issue Type: Bug >Reporter: Michael Park > Labels: mesosphere > > [~jvanremoortere] identified a large performance bottleneck in role > validation logic and submitted a > [fix|https://github.com/apache/mesos/commit/5e618ac2197e856f09511850bebe51fea71c1844] > for the 0.27.0 release. > However, templated versions of {{bool operator==(const std::string&, const > char \*)}} and {{bool operator==(const char\*, const std::string&)}} are part > of the standard library > http://en.cppreference.com/w/cpp/string/basic_string/operator_cmp, and > therefore should not have triggered temporary string constructions. > The first thought was perhaps they don't happen to be implemented in > {{libstdc++}}, but having looked at its implementation, I was able to find > the definitions of these operators. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3049) Failing CGROUPS_ROOT_PerfRollForward test
[ https://issues.apache.org/jira/browse/MESOS-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125332#comment-15125332 ] haosdent commented on MESOS-3049: - Could I reopen this? Because I still met this in CentOS 7.1. > Failing CGROUPS_ROOT_PerfRollForward test > - > > Key: MESOS-3049 > URL: https://issues.apache.org/jira/browse/MESOS-3049 > Project: Mesos > Issue Type: Bug > Components: containerization, test >Affects Versions: 0.23.0 > Environment: CentOS Linux release 7.1.1503 > 0.23.0-rc3 >Reporter: Adam B >Assignee: Timothy Chen > > {code} > [ RUN ] MesosContainerizerSlaveRecoveryTest.CGROUPS_ROOT_PerfRollForward > I0715 00:28:42.600359 17919 exec.cpp:132] Version: 0.23.0 > I0715 00:28:42.609073 17983 exec.cpp:206] Executor registered on slave > 20150715-002842-201330860-41108-794-S0 > Registered executor on srv2.hw.ca1.mesosphere.com > Starting task b675699c-8427-4934-8bb5-62f91f3d74d0 > Forked command at 17988 > sh -c 'sleep 1000' > ../../src/tests/slave_recovery_tests.cpp:3446: Failure > containerizer2: Could not create isolator cgroups/perf_event: Failed to > create PerfEvent isolator, invalid events: { cycles, task-clock } > ../../3rdparty/libprocess/include/process/gmock.hpp:353: Failure > Actual function call count doesn't match EXPECT_CALL(filter->mock, > filter(testing::A()))... > Expected args: dispatch matcher (1-byte object <04>, 16-byte object > <29-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00>) > Expected: to be called once >Actual: never called - unsatisfied and active > [ FAILED ] MesosContainerizerSlaveRecoveryTest.CGROUPS_ROOT_PerfRollForward > (462 ms) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3049) Failing CGROUPS_ROOT_PerfRollForward test
[ https://issues.apache.org/jira/browse/MESOS-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125331#comment-15125331 ] haosdent commented on MESOS-3049: - Could I reopen this? Because I still met this in CentOS 7.1. > Failing CGROUPS_ROOT_PerfRollForward test > - > > Key: MESOS-3049 > URL: https://issues.apache.org/jira/browse/MESOS-3049 > Project: Mesos > Issue Type: Bug > Components: containerization, test >Affects Versions: 0.23.0 > Environment: CentOS Linux release 7.1.1503 > 0.23.0-rc3 >Reporter: Adam B >Assignee: Timothy Chen > > {code} > [ RUN ] MesosContainerizerSlaveRecoveryTest.CGROUPS_ROOT_PerfRollForward > I0715 00:28:42.600359 17919 exec.cpp:132] Version: 0.23.0 > I0715 00:28:42.609073 17983 exec.cpp:206] Executor registered on slave > 20150715-002842-201330860-41108-794-S0 > Registered executor on srv2.hw.ca1.mesosphere.com > Starting task b675699c-8427-4934-8bb5-62f91f3d74d0 > Forked command at 17988 > sh -c 'sleep 1000' > ../../src/tests/slave_recovery_tests.cpp:3446: Failure > containerizer2: Could not create isolator cgroups/perf_event: Failed to > create PerfEvent isolator, invalid events: { cycles, task-clock } > ../../3rdparty/libprocess/include/process/gmock.hpp:353: Failure > Actual function call count doesn't match EXPECT_CALL(filter->mock, > filter(testing::A()))... > Expected args: dispatch matcher (1-byte object <04>, 16-byte object > <29-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00>) > Expected: to be called once >Actual: never called - unsatisfied and active > [ FAILED ] MesosContainerizerSlaveRecoveryTest.CGROUPS_ROOT_PerfRollForward > (462 ms) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4039) PerfEventIsolatorTest.ROOT_CGROUPS_Sample fails
[ https://issues.apache.org/jira/browse/MESOS-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125330#comment-15125330 ] haosdent commented on MESOS-4039: - Today I try this in CentOS 7 which perf version is "perf version 3.10.0-229.11.1.el7.x86_64.debug". Still failed. > PerfEventIsolatorTest.ROOT_CGROUPS_Sample fails > --- > > Key: MESOS-4039 > URL: https://issues.apache.org/jira/browse/MESOS-4039 > Project: Mesos > Issue Type: Bug >Reporter: Greg Mann >Assignee: Jan Schlicht > Labels: mesosphere, test-fail > > PerfEventIsolatorTest.ROOT_CGROUPS_Sample fails on CentOS 6.6: > {code} > [--] 1 test from PerfEventIsolatorTest > [ RUN ] PerfEventIsolatorTest.ROOT_CGROUPS_Sample > ../../src/tests/containerizer/isolator_tests.cpp:848: Failure > isolator: Perf is not supported > [ FAILED ] PerfEventIsolatorTest.ROOT_CGROUPS_Sample (79 ms) > [--] 1 test from PerfEventIsolatorTest (79 ms total) > [--] Global test environment tear-down > [==] 1 test from 1 test case ran. (86 ms total) > [ PASSED ] 0 tests. > [ FAILED ] 1 test, listed below: > [ FAILED ] PerfEventIsolatorTest.ROOT_CGROUPS_Sample > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-4561) Investigate `std::string` to C string comparison.
[ https://issues.apache.org/jira/browse/MESOS-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125320#comment-15125320 ] haosdent edited comment on MESOS-4561 at 1/31/16 12:58 PM: --- I am not sure how to test, but I think it may related to this [Why '==' is slow on std::string|http://stackoverflow.com/questions/28734684/why-is-slow-on-stdstring] when you benchmark. According my test, the result are nearly same. My test code: {code} TEST(RolesTest, Benchmark) { int64_t ops = 1000; int64_t cycles = 100; string validateNames[ops]; for (int64_t i = 0; i < ops; ++i) { validateNames[i] = stringify(i); } Stopwatch watch; watch.start(); for (int64_t i = 0; i < ops * cycles; ++i) { roles::validate(validateNames[i % ops]); } Duration elapsed = watch.elapsed(); cout << "Estimated Total: " << (elapsed.ns() / ops / cycles) << " nanos / ops" << endl; } {code} The result when use [new version|https://github.com/apache/mesos/blob/5e618ac2197e856f09511850bebe51fea71c1844/src/common/roles.cpp] to test. {code} [==] Running 1 test from 1 test case. [--] Global test environment set-up. [--] 1 test from RolesTest [ RUN ] RolesTest.Benchmark Estimated Total: 49 nanos / ops [ OK ] RolesTest.Benchmark (49196 ms) [--] 1 test from RolesTest (49196 ms total) [--] Global test environment tear-down [==] 1 test from 1 test case ran. (49208 ms total) [ PASSED ] 1 test. {code} After revert to [old version|https://github.com/apache/mesos/blob/ffcef51f06f41f707e7026070f2335b956ed9d2c/src/common/roles.cpp], the result looks don't have big different: {code} [==] Running 1 test from 1 test case. [--] Global test environment set-up. [--] 1 test from RolesTest [ RUN ] RolesTest.Benchmark Estimated Total: 46 nanos / ops [ OK ] RolesTest.Benchmark (46327 ms) [--] 1 test from RolesTest (46327 ms total) [--] Global test environment tear-down [==] 1 test from 1 test case ran. (46339 ms total) [ PASSED ] 1 test. {code} was (Author: haosd...@gmail.com): I am not sure how to test, but I think it may related to this {{Why '==' is slow on std::string|http://stackoverflow.com/questions/28734684/why-is-slow-on-stdstring}} when you benchmark. According my test, the result are nearly same. My test code: {code} TEST(RolesTest, Benchmark) { int64_t ops = 1000; int64_t cycles = 100; string validateNames[ops]; for (int64_t i = 0; i < ops; ++i) { validateNames[i] = stringify(i); } Stopwatch watch; watch.start(); for (int64_t i = 0; i < ops * cycles; ++i) { roles::validate(validateNames[i % ops]); } Duration elapsed = watch.elapsed(); cout << "Estimated Total: " << (elapsed.ns() / ops / cycles) << " nanos / ops" << endl; } {code} The result when use {{new version|https://github.com/apache/mesos/blob/5e618ac2197e856f09511850bebe51fea71c1844/src/common/roles.cpp}} to test. {code} [==] Running 1 test from 1 test case. [--] Global test environment set-up. [--] 1 test from RolesTest [ RUN ] RolesTest.Benchmark Estimated Total: 49 nanos / ops [ OK ] RolesTest.Benchmark (49196 ms) [--] 1 test from RolesTest (49196 ms total) [--] Global test environment tear-down [==] 1 test from 1 test case ran. (49208 ms total) [ PASSED ] 1 test. {code} After revert to {{old version|https://github.com/apache/mesos/blob/ffcef51f06f41f707e7026070f2335b956ed9d2c/src/common/roles.cpp}}, the result looks don't have big different: {code} [==] Running 1 test from 1 test case. [--] Global test environment set-up. [--] 1 test from RolesTest [ RUN ] RolesTest.Benchmark Estimated Total: 46 nanos / ops [ OK ] RolesTest.Benchmark (46327 ms) [--] 1 test from RolesTest (46327 ms total) [--] Global test environment tear-down [==] 1 test from 1 test case ran. (46339 ms total) [ PASSED ] 1 test. {code} > Investigate `std::string` to C string comparison. > - > > Key: MESOS-4561 > URL: https://issues.apache.org/jira/browse/MESOS-4561 > Project: Mesos > Issue Type: Bug >Reporter: Michael Park > Labels: mesosphere > > [~jvanremoortere] identified a large performance bottleneck in role > validation logic and submitted a > [fix|https://github.com/apache/mesos/commit/5e618ac2197e856f09511850bebe51fea71c1844] > for the 0.27.0 release. > However, templated versions of {{bool operator==(const std::string&, const > char \*)}} and {{bool operator==(const char\*, const std::string&)}} are part > of the standard library > http://en.cppreference.com/w/cpp/string/basic_string/operator_cmp, and > therefore should not have trigg
[jira] [Commented] (MESOS-4561) Investigate `std::string` to C string comparison.
[ https://issues.apache.org/jira/browse/MESOS-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125320#comment-15125320 ] haosdent commented on MESOS-4561: - I am not sure how to test, but I think it may related to this {{Why '==' is slow on std::string|http://stackoverflow.com/questions/28734684/why-is-slow-on-stdstring}} when you benchmark. According my test, the result are nearly same. My test code: {code} TEST(RolesTest, Benchmark) { int64_t ops = 1000; int64_t cycles = 100; string validateNames[ops]; for (int64_t i = 0; i < ops; ++i) { validateNames[i] = stringify(i); } Stopwatch watch; watch.start(); for (int64_t i = 0; i < ops * cycles; ++i) { roles::validate(validateNames[i % ops]); } Duration elapsed = watch.elapsed(); cout << "Estimated Total: " << (elapsed.ns() / ops / cycles) << " nanos / ops" << endl; } {code} The result when use {{new version|https://github.com/apache/mesos/blob/5e618ac2197e856f09511850bebe51fea71c1844/src/common/roles.cpp}} to test. {code} [==] Running 1 test from 1 test case. [--] Global test environment set-up. [--] 1 test from RolesTest [ RUN ] RolesTest.Benchmark Estimated Total: 49 nanos / ops [ OK ] RolesTest.Benchmark (49196 ms) [--] 1 test from RolesTest (49196 ms total) [--] Global test environment tear-down [==] 1 test from 1 test case ran. (49208 ms total) [ PASSED ] 1 test. {code} After revert to {{old version|https://github.com/apache/mesos/blob/ffcef51f06f41f707e7026070f2335b956ed9d2c/src/common/roles.cpp}}, the result looks don't have big different: {code} [==] Running 1 test from 1 test case. [--] Global test environment set-up. [--] 1 test from RolesTest [ RUN ] RolesTest.Benchmark Estimated Total: 46 nanos / ops [ OK ] RolesTest.Benchmark (46327 ms) [--] 1 test from RolesTest (46327 ms total) [--] Global test environment tear-down [==] 1 test from 1 test case ran. (46339 ms total) [ PASSED ] 1 test. {code} > Investigate `std::string` to C string comparison. > - > > Key: MESOS-4561 > URL: https://issues.apache.org/jira/browse/MESOS-4561 > Project: Mesos > Issue Type: Bug >Reporter: Michael Park > Labels: mesosphere > > [~jvanremoortere] identified a large performance bottleneck in role > validation logic and submitted a > [fix|https://github.com/apache/mesos/commit/5e618ac2197e856f09511850bebe51fea71c1844] > for the 0.27.0 release. > However, templated versions of {{bool operator==(const std::string&, const > char \*)}} and {{bool operator==(const char\*, const std::string&)}} are part > of the standard library > http://en.cppreference.com/w/cpp/string/basic_string/operator_cmp, and > therefore should not have triggered temporary string constructions. > The first thought was perhaps they don't happen to be implemented in > {{libstdc++}}, but having looked at its implementation, I was able to find > the definitions of these operators. -- This message was sent by Atlassian JIRA (v6.3.4#6332)