[ https://issues.apache.org/jira/browse/HADOOP-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HADOOP-15549: --------------------------------- Status: Patch Available (was: Open) Re-measured on trunk instead of 3.0.0, and also used 'taskset -c 1' to bind everything to one core to try to reduce variance: 2.8.2: {code} 535.052533 task-clock (msec) # 0.918 CPUs utilized ( +- 0.88% ) 3,055 context-switches # 0.006 M/sec ( +- 1.02% ) 1 cpu-migrations # 0.002 K/sec 12,644 page-faults # 0.024 M/sec ( +- 0.06% ) 1,953,309,627 cycles # 3.651 GHz ( +- 0.16% ) 2,221,327,797 instructions # 1.14 insn per cycle ( +- 0.17% ) 417,919,978 branches # 781.082 M/sec ( +- 0.17% ) 18,726,810 branch-misses # 4.48% of all branches ( +- 0.19% ) 0.582855783 seconds time elapsed ( +- 1.70% ) {code} 3.2 without patch: {code} 751.038338 task-clock (msec) # 0.927 CPUs utilized ( +- 0.43% ) 3,646 context-switches # 0.005 M/sec ( +- 2.26% ) 1 cpu-migrations # 0.001 K/sec ( +- 25.00% ) 19,233 page-faults # 0.026 M/sec ( +- 0.42% ) 2,735,218,817 cycles # 3.642 GHz ( +- 0.62% ) 3,218,012,767 instructions # 1.18 insn per cycle ( +- 0.54% ) 604,477,739 branches # 804.856 M/sec ( +- 0.48% ) 25,664,033 branch-misses # 4.25% of all branches ( +- 0.55% ) 0.810230298 seconds time elapsed ( +- 0.98% ) {code} 3.2 with patch: {code} 3.2 with patch: 679.940626 task-clock (msec) # 0.918 CPUs utilized ( +- 1.14% ) 3,302 context-switches # 0.005 M/sec ( +- 1.16% ) 1 cpu-migrations # 0.001 K/sec 16,819 page-faults # 0.025 M/sec ( +- 0.06% ) 2,375,283,537 cycles # 3.493 GHz ( +- 0.33% ) 2,722,724,476 instructions # 1.15 insn per cycle ( +- 0.27% ) 511,944,028 branches # 752.925 M/sec ( +- 0.24% ) 21,981,131 branch-misses # 4.29% of all branches ( +- 0.33% ) 0.740316578 seconds time elapsed ( +- 0.96% ) {code} > Upgrade to commons-configuration 2.1 regresses task CPU consumption > ------------------------------------------------------------------- > > Key: HADOOP-15549 > URL: https://issues.apache.org/jira/browse/HADOOP-15549 > Project: Hadoop Common > Issue Type: Bug > Components: metrics > Affects Versions: 3.0.2 > Reporter: Todd Lipcon > Assignee: Todd Lipcon > Priority: Major > Attachments: hadoop-15549.txt > > > HADOOP-13660 upgraded from commons-configuration 1.x to 2.x. > commons-configuration is used when parsing the metrics configuration > properties file. The new builder API used in the new version apparently makes > use of a bunch of very bloated reflection and classloading nonsense to > achieve the same goal, and this results in a regression of >100ms of CPU time > as measured by a program which simply initializes DefaultMetricsSystem. > This isn't a big deal for long-running daemons, but for MR tasks which might > only run a few seconds on poorly-tuned jobs, this can be noticeable. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org