[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518508#comment-16518508 ]
Todd Lipcon commented on HADOOP-15550: -------------------------------------- Benchmarked with a simple program that does {{new Path("/").getFileSystem(new Configuration());}} The attached patch avoids loading about 400 classes, and saves some measurable CPU: {code} without patch (2219 classes loaded): 1378.393961 task-clock (msec) # 1.959 CPUs utilized ( +- 0.57% ) 2,076 context-switches # 0.002 M/sec ( +- 0.62% ) 45 cpu-migrations # 0.033 K/sec ( +- 3.75% ) 30,529 page-faults # 0.022 M/sec ( +- 0.24% ) 4,540,069,263 cycles # 3.294 GHz ( +- 0.92% ) 5,282,002,987 instructions # 1.16 insn per cycle ( +- 0.93% ) 991,080,821 branches # 719.011 M/sec ( +- 0.90% ) 40,313,544 branch-misses # 4.07% of all branches ( +- 0.67% ) 0.703624736 seconds time elapsed ( +- 0.72% ) with patch (1821 classes loaded): 1269.949263 task-clock (msec) # 2.082 CPUs utilized ( +- 1.11% ) 2,008 context-switches # 0.002 M/sec ( +- 0.76% ) 51 cpu-migrations # 0.040 K/sec ( +- 8.14% ) 25,034 page-faults # 0.020 M/sec ( +- 0.26% ) 4,157,369,649 cycles # 3.274 GHz ( +- 0.78% ) 4,674,086,838 instructions # 1.12 insn per cycle ( +- 0.42% ) 870,359,803 branches # 685.350 M/sec ( +- 0.41% ) 36,028,258 branch-misses # 4.14% of all branches ( +- 0.44% ) 0.610038881 seconds time elapsed ( +- 1.54% ) {code} > Avoid static initialization of ObjectMappers > -------------------------------------------- > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance > Affects Versions: 3.2.0 > Reporter: Todd Lipcon > Assignee: Todd Lipcon > Priority: Minor > Attachments: hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org