[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643956#comment-16643956 ] Jonathan Eagles commented on HADOOP-15550: -- Going ahead on the proposed plan. I have cherry-picked this commit to branch-3.1. Created HADOOP-15835 to gain a partial benefit of ObjectMapper reuse. > Avoid static initialization of ObjectMappers > > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance >Affects Versions: 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Fix For: 3.2.0, 3.1.2 > > Attachments: hadoop-15550.txt, hadoop-15550.txt, hadoop-15550.txt, > hadoop-15550.txt, hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640251#comment-16640251 ] Jonathan Eagles commented on HADOOP-15550: -- I have some interest in back-porting this improvement to 2.8 (Mainly to get rid of the new ObjectMapper creation for KMSJSONWriter). Seems like a simple cherry-pick to 3.1 is possible when JsonSerialization library was added as part of HADOOP-13786. Back-porting to earlier lines either means partial back-porting HADOOP-13786 or creating a small KMSJSONWriter custom performance patch to remove the ObjectMapper creation. [~ste...@apache.org], [~tlipcon] do you have preference on this? It seems the most straight forward to 1) back-port this jira to 3.1. and 2) Create a separate KMSJSONWriter jira for earlier lines, but I am open to suggestions. > Avoid static initialization of ObjectMappers > > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance >Affects Versions: 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Fix For: 3.2.0 > > Attachments: hadoop-15550.txt, hadoop-15550.txt, hadoop-15550.txt, > hadoop-15550.txt, hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522925#comment-16522925 ] Hudson commented on HADOOP-15550: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14479 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14479/]) HADOOP-15550. Avoid static initialization of ObjectMappers (todd: rev 7a3c6e9c3cd9ffdc71946fd12f5c3d59718c4939) * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticator.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HttpExceptionUtils.java * (edit) hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSJSONWriter.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JsonSerialization.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * (edit) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/security/JsonUtils.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java > Avoid static initialization of ObjectMappers > > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance >Affects Versions: 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Fix For: 3.2.0 > > Attachments: hadoop-15550.txt, hadoop-15550.txt, hadoop-15550.txt, > hadoop-15550.txt, hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522898#comment-16522898 ] Xiao Chen commented on HADOOP-15550: +1, thanks for identifying and fixing this Todd! Also thanks Steve for the review and idea. > Avoid static initialization of ObjectMappers > > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance >Affects Versions: 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Attachments: hadoop-15550.txt, hadoop-15550.txt, hadoop-15550.txt, > hadoop-15550.txt, hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[
https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522859#comment-16522859
]
genericqa commented on HADOOP-15550:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
3s{color} | {color:blue} The patch file was not named according to hadoop's
naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute
for instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 28m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 11s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
57s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 28m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 25s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
59s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
11s{color} | {color:green} hadoop-kms in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
42s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
0s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
13s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522679#comment-16522679 ] Todd Lipcon commented on HADOOP-15550: -- Fixed those members to be final. Thanks for catching that, Xiao. > Avoid static initialization of ObjectMappers > > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance >Affects Versions: 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Attachments: hadoop-15550.txt, hadoop-15550.txt, hadoop-15550.txt, > hadoop-15550.txt, hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[
https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520740#comment-16520740
]
Xiao Chen commented on HADOOP-15550:
Thanks Todd and Steve. Latest patch looks great!
Sorry for the nit, {{WRITER}} and {{MAP_READER}} could be final (seem to recall
checkstyle should complain that, not sure why it doesn't this time). +1 from me
pending that.
> Avoid static initialization of ObjectMappers
>
>
> Key: HADOOP-15550
> URL: https://issues.apache.org/jira/browse/HADOOP-15550
> Project: Hadoop Common
> Issue Type: Bug
> Components: performance
>Affects Versions: 3.2.0
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
>Priority: Minor
> Attachments: hadoop-15550.txt, hadoop-15550.txt, hadoop-15550.txt,
> hadoop-15550.txt
>
>
> Various classes statically initialize an ObjectMapper READER instance. This
> ends up doing a bunch of class-loading of Jackson libraries that can add up
> to a fair amount of CPU, even if the reader ends up not being used. This is
> particularly the case with WebHdfsFileSystem, which is class-loaded by a
> serviceloader even when unused in a particular job. We should lazy-init these
> members instead of doing so as a static class member.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[
https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520702#comment-16520702
]
genericqa commented on HADOOP-15550:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
3s{color} | {color:blue} The patch file was not named according to hadoop's
naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute
for instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 33m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 34m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 9s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
48s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 26m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 26m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 51s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
49s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
1s{color} | {color:green} hadoop-kms in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
38s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
45s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
12s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}167m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[
https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519953#comment-16519953
]
genericqa commented on HADOOP-15550:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
3s{color} | {color:blue} The patch file was not named according to hadoop's
naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute
for instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
54s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 6s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 28m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
54s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 15s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
40s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
59s{color} | {color:green} hadoop-kms in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
39s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
47s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
13s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519877#comment-16519877 ] Todd Lipcon commented on HADOOP-15550: -- [~ste...@apache.org] thanks for taking a look. I uploaded a new patch and verified that it has the same effect. I no longer needed to lazy-init, because now it's the classloading of JsonSerialization that triggers Jackson classloading. I figure anyone who classloads the former is probably going to use Jackson and it wasn't worth the lazy load of Jackson. > Avoid static initialization of ObjectMappers > > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance >Affects Versions: 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Attachments: hadoop-15550.txt, hadoop-15550.txt, hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[
https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519756#comment-16519756
]
genericqa commented on HADOOP-15550:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
30s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hadoop's
naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute
for instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 45s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
4s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 28m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 4s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
51s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
39s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
44s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}146m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-15550 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12928655/hadoop-15550.txt |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519726#comment-16519726 ] Steve Loughran commented on HADOOP-15550: - Given it's ~ the same code everywhere, could this actually be made a method more broadly used? FWIW, I've been using non-static implementations in the org.apache.hadoop.util.JsonSerialization class; adding a static method to do on-demand creation of that class would suite me... > Avoid static initialization of ObjectMappers > > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance >Affects Versions: 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Attachments: hadoop-15550.txt, hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519656#comment-16519656 ] Xiao Chen commented on HADOOP-15550: +1. Thanks Todd! (Sorry I wasn't explicit, the KMSCP and DTA classes are client-side) > Avoid static initialization of ObjectMappers > > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance >Affects Versions: 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Attachments: hadoop-15550.txt, hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519563#comment-16519563 ] Todd Lipcon commented on HADOOP-15550: -- Sure, I'll make that change and upload a new rev later today. > Avoid static initialization of ObjectMappers > > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance >Affects Versions: 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Attachments: hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[
https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519556#comment-16519556
]
Xiao Chen commented on HADOOP-15550:
Thanks for the explanation. {{HttpExceptionUtils#validateResponse}} is called
by KMSClientProvider and DelegationTokenAuthenticator. So I suggest we change
this one. Other places as-is seems fine by me. :)
+1 pending that and pre-commit (agree no tests necessary)
> Avoid static initialization of ObjectMappers
>
>
> Key: HADOOP-15550
> URL: https://issues.apache.org/jira/browse/HADOOP-15550
> Project: Hadoop Common
> Issue Type: Bug
> Components: performance
>Affects Versions: 3.2.0
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
>Priority: Minor
> Attachments: hadoop-15550.txt
>
>
> Various classes statically initialize an ObjectMapper READER instance. This
> ends up doing a bunch of class-loading of Jackson libraries that can add up
> to a fair amount of CPU, even if the reader ends up not being used. This is
> particularly the case with WebHdfsFileSystem, which is class-loaded by a
> serviceloader even when unused in a particular job. We should lazy-init these
> members instead of doing so as a static class member.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16519531#comment-16519531 ] Todd Lipcon commented on HADOOP-15550: -- I guess no harm in changing it everywhere. I just looked for the ones that seemed like they were in classes that would likely be loaded by a client (eg MR task) but not actually used in many cases. For example, WebHDFSFileSystem is loaded because it gets picked up by the ServiceLoader used in FileSystem. By contrast, when I looked at HttpExceptionUtils it seemed like it was a server-side only class, and shaving 50ms here or there off daemon startup didn't seem particularly compelling :) > Avoid static initialization of ObjectMappers > > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance >Affects Versions: 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Attachments: hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[ https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518965#comment-16518965 ] Xiao Chen commented on HADOOP-15550: Thanks for the work here Todd. Looks pretty good to me. Just curious, what's the rational on choosing what to change on this jira? Should we also fix HttpExceptionUtils? > Avoid static initialization of ObjectMappers > > > Key: HADOOP-15550 > URL: https://issues.apache.org/jira/browse/HADOOP-15550 > Project: Hadoop Common > Issue Type: Bug > Components: performance >Affects Versions: 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Attachments: hadoop-15550.txt > > > Various classes statically initialize an ObjectMapper READER instance. This > ends up doing a bunch of class-loading of Jackson libraries that can add up > to a fair amount of CPU, even if the reader ends up not being used. This is > particularly the case with WebHdfsFileSystem, which is class-loaded by a > serviceloader even when unused in a particular job. We should lazy-init these > members instead of doing so as a static class member. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[
https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518630#comment-16518630
]
genericqa commented on HADOOP-15550:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hadoop's
naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute
for instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 21s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 8s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
26s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
48s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
5s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}146m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-15550 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12928522/hadoop-15550.txt |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient
[jira] [Commented] (HADOOP-15550) Avoid static initialization of ObjectMappers
[
https://issues.apache.org/jira/browse/HADOOP-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518508#comment-16518508
]
Todd Lipcon commented on HADOOP-15550:
--
Benchmarked with a simple program that does {{new Path("/").getFileSystem(new
Configuration());}}
The attached patch avoids loading about 400 classes, and saves some measurable
CPU:
{code}
without patch (2219 classes loaded):
1378.393961 task-clock (msec) #1.959 CPUs utilized
( +- 0.57% )
2,076 context-switches #0.002 M/sec
( +- 0.62% )
45 cpu-migrations#0.033 K/sec
( +- 3.75% )
30,529 page-faults #0.022 M/sec
( +- 0.24% )
4,540,069,263 cycles#3.294 GHz
( +- 0.92% )
5,282,002,987 instructions #1.16 insn per cycle
( +- 0.93% )
991,080,821 branches # 719.011 M/sec
( +- 0.90% )
40,313,544 branch-misses #4.07% of all branches
( +- 0.67% )
0.703624736 seconds time elapsed
( +- 0.72% )
with patch (1821 classes loaded):
1269.949263 task-clock (msec) #2.082 CPUs utilized
( +- 1.11% )
2,008 context-switches #0.002 M/sec
( +- 0.76% )
51 cpu-migrations#0.040 K/sec
( +- 8.14% )
25,034 page-faults #0.020 M/sec
( +- 0.26% )
4,157,369,649 cycles#3.274 GHz
( +- 0.78% )
4,674,086,838 instructions #1.12 insn per cycle
( +- 0.42% )
870,359,803 branches # 685.350 M/sec
( +- 0.41% )
36,028,258 branch-misses #4.14% of all branches
( +- 0.44% )
0.610038881 seconds time elapsed
( +- 1.54% )
{code}
> Avoid static initialization of ObjectMappers
>
>
> Key: HADOOP-15550
> URL: https://issues.apache.org/jira/browse/HADOOP-15550
> Project: Hadoop Common
> Issue Type: Bug
> Components: performance
>Affects Versions: 3.2.0
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
>Priority: Minor
> Attachments: hadoop-15550.txt
>
>
> Various classes statically initialize an ObjectMapper READER instance. This
> ends up doing a bunch of class-loading of Jackson libraries that can add up
> to a fair amount of CPU, even if the reader ends up not being used. This is
> particularly the case with WebHdfsFileSystem, which is class-loaded by a
> serviceloader even when unused in a particular job. We should lazy-init these
> members instead of doing so as a static class member.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
