[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124670#comment-15124670
]
Akira AJISAKA commented on HDFS-6565:
-
Hi [~sztanko], I don't think replacing jetty with jackson itself is the cause.
I'm thinking the cause is that the patch does not re-use {{ObjectMapper}}
instance.
Filed HDFS-9724.
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Fix For: 2.7.0
>
> Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch,
> HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121913#comment-15121913 ] Demeter Sztanko commented on HDFS-6565: --- We are experiencing performance degradation of webhdfs, when listing directories with large amount of files (>1000). Our profiling is indicating that most of resources are spent within Jackson library. Do you think replacing jetty with jackson could be the cause of this? > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Fix For: 2.7.0 > > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, > HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347038#comment-14347038 ] Hudson commented on HDFS-6565: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2072 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2072/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Fix For: 2.7.0 > > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, > HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347001#comment-14347001 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #122 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/122/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Fix For: 2.7.0 > > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, > HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346935#comment-14346935 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #113 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/113/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Fix For: 2.7.0 > > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, > HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346926#comment-14346926 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2054 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2054/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Fix For: 2.7.0 > > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, > HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346721#comment-14346721 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-Yarn-trunk #856 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/856/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Fix For: 2.7.0 > > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, > HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346711#comment-14346711 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #122 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/122/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Fix For: 2.7.0 > > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, > HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346272#comment-14346272 ] Akira AJISAKA commented on HDFS-6565: - Thanks Haohui! > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Fix For: 2.7.0 > > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, > HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346267#comment-14346267
]
Hadoop QA commented on HDFS-6565:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12702293/HDFS-6565-005.patch
against trunk revision e17e5ba.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode
org.apache.hadoop.hdfs.server.namenode.TestFileTruncate
The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9715//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9715//console
This message is automatically generated.
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Fix For: 2.7.0
>
> Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch,
> HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346229#comment-14346229 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-trunk-Commit #7249 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7249/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Fix For: 2.7.0 > > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, > HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346216#comment-14346216 ] Haohui Mai commented on HDFS-6565: -- +1. I'm committing this. > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, > HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346071#comment-14346071
]
Hadoop QA commented on HDFS-6565:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12702263/HDFS-6565-004.patch
against trunk revision e17e5ba.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.qjournal.client.TestQJMWithFaults
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9713//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9713//console
This message is automatically generated.
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch,
> HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345942#comment-14345942
]
Haohui Mai commented on HDFS-6565:
--
As a result the following methods are no longer used:
{code}
public static String toJsonString(final Token[]
tokens
private static List> toTokenList(final List objects)
{code}
Can you please remove these methods as well?
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch,
> HDFS-6565-004.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345761#comment-14345761
]
Haohui Mai commented on HDFS-6565:
--
Can you please update the patch to remove the {{toTokenList()}} methods? Other
than that the v3 patch looks pretty good to me.
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345758#comment-14345758 ] Haohui Mai commented on HDFS-6565: -- bq. Okay. I suppose that should be documented. Would you mind adding @InterfaceAudience.Private to the class? There are consensus on assuming unannotated class are Private-Unstable. Please see http://marc.info/?t=14060662831&r=1&w=2 > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344770#comment-14344770
]
Hadoop QA commented on HDFS-6565:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12702086/HDFS-6565-003.patch
against trunk revision d1c6acc.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-hdfs-project/hadoop-hdfs
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9708//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9708//console
This message is automatically generated.
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344679#comment-14344679
]
Akira AJISAKA commented on HDFS-6565:
-
bq. The class is not intended to be used other classes other than
WebHdfsFileSystem and NameNodeWebHdfsMethods.
Okay. I suppose that should be documented. Would you mind adding
{{@InterfaceAudience.Private}} to the class?
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344675#comment-14344675
]
Haohui Mai commented on HDFS-6565:
--
bq. It's a public API. I'm thinking deleting it will break compatibility.
The class is not intended to be used other classes other than
{{WebHdfsFileSystem}} and {{NameNodeWebHdfsMethods}}. I think that it is fine
to remove it.
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344585#comment-14344585
]
Akira AJISAKA commented on HDFS-6565:
-
Thanks [~wheat9] for the review.
bq. Looks like the function can be deleted.
It's a public API. I'm thinking deleting it will break compatibility.
{quote}
Simply make it like:
{code}
} catch (IOException ignored) {
}
{code}
{quote}
I'll refrect that in the next patch.
bq. There are many changes like this – it seems that they are unnecessary?
It is necessary. See my previous
[comment|https://issues.apache.org/jira/browse/HDFS-6565?focusedCommentId=14041409&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14041409].
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Attachments: HDFS-6565-002.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344555#comment-14344555
]
Haohui Mai commented on HDFS-6565:
--
Looks pretty good to me.
{code}
public static List> toTokenList(final Map json) throws
IOException {
{code}
Looks like the function can be deleted.
{code}
+} catch (IOException e) {
+ // can not reach here
+}
{code}
Simply make it like:
{code}
} catch (IOException ignored) {
}
{code}
{code}
+final long len = ((Number) m.get("length")).longValue();
{code}
There are many changes like this -- it seems that they are unnecessary?
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Attachments: HDFS-6565-002.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344400#comment-14344400 ] Akira AJISAKA commented on HDFS-6565: - The test failure looks unrelated to the patch. I ran the test locally and it passed. > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Attachments: HDFS-6565-002.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344353#comment-14344353
]
Hadoop QA commented on HDFS-6565:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12702006/HDFS-6565-002.patch
against trunk revision 5d0bae5.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9701//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9701//console
This message is automatically generated.
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Attachments: HDFS-6565-002.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14041608#comment-14041608
]
Hadoop QA commented on HDFS-6565:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12652077/HDFS-6565.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7217//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7217//console
This message is automatically generated.
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Attachments: HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14041409#comment-14041409
]
Akira AJISAKA commented on HDFS-6565:
-
Attaching a patch to remove jetty json library from JsonUtil and
WebHdfsFileSystem.
The way Jackson parse JSON number is different from jetty json:
* Jackson: number -> Integer, Long, or BigInteger (smallest applicable)
* jetty json: number -> Long
so I changed the code for parsing JSON number
{code}
(Long) m.get("blockId") // doesn't work if m.get("blockId") is Integer
{code}
to
{code}
((Number) m.get("blockId")).longValue() // support all classes extends Number
{code}
In addition, the way Jackson parse JSON array is different from jetty json:
* Jackson: array -> ArrayList
* jetty json: array -> Object[]
so I changed the code for parsing JSON array
{code}
(Object[]) m.get("locatedBlocks")
{code}
to
{code}
(List) m.get("locatedBlocks")
{code}
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Attachments: HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
