[jira] [Commented] (HDFS-6305) WebHdfs response decoding may throw RuntimeExceptions
[
https://issues.apache.org/jira/browse/HDFS-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16280707#comment-16280707
]
Rushabh S Shah commented on HDFS-6305:
--
I think we dropped {{TestWebHdfsBadResponses}} class while committing the patch.
> WebHdfs response decoding may throw RuntimeExceptions
> -
>
> Key: HDFS-6305
> URL: https://issues.apache.org/jira/browse/HDFS-6305
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 2.0.0-alpha, 3.0.0-alpha1
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Fix For: 2.5.0
>
> Attachments: HDFS-6305.patch
>
>
> WebHdfs does not guard against exceptions while decoding the response
> payload. The json parser will throw RunTime exceptions on malformed
> responses. The json decoding routines do not validate the expected fields
> are present which may cause NPEs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-6305) WebHdfs response decoding may throw RuntimeExceptions
[ https://issues.apache.org/jira/browse/HDFS-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13999026#comment-13999026 ] Hudson commented on HDFS-6305: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5605 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5605/]) HDFS-6305. WebHdfs response decoding may throw RuntimeExceptions (Daryn Sharp via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1594273) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/HttpOpParam.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/WebHdfsTestUtil.java > WebHdfs response decoding may throw RuntimeExceptions > - > > Key: HDFS-6305 > URL: https://issues.apache.org/jira/browse/HDFS-6305 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Fix For: 3.0.0, 2.5.0 > > Attachments: HDFS-6305.patch > > > WebHdfs does not guard against exceptions while decoding the response > payload. The json parser will throw RunTime exceptions on malformed > responses. The json decoding routines do not validate the expected fields > are present which may cause NPEs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6305) WebHdfs response decoding may throw RuntimeExceptions
[ https://issues.apache.org/jira/browse/HDFS-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13998236#comment-13998236 ] Hudson commented on HDFS-6305: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1753 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1753/]) HDFS-6305. WebHdfs response decoding may throw RuntimeExceptions (Daryn Sharp via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1594273) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/HttpOpParam.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/WebHdfsTestUtil.java > WebHdfs response decoding may throw RuntimeExceptions > - > > Key: HDFS-6305 > URL: https://issues.apache.org/jira/browse/HDFS-6305 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Fix For: 3.0.0, 2.5.0 > > Attachments: HDFS-6305.patch > > > WebHdfs does not guard against exceptions while decoding the response > payload. The json parser will throw RunTime exceptions on malformed > responses. The json decoding routines do not validate the expected fields > are present which may cause NPEs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6305) WebHdfs response decoding may throw RuntimeExceptions
[ https://issues.apache.org/jira/browse/HDFS-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13998198#comment-13998198 ] Hudson commented on HDFS-6305: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1779 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1779/]) HDFS-6305. WebHdfs response decoding may throw RuntimeExceptions (Daryn Sharp via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1594273) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/HttpOpParam.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/WebHdfsTestUtil.java > WebHdfs response decoding may throw RuntimeExceptions > - > > Key: HDFS-6305 > URL: https://issues.apache.org/jira/browse/HDFS-6305 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Fix For: 3.0.0, 2.5.0 > > Attachments: HDFS-6305.patch > > > WebHdfs does not guard against exceptions while decoding the response > payload. The json parser will throw RunTime exceptions on malformed > responses. The json decoding routines do not validate the expected fields > are present which may cause NPEs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6305) WebHdfs response decoding may throw RuntimeExceptions
[ https://issues.apache.org/jira/browse/HDFS-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995737#comment-13995737 ] Jonathan Eagles commented on HDFS-6305: --- Kicked jenkins. there are 3 or 4 builds ahead of this one. It may be late tonight to see the results. Jon > WebHdfs response decoding may throw RuntimeExceptions > - > > Key: HDFS-6305 > URL: https://issues.apache.org/jira/browse/HDFS-6305 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HDFS-6305.patch > > > WebHdfs does not guard against exceptions while decoding the response > payload. The json parser will throw RunTime exceptions on malformed > responses. The json decoding routines do not validate the expected fields > are present which may cause NPEs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6305) WebHdfs response decoding may throw RuntimeExceptions
[
https://issues.apache.org/jira/browse/HDFS-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996160#comment-13996160
]
Hadoop QA commented on HDFS-6305:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12642531/HDFS-6305.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6889//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6889//console
This message is automatically generated.
> WebHdfs response decoding may throw RuntimeExceptions
> -
>
> Key: HDFS-6305
> URL: https://issues.apache.org/jira/browse/HDFS-6305
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HDFS-6305.patch
>
>
> WebHdfs does not guard against exceptions while decoding the response
> payload. The json parser will throw RunTime exceptions on malformed
> responses. The json decoding routines do not validate the expected fields
> are present which may cause NPEs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6305) WebHdfs response decoding may throw RuntimeExceptions
[
https://issues.apache.org/jira/browse/HDFS-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990848#comment-13990848
]
Daryn Sharp commented on HDFS-6305:
---
Rather than change all of the decoding methods (yet), I went the quick route of
converting any non-IOException during decoding to IOExceptions.
{code}
+ catch (Exception e) { // catch json parser errors
+final IOException ioe =
+new IOException("Response decoding failure: "+e.toString(), e);
{code}
This also handles the case where the json parser itself throws runtime
exceptions if the json is incomplete or malformed as stressed in the test cases.
> WebHdfs response decoding may throw RuntimeExceptions
> -
>
> Key: HDFS-6305
> URL: https://issues.apache.org/jira/browse/HDFS-6305
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HDFS-6305.patch
>
>
> WebHdfs does not guard against exceptions while decoding the response
> payload. The json parser will throw RunTime exceptions on malformed
> responses. The json decoding routines do not validate the expected fields
> are present which may cause NPEs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6305) WebHdfs response decoding may throw RuntimeExceptions
[ https://issues.apache.org/jira/browse/HDFS-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988180#comment-13988180 ] Chris Nauroth commented on HDFS-6305: - Nice work, Daryn. Thank you for adding the failure tests too. I have just one question. bq. The json decoding routines do not validate the expected fields are present which may cause NPEs. It wasn't clear to me if this patch is really doing anything to address this part. I don't see addition of explicit validation checks, and it also doesn't look like an NPE would be handled any differently. Can you please clarify? Thanks! > WebHdfs response decoding may throw RuntimeExceptions > - > > Key: HDFS-6305 > URL: https://issues.apache.org/jira/browse/HDFS-6305 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HDFS-6305.patch > > > WebHdfs does not guard against exceptions while decoding the response > payload. The json parser will throw RunTime exceptions on malformed > responses. The json decoding routines do not validate the expected fields > are present which may cause NPEs. -- This message was sent by Atlassian JIRA (v6.2#6252)
