[
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773172#comment-16773172
]
Surendra Singh Lilhore commented on HDFS-14216:
-----------------------------------------------
Anyway client is trying to exclude DN, if it is not found that also fine, no
need to throw any exception.
{quote} I wonder how this is handled on the RPC side?
{quote}
If client is doing any operation related to DN, then it should get the latest
datanode report from namenode periodically.
One more thing, asserting NullPointerException in UT is not good idea. UT
should verify the functional failure case.
> NullPointerException happens in NamenodeWebHdfs
> -----------------------------------------------
>
> Key: HDFS-14216
> URL: https://issues.apache.org/jira/browse/HDFS-14216
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: lujie
> Assignee: lujie
> Priority: Critical
> Attachments: HDFS-14216_1.patch, HDFS-14216_2.patch,
> HDFS-14216_3.patch, HDFS-14216_4.patch, hadoop-hires-namenode-hadoop11.log
>
>
> workload
> {code:java}
> curl -i -X PUT -T $HOMEPARH/test.txt
> "http://hadoop1:9870/webhdfs/v1/input?op=CREATE&excludedatanodes=hadoop2";
> {code}
> the method
> {code:java}
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
> excludeDatanodes){
> HashSet<Node> excludes = new HashSet<Node>();
> if (excludeDatanodes != null) {
> for (String host : StringUtils
> .getTrimmedStringCollection(excludeDatanodes)) {
> int idx = host.indexOf(":");
> if (idx != -1) {
> excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
> host.substring(0, idx), Integer.parseInt(host.substring(idx +
> 1))));
> } else {
>
> excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
> }
> }
> }
> }
> {code}
> when datanode(e.g.hadoop2) is {color:#d04437}just wiped before
> line280{color}, or{color:#333333}
> {color}{color:#ff0000}we{color}{color:#ff0000} give the wrong DN
> name{color}*,*then bm.getDatanodeManager().getDatanodeByHost(host) will
> return null, *_excludes_* *containes null*. while *_excludes_* are used
> later, NPE happens:
> {code:java}
> java.lang.NullPointerException
> at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
> at
> org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
> at
> org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
> at
> org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
> at
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
> at
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
> at
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
> at
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
> at
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
> at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
> at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
