[jira] [Updated] (HDFS-5710) FSDirectory#getFullPathName should check inodes against null

2014-01-13 Thread Jing Zhao (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jing Zhao updated HDFS-5710:


   Resolution: Fixed
Fix Version/s: 2.4.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Thanks to Ted for the report and Thank you Uma for the fix! I've committed this 
to trunk and branch-2.

> FSDirectory#getFullPathName should check inodes against null
> 
>
> Key: HDFS-5710
> URL: https://issues.apache.org/jira/browse/HDFS-5710
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Ted Yu
>Assignee: Uma Maheswara Rao G
> Fix For: 2.4.0
>
> Attachments: HDFS-5710.patch, hdfs-5710-output.html
>
>
> From 
> https://builds.apache.org/job/hbase-0.96-hadoop2/166/testReport/junit/org.apache.hadoop.hbase.mapreduce/TestTableInputFormatScan1/org_apache_hadoop_hbase_mapreduce_TestTableInputFormatScan1/
>  :
> {code}
> 2014-01-01 00:10:15,571 INFO  [IPC Server handler 2 on 50198] 
> blockmanagement.BlockManager(1009): BLOCK* addToInvalidates: 
> blk_1073741967_1143 127.0.0.1:40188 127.0.0.1:46149 127.0.0.1:41496 
> 2014-01-01 00:10:16,559 WARN  
> [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@93935b]
>  namenode.FSDirectory(1854): Could not get full path. Corresponding file 
> might have deleted already.
> 2014-01-01 00:10:16,560 FATAL 
> [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@93935b]
>  blockmanagement.BlockManager$ReplicationMonitor(3127): ReplicationMonitor 
> thread received Runtime exception. 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.getFullPathName(FSDirectory.java:1871)
>   at 
> org.apache.hadoop.hdfs.server.namenode.INode.getFullPathName(INode.java:482)
>   at 
> org.apache.hadoop.hdfs.server.namenode.INodeFile.getName(INodeFile.java:316)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy.chooseTarget(BlockPlacementPolicy.java:118)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWorkForBlocks(BlockManager.java:1259)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWork(BlockManager.java:1167)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeDatanodeWork(BlockManager.java:3158)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3112)
>   at java.lang.Thread.run(Thread.java:724)
> {code}
> Looks like getRelativePathINodes() returned null but getFullPathName() didn't 
> check inodes against null, leading to NPE.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HDFS-5710) FSDirectory#getFullPathName should check inodes against null

2014-01-07 Thread Jing Zhao (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jing Zhao updated HDFS-5710:


Status: Patch Available  (was: Open)

> FSDirectory#getFullPathName should check inodes against null
> 
>
> Key: HDFS-5710
> URL: https://issues.apache.org/jira/browse/HDFS-5710
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Ted Yu
>Assignee: Uma Maheswara Rao G
> Attachments: HDFS-5710.patch, hdfs-5710-output.html
>
>
> From 
> https://builds.apache.org/job/hbase-0.96-hadoop2/166/testReport/junit/org.apache.hadoop.hbase.mapreduce/TestTableInputFormatScan1/org_apache_hadoop_hbase_mapreduce_TestTableInputFormatScan1/
>  :
> {code}
> 2014-01-01 00:10:15,571 INFO  [IPC Server handler 2 on 50198] 
> blockmanagement.BlockManager(1009): BLOCK* addToInvalidates: 
> blk_1073741967_1143 127.0.0.1:40188 127.0.0.1:46149 127.0.0.1:41496 
> 2014-01-01 00:10:16,559 WARN  
> [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@93935b]
>  namenode.FSDirectory(1854): Could not get full path. Corresponding file 
> might have deleted already.
> 2014-01-01 00:10:16,560 FATAL 
> [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@93935b]
>  blockmanagement.BlockManager$ReplicationMonitor(3127): ReplicationMonitor 
> thread received Runtime exception. 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.getFullPathName(FSDirectory.java:1871)
>   at 
> org.apache.hadoop.hdfs.server.namenode.INode.getFullPathName(INode.java:482)
>   at 
> org.apache.hadoop.hdfs.server.namenode.INodeFile.getName(INodeFile.java:316)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy.chooseTarget(BlockPlacementPolicy.java:118)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWorkForBlocks(BlockManager.java:1259)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWork(BlockManager.java:1167)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeDatanodeWork(BlockManager.java:3158)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3112)
>   at java.lang.Thread.run(Thread.java:724)
> {code}
> Looks like getRelativePathINodes() returned null but getFullPathName() didn't 
> check inodes against null, leading to NPE.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HDFS-5710) FSDirectory#getFullPathName should check inodes against null

2014-01-07 Thread Uma Maheswara Rao G (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uma Maheswara Rao G updated HDFS-5710:
--

Attachment: HDFS-5710.patch

Just returning empty string in case if inodes become null when its called with 
out holding global lock. getFullPathName called many places. Instead of 
retuning null and checking evrywhere null, returning empty string may be ok.
Attached simple patch with the change.

> FSDirectory#getFullPathName should check inodes against null
> 
>
> Key: HDFS-5710
> URL: https://issues.apache.org/jira/browse/HDFS-5710
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Ted Yu
>Assignee: Uma Maheswara Rao G
> Attachments: HDFS-5710.patch, hdfs-5710-output.html
>
>
> From 
> https://builds.apache.org/job/hbase-0.96-hadoop2/166/testReport/junit/org.apache.hadoop.hbase.mapreduce/TestTableInputFormatScan1/org_apache_hadoop_hbase_mapreduce_TestTableInputFormatScan1/
>  :
> {code}
> 2014-01-01 00:10:15,571 INFO  [IPC Server handler 2 on 50198] 
> blockmanagement.BlockManager(1009): BLOCK* addToInvalidates: 
> blk_1073741967_1143 127.0.0.1:40188 127.0.0.1:46149 127.0.0.1:41496 
> 2014-01-01 00:10:16,559 WARN  
> [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@93935b]
>  namenode.FSDirectory(1854): Could not get full path. Corresponding file 
> might have deleted already.
> 2014-01-01 00:10:16,560 FATAL 
> [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@93935b]
>  blockmanagement.BlockManager$ReplicationMonitor(3127): ReplicationMonitor 
> thread received Runtime exception. 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.getFullPathName(FSDirectory.java:1871)
>   at 
> org.apache.hadoop.hdfs.server.namenode.INode.getFullPathName(INode.java:482)
>   at 
> org.apache.hadoop.hdfs.server.namenode.INodeFile.getName(INodeFile.java:316)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy.chooseTarget(BlockPlacementPolicy.java:118)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWorkForBlocks(BlockManager.java:1259)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWork(BlockManager.java:1167)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeDatanodeWork(BlockManager.java:3158)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3112)
>   at java.lang.Thread.run(Thread.java:724)
> {code}
> Looks like getRelativePathINodes() returned null but getFullPathName() didn't 
> check inodes against null, leading to NPE.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HDFS-5710) FSDirectory#getFullPathName should check inodes against null

2013-12-31 Thread Uma Maheswara Rao G (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uma Maheswara Rao G updated HDFS-5710:
--

Assignee: Uma Maheswara Rao G

> FSDirectory#getFullPathName should check inodes against null
> 
>
> Key: HDFS-5710
> URL: https://issues.apache.org/jira/browse/HDFS-5710
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Ted Yu
>Assignee: Uma Maheswara Rao G
> Attachments: hdfs-5710-output.html
>
>
> From 
> https://builds.apache.org/job/hbase-0.96-hadoop2/166/testReport/junit/org.apache.hadoop.hbase.mapreduce/TestTableInputFormatScan1/org_apache_hadoop_hbase_mapreduce_TestTableInputFormatScan1/
>  :
> {code}
> 2014-01-01 00:10:15,571 INFO  [IPC Server handler 2 on 50198] 
> blockmanagement.BlockManager(1009): BLOCK* addToInvalidates: 
> blk_1073741967_1143 127.0.0.1:40188 127.0.0.1:46149 127.0.0.1:41496 
> 2014-01-01 00:10:16,559 WARN  
> [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@93935b]
>  namenode.FSDirectory(1854): Could not get full path. Corresponding file 
> might have deleted already.
> 2014-01-01 00:10:16,560 FATAL 
> [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@93935b]
>  blockmanagement.BlockManager$ReplicationMonitor(3127): ReplicationMonitor 
> thread received Runtime exception. 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.getFullPathName(FSDirectory.java:1871)
>   at 
> org.apache.hadoop.hdfs.server.namenode.INode.getFullPathName(INode.java:482)
>   at 
> org.apache.hadoop.hdfs.server.namenode.INodeFile.getName(INodeFile.java:316)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy.chooseTarget(BlockPlacementPolicy.java:118)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWorkForBlocks(BlockManager.java:1259)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWork(BlockManager.java:1167)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeDatanodeWork(BlockManager.java:3158)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3112)
>   at java.lang.Thread.run(Thread.java:724)
> {code}
> Looks like getRelativePathINodes() returned null but getFullPathName() didn't 
> check inodes against null, leading to NPE.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (HDFS-5710) FSDirectory#getFullPathName should check inodes against null

2013-12-31 Thread Ted Yu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HDFS-5710:
-

Summary: FSDirectory#getFullPathName should check inodes against null  
(was: FSDirectory#getFullPathName should check for inodes against null)

> FSDirectory#getFullPathName should check inodes against null
> 
>
> Key: HDFS-5710
> URL: https://issues.apache.org/jira/browse/HDFS-5710
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Ted Yu
> Attachments: hdfs-5710-output.html
>
>
> From 
> https://builds.apache.org/job/hbase-0.96-hadoop2/166/testReport/junit/org.apache.hadoop.hbase.mapreduce/TestTableInputFormatScan1/org_apache_hadoop_hbase_mapreduce_TestTableInputFormatScan1/
>  :
> {code}
> 2014-01-01 00:10:15,571 INFO  [IPC Server handler 2 on 50198] 
> blockmanagement.BlockManager(1009): BLOCK* addToInvalidates: 
> blk_1073741967_1143 127.0.0.1:40188 127.0.0.1:46149 127.0.0.1:41496 
> 2014-01-01 00:10:16,559 WARN  
> [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@93935b]
>  namenode.FSDirectory(1854): Could not get full path. Corresponding file 
> might have deleted already.
> 2014-01-01 00:10:16,560 FATAL 
> [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@93935b]
>  blockmanagement.BlockManager$ReplicationMonitor(3127): ReplicationMonitor 
> thread received Runtime exception. 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.getFullPathName(FSDirectory.java:1871)
>   at 
> org.apache.hadoop.hdfs.server.namenode.INode.getFullPathName(INode.java:482)
>   at 
> org.apache.hadoop.hdfs.server.namenode.INodeFile.getName(INodeFile.java:316)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy.chooseTarget(BlockPlacementPolicy.java:118)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWorkForBlocks(BlockManager.java:1259)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWork(BlockManager.java:1167)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeDatanodeWork(BlockManager.java:3158)
>   at 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3112)
>   at java.lang.Thread.run(Thread.java:724)
> {code}
> Looks like getRelativePathINodes() returned null but getFullPathName() didn't 
> check inodes against null, leading to NPE.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)