[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[ https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13943939#comment-13943939 ] Fengdong Yu commented on HDFS-6130: --- update: I miss a step between step 2 and step3. Add as step2.1, otherwise, all upgrades succeed. step2.1: {code} hdfs dfs -put test.data / {code} So, after upgrade from Apache1.x to the trunk, we MUST writer HDFS before ha enabled in the next step. I don't find any unit tests cover this scenrio. NPE during namenode upgrade from old release Key: HDFS-6130 URL: https://issues.apache.org/jira/browse/HDFS-6130 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 2.4.0 Reporter: Fengdong Yu I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance, I can upgrade successfully if I don't configurage HA, but if HA enabled, there is NPE when I run ' hdfs namenode -initializeSharedEdits' {code} 14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is enabled 14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 60 millis 14/03/20 15:06:41 INFO util.GSet: Computing capacity for map NameNodeRetryCache 14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit 14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB = 275.3 KB 14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries 14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false 14/03/20 15:06:41 INFO common.Storage: Lock on /data/hadoop/data1/dfs/name/in_use.lock acquired by nodename 7326@10-150-170-176 14/03/20 15:06:42 INFO common.Storage: Lock on /data/hadoop/data2/dfs/name/in_use.lock acquired by nodename 7326@10-150-170-176 14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected. 14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes. 14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join java.lang.NullPointerException at org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984) at org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205) at org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162) at org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243) at org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168) at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653) at org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360) 14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1 14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG: / SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176 / {code} -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6120) Fix and improve safe mode log messages
[ https://issues.apache.org/jira/browse/HDFS-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944012#comment-13944012 ] Hudson commented on HDFS-6120: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #517 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/517/]) HDFS-6120. Fix and improve safe mode log messages. (Arpit Agarwal) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580047) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSafeMode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHASafeMode.java Fix and improve safe mode log messages -- Key: HDFS-6120 URL: https://issues.apache.org/jira/browse/HDFS-6120 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.3.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Fix For: 3.0.0, 2.4.0 Attachments: HDFS-6120.01.patch, HDFS-6120.02.patch, HDFS-6120.03.patch, HDFS-6120.04.patch, HDFS-6120.04.patch In HA mode the SBN can enter safe-mode extension and stay there even after the extension period has elapsed but continue to return the safemode message stating that The threshold has been reached and safe mode will be turned off soon. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6140) WebHDFS cannot create a file with spaces in the name after HA failover changes.
[ https://issues.apache.org/jira/browse/HDFS-6140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944014#comment-13944014 ] Hudson commented on HDFS-6140: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #517 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/517/]) HDFS-6140. WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580173) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java WebHDFS cannot create a file with spaces in the name after HA failover changes. --- Key: HDFS-6140 URL: https://issues.apache.org/jira/browse/HDFS-6140 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 3.0.0, 2.4.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 3.0.0, 2.4.0 Attachments: HDFS-6140.1.patch Attempting to create a file with spaces in the name through WebHDFS now fails with an {{IllegalArgumentException}} while attempting to construct a URI. This had been fixed earlier in HDFS-4944. The failure now is a regression introduced by HDFS-6100. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6138) User Guide for how to use viewfs with federation
[ https://issues.apache.org/jira/browse/HDFS-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944024#comment-13944024 ] Hudson commented on HDFS-6138: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #517 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/517/]) HDFS-6138. Add a user guide for how to use viewfs with federation. Contributed by sanjay and szetszwo (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580013) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ViewFs.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/resources/images/viewfs_TypicalMountTable.png * /hadoop/common/trunk/hadoop-project/src/site/site.xml User Guide for how to use viewfs with federation Key: HDFS-6138 URL: https://issues.apache.org/jira/browse/HDFS-6138 Project: Hadoop HDFS Issue Type: Improvement Components: documentation Affects Versions: 2.0.0-alpha Reporter: Sanjay Radia Assignee: Sanjay Radia Priority: Minor Fix For: 2.4.0 Attachments: Viewfs Guide.pdf, c7426_20111214.patch, c7426_20111215.patch, c7426_20111215b.patch, c7426_20111218.patch, c7426_20111220.patch, c7426_20111220_site.tar.gz, c7426_20140320.patch, c7426_20140320b.patch, c7426_20140321.patch, c7426_20140321b.patch, viewfs_TypicalMountTable.png -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-5138) Support HDFS upgrade in HA
[ https://issues.apache.org/jira/browse/HDFS-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944060#comment-13944060 ] Arun C Murthy commented on HDFS-5138: - [~jingzhao] / [~sureshms] - Should we close this one and mark HDFS-6135 as the blocker? Thanks. Support HDFS upgrade in HA -- Key: HDFS-5138 URL: https://issues.apache.org/jira/browse/HDFS-5138 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.1.1-beta Reporter: Kihwal Lee Assignee: Aaron T. Myers Priority: Blocker Fix For: 3.0.0 Attachments: HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, hdfs-5138-branch-2.txt With HA enabled, NN wo't start with -upgrade. Since there has been a layout version change between 2.0.x and 2.1.x, starting NN in upgrade mode was necessary when deploying 2.1.x to an existing 2.0.x cluster. But the only way to get around this was to disable HA and upgrade. The NN and the cluster cannot be flipped back to HA until the upgrade is finalized. If HA is disabled only on NN for layout upgrade and HA is turned back on without involving DNs, things will work, but finaliizeUpgrade won't work (the NN is in HA and it cannot be in upgrade mode) and DN's upgrade snapshots won't get removed. We will need a different ways of doing layout upgrade and upgrade snapshot. I am marking this as a 2.1.1-beta blocker based on feedback from others. If there is a reasonable workaround that does not increase maintenance window greatly, we can lower its priority from blocker to critical. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-5840) Follow-up to HDFS-5138 to improve error handling during partial upgrade failures
[ https://issues.apache.org/jira/browse/HDFS-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944061#comment-13944061 ] Arun C Murthy commented on HDFS-5840: - [~sureshms] / [~atm] - Should this be a blocker for 2.4? Thanks. Follow-up to HDFS-5138 to improve error handling during partial upgrade failures Key: HDFS-5840 URL: https://issues.apache.org/jira/browse/HDFS-5840 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Fix For: 3.0.0 Attachments: HDFS-5840.patch Suresh posted some good comment in HDFS-5138 after that patch had already been committed to trunk. This JIRA is to address those. See the first comment of this JIRA for the full content of the review. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6140) WebHDFS cannot create a file with spaces in the name after HA failover changes.
[ https://issues.apache.org/jira/browse/HDFS-6140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944067#comment-13944067 ] Hudson commented on HDFS-6140: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1709 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1709/]) HDFS-6140. WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580173) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java WebHDFS cannot create a file with spaces in the name after HA failover changes. --- Key: HDFS-6140 URL: https://issues.apache.org/jira/browse/HDFS-6140 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 3.0.0, 2.4.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 3.0.0, 2.4.0 Attachments: HDFS-6140.1.patch Attempting to create a file with spaces in the name through WebHDFS now fails with an {{IllegalArgumentException}} while attempting to construct a URI. This had been fixed earlier in HDFS-4944. The failure now is a regression introduced by HDFS-6100. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6120) Fix and improve safe mode log messages
[ https://issues.apache.org/jira/browse/HDFS-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944065#comment-13944065 ] Hudson commented on HDFS-6120: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1709 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1709/]) HDFS-6120. Fix and improve safe mode log messages. (Arpit Agarwal) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580047) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSafeMode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHASafeMode.java Fix and improve safe mode log messages -- Key: HDFS-6120 URL: https://issues.apache.org/jira/browse/HDFS-6120 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.3.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Fix For: 3.0.0, 2.4.0 Attachments: HDFS-6120.01.patch, HDFS-6120.02.patch, HDFS-6120.03.patch, HDFS-6120.04.patch, HDFS-6120.04.patch In HA mode the SBN can enter safe-mode extension and stay there even after the extension period has elapsed but continue to return the safemode message stating that The threshold has been reached and safe mode will be turned off soon. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6138) User Guide for how to use viewfs with federation
[ https://issues.apache.org/jira/browse/HDFS-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944077#comment-13944077 ] Hudson commented on HDFS-6138: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1709 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1709/]) HDFS-6138. Add a user guide for how to use viewfs with federation. Contributed by sanjay and szetszwo (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580013) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ViewFs.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/resources/images/viewfs_TypicalMountTable.png * /hadoop/common/trunk/hadoop-project/src/site/site.xml User Guide for how to use viewfs with federation Key: HDFS-6138 URL: https://issues.apache.org/jira/browse/HDFS-6138 Project: Hadoop HDFS Issue Type: Improvement Components: documentation Affects Versions: 2.0.0-alpha Reporter: Sanjay Radia Assignee: Sanjay Radia Priority: Minor Fix For: 2.4.0 Attachments: Viewfs Guide.pdf, c7426_20111214.patch, c7426_20111215.patch, c7426_20111215b.patch, c7426_20111218.patch, c7426_20111220.patch, c7426_20111220_site.tar.gz, c7426_20140320.patch, c7426_20140320b.patch, c7426_20140321.patch, c7426_20140321b.patch, viewfs_TypicalMountTable.png -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6120) Fix and improve safe mode log messages
[ https://issues.apache.org/jira/browse/HDFS-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944102#comment-13944102 ] Hudson commented on HDFS-6120: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1734 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1734/]) HDFS-6120. Fix and improve safe mode log messages. (Arpit Agarwal) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580047) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSafeMode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHASafeMode.java Fix and improve safe mode log messages -- Key: HDFS-6120 URL: https://issues.apache.org/jira/browse/HDFS-6120 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.3.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Fix For: 3.0.0, 2.4.0 Attachments: HDFS-6120.01.patch, HDFS-6120.02.patch, HDFS-6120.03.patch, HDFS-6120.04.patch, HDFS-6120.04.patch In HA mode the SBN can enter safe-mode extension and stay there even after the extension period has elapsed but continue to return the safemode message stating that The threshold has been reached and safe mode will be turned off soon. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6140) WebHDFS cannot create a file with spaces in the name after HA failover changes.
[ https://issues.apache.org/jira/browse/HDFS-6140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944104#comment-13944104 ] Hudson commented on HDFS-6140: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1734 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1734/]) HDFS-6140. WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580173) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java WebHDFS cannot create a file with spaces in the name after HA failover changes. --- Key: HDFS-6140 URL: https://issues.apache.org/jira/browse/HDFS-6140 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 3.0.0, 2.4.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 3.0.0, 2.4.0 Attachments: HDFS-6140.1.patch Attempting to create a file with spaces in the name through WebHDFS now fails with an {{IllegalArgumentException}} while attempting to construct a URI. This had been fixed earlier in HDFS-4944. The failure now is a regression introduced by HDFS-6100. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6138) User Guide for how to use viewfs with federation
[ https://issues.apache.org/jira/browse/HDFS-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944114#comment-13944114 ] Hudson commented on HDFS-6138: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1734 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1734/]) HDFS-6138. Add a user guide for how to use viewfs with federation. Contributed by sanjay and szetszwo (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580013) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ViewFs.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/resources/images/viewfs_TypicalMountTable.png * /hadoop/common/trunk/hadoop-project/src/site/site.xml User Guide for how to use viewfs with federation Key: HDFS-6138 URL: https://issues.apache.org/jira/browse/HDFS-6138 Project: Hadoop HDFS Issue Type: Improvement Components: documentation Affects Versions: 2.0.0-alpha Reporter: Sanjay Radia Assignee: Sanjay Radia Priority: Minor Fix For: 2.4.0 Attachments: Viewfs Guide.pdf, c7426_20111214.patch, c7426_20111215.patch, c7426_20111215b.patch, c7426_20111218.patch, c7426_20111220.patch, c7426_20111220_site.tar.gz, c7426_20140320.patch, c7426_20140320b.patch, c7426_20140321.patch, c7426_20140321b.patch, viewfs_TypicalMountTable.png -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HDFS-6142) StandbyException wrapped to InvalidToken exception
Ding Yuan created HDFS-6142: --- Summary: StandbyException wrapped to InvalidToken exception Key: HDFS-6142 URL: https://issues.apache.org/jira/browse/HDFS-6142 Project: Hadoop HDFS Issue Type: Bug Components: security Affects Versions: 2.2.0 Reporter: Ding Yuan The following code in org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSecretManager.java: {noformat} public byte[] retrievePassword( DelegationTokenIdentifier identifier) throws InvalidToken { try { // this check introduces inconsistency in the authentication to a // HA standby NN. non-token auths are allowed into the namespace which // decides whether to throw a StandbyException. tokens are a bit // different in that a standby may be behind and thus not yet know // of all tokens issued by the active NN. the following check does // not allow ANY token auth, however it should allow known tokens in namesystem.checkOperation(OperationCategory.READ); } catch (StandbyException se) { // FIXME: this is a hack to get around changing method signatures by // tunneling a non-InvalidToken exception as the cause which the // RPC server will unwrap before returning to the client InvalidToken wrappedStandby = new InvalidToken(StandbyException); wrappedStandby.initCause(se); throw wrappedStandby; } return super.retrievePassword(identifier); } {noformat} A StandbyException from namesystem.checkOperation is wrapped to InvalidToken exception. The comment suggests that the RPC server will unwrap it to StandbyException before sending back to the client, but this may not be the case for every code path. For example, in datanode's DataXceiver.copyBlock, it will call checkAccess which eventually might call retrievePassword, but when copyBlock catches an InvalidToken exception, it would simply send to the client that exception without unwrapping it. I am not exactly sure about the possible consequence, but it seems client treats StandbyException (which is perhaps much more serious) very different from InvalidToken exception. -- This message was sent by Atlassian JIRA (v6.2#6252)