[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13943939#comment-13943939
]
Fengdong Yu commented on HDFS-6130:
---
update:
I miss a step between step 2 and step3. Add as step2.1, otherwise, all upgrades
succeed.
step2.1:
{code}
hdfs dfs -put test.data /
{code}
So, after upgrade from Apache1.x to the trunk, we MUST writer HDFS before ha
enabled in the next step.
I don't find any unit tests cover this scenrio.
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6120) Fix and improve safe mode log messages
[ https://issues.apache.org/jira/browse/HDFS-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944012#comment-13944012 ] Hudson commented on HDFS-6120: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #517 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/517/]) HDFS-6120. Fix and improve safe mode log messages. (Arpit Agarwal) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580047) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSafeMode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHASafeMode.java Fix and improve safe mode log messages -- Key: HDFS-6120 URL: https://issues.apache.org/jira/browse/HDFS-6120 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.3.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Fix For: 3.0.0, 2.4.0 Attachments: HDFS-6120.01.patch, HDFS-6120.02.patch, HDFS-6120.03.patch, HDFS-6120.04.patch, HDFS-6120.04.patch In HA mode the SBN can enter safe-mode extension and stay there even after the extension period has elapsed but continue to return the safemode message stating that The threshold has been reached and safe mode will be turned off soon. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6140) WebHDFS cannot create a file with spaces in the name after HA failover changes.
[
https://issues.apache.org/jira/browse/HDFS-6140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944014#comment-13944014
]
Hudson commented on HDFS-6140:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk #517 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/517/])
HDFS-6140. WebHDFS cannot create a file with spaces in the name after HA
failover changes. Contributed by Chris Nauroth. (cnauroth:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580173)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java
WebHDFS cannot create a file with spaces in the name after HA failover
changes.
---
Key: HDFS-6140
URL: https://issues.apache.org/jira/browse/HDFS-6140
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 3.0.0, 2.4.0
Reporter: Chris Nauroth
Assignee: Chris Nauroth
Fix For: 3.0.0, 2.4.0
Attachments: HDFS-6140.1.patch
Attempting to create a file with spaces in the name through WebHDFS now fails
with an {{IllegalArgumentException}} while attempting to construct a URI.
This had been fixed earlier in HDFS-4944. The failure now is a regression
introduced by HDFS-6100.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6138) User Guide for how to use viewfs with federation
[ https://issues.apache.org/jira/browse/HDFS-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944024#comment-13944024 ] Hudson commented on HDFS-6138: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #517 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/517/]) HDFS-6138. Add a user guide for how to use viewfs with federation. Contributed by sanjay and szetszwo (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580013) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ViewFs.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/resources/images/viewfs_TypicalMountTable.png * /hadoop/common/trunk/hadoop-project/src/site/site.xml User Guide for how to use viewfs with federation Key: HDFS-6138 URL: https://issues.apache.org/jira/browse/HDFS-6138 Project: Hadoop HDFS Issue Type: Improvement Components: documentation Affects Versions: 2.0.0-alpha Reporter: Sanjay Radia Assignee: Sanjay Radia Priority: Minor Fix For: 2.4.0 Attachments: Viewfs Guide.pdf, c7426_20111214.patch, c7426_20111215.patch, c7426_20111215b.patch, c7426_20111218.patch, c7426_20111220.patch, c7426_20111220_site.tar.gz, c7426_20140320.patch, c7426_20140320b.patch, c7426_20140321.patch, c7426_20140321b.patch, viewfs_TypicalMountTable.png -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-5138) Support HDFS upgrade in HA
[ https://issues.apache.org/jira/browse/HDFS-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944060#comment-13944060 ] Arun C Murthy commented on HDFS-5138: - [~jingzhao] / [~sureshms] - Should we close this one and mark HDFS-6135 as the blocker? Thanks. Support HDFS upgrade in HA -- Key: HDFS-5138 URL: https://issues.apache.org/jira/browse/HDFS-5138 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.1.1-beta Reporter: Kihwal Lee Assignee: Aaron T. Myers Priority: Blocker Fix For: 3.0.0 Attachments: HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, hdfs-5138-branch-2.txt With HA enabled, NN wo't start with -upgrade. Since there has been a layout version change between 2.0.x and 2.1.x, starting NN in upgrade mode was necessary when deploying 2.1.x to an existing 2.0.x cluster. But the only way to get around this was to disable HA and upgrade. The NN and the cluster cannot be flipped back to HA until the upgrade is finalized. If HA is disabled only on NN for layout upgrade and HA is turned back on without involving DNs, things will work, but finaliizeUpgrade won't work (the NN is in HA and it cannot be in upgrade mode) and DN's upgrade snapshots won't get removed. We will need a different ways of doing layout upgrade and upgrade snapshot. I am marking this as a 2.1.1-beta blocker based on feedback from others. If there is a reasonable workaround that does not increase maintenance window greatly, we can lower its priority from blocker to critical. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-5840) Follow-up to HDFS-5138 to improve error handling during partial upgrade failures
[ https://issues.apache.org/jira/browse/HDFS-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944061#comment-13944061 ] Arun C Murthy commented on HDFS-5840: - [~sureshms] / [~atm] - Should this be a blocker for 2.4? Thanks. Follow-up to HDFS-5138 to improve error handling during partial upgrade failures Key: HDFS-5840 URL: https://issues.apache.org/jira/browse/HDFS-5840 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 3.0.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Fix For: 3.0.0 Attachments: HDFS-5840.patch Suresh posted some good comment in HDFS-5138 after that patch had already been committed to trunk. This JIRA is to address those. See the first comment of this JIRA for the full content of the review. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6140) WebHDFS cannot create a file with spaces in the name after HA failover changes.
[
https://issues.apache.org/jira/browse/HDFS-6140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944067#comment-13944067
]
Hudson commented on HDFS-6140:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1709 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1709/])
HDFS-6140. WebHDFS cannot create a file with spaces in the name after HA
failover changes. Contributed by Chris Nauroth. (cnauroth:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580173)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java
WebHDFS cannot create a file with spaces in the name after HA failover
changes.
---
Key: HDFS-6140
URL: https://issues.apache.org/jira/browse/HDFS-6140
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 3.0.0, 2.4.0
Reporter: Chris Nauroth
Assignee: Chris Nauroth
Fix For: 3.0.0, 2.4.0
Attachments: HDFS-6140.1.patch
Attempting to create a file with spaces in the name through WebHDFS now fails
with an {{IllegalArgumentException}} while attempting to construct a URI.
This had been fixed earlier in HDFS-4944. The failure now is a regression
introduced by HDFS-6100.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6120) Fix and improve safe mode log messages
[ https://issues.apache.org/jira/browse/HDFS-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944065#comment-13944065 ] Hudson commented on HDFS-6120: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1709 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1709/]) HDFS-6120. Fix and improve safe mode log messages. (Arpit Agarwal) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580047) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSafeMode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHASafeMode.java Fix and improve safe mode log messages -- Key: HDFS-6120 URL: https://issues.apache.org/jira/browse/HDFS-6120 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.3.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Fix For: 3.0.0, 2.4.0 Attachments: HDFS-6120.01.patch, HDFS-6120.02.patch, HDFS-6120.03.patch, HDFS-6120.04.patch, HDFS-6120.04.patch In HA mode the SBN can enter safe-mode extension and stay there even after the extension period has elapsed but continue to return the safemode message stating that The threshold has been reached and safe mode will be turned off soon. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6138) User Guide for how to use viewfs with federation
[ https://issues.apache.org/jira/browse/HDFS-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944077#comment-13944077 ] Hudson commented on HDFS-6138: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1709 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1709/]) HDFS-6138. Add a user guide for how to use viewfs with federation. Contributed by sanjay and szetszwo (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580013) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ViewFs.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/resources/images/viewfs_TypicalMountTable.png * /hadoop/common/trunk/hadoop-project/src/site/site.xml User Guide for how to use viewfs with federation Key: HDFS-6138 URL: https://issues.apache.org/jira/browse/HDFS-6138 Project: Hadoop HDFS Issue Type: Improvement Components: documentation Affects Versions: 2.0.0-alpha Reporter: Sanjay Radia Assignee: Sanjay Radia Priority: Minor Fix For: 2.4.0 Attachments: Viewfs Guide.pdf, c7426_20111214.patch, c7426_20111215.patch, c7426_20111215b.patch, c7426_20111218.patch, c7426_20111220.patch, c7426_20111220_site.tar.gz, c7426_20140320.patch, c7426_20140320b.patch, c7426_20140321.patch, c7426_20140321b.patch, viewfs_TypicalMountTable.png -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6120) Fix and improve safe mode log messages
[ https://issues.apache.org/jira/browse/HDFS-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944102#comment-13944102 ] Hudson commented on HDFS-6120: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1734 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1734/]) HDFS-6120. Fix and improve safe mode log messages. (Arpit Agarwal) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580047) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSafeMode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHASafeMode.java Fix and improve safe mode log messages -- Key: HDFS-6120 URL: https://issues.apache.org/jira/browse/HDFS-6120 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.3.0 Reporter: Arpit Agarwal Assignee: Arpit Agarwal Fix For: 3.0.0, 2.4.0 Attachments: HDFS-6120.01.patch, HDFS-6120.02.patch, HDFS-6120.03.patch, HDFS-6120.04.patch, HDFS-6120.04.patch In HA mode the SBN can enter safe-mode extension and stay there even after the extension period has elapsed but continue to return the safemode message stating that The threshold has been reached and safe mode will be turned off soon. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6140) WebHDFS cannot create a file with spaces in the name after HA failover changes.
[
https://issues.apache.org/jira/browse/HDFS-6140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944104#comment-13944104
]
Hudson commented on HDFS-6140:
--
SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1734 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1734/])
HDFS-6140. WebHDFS cannot create a file with spaces in the name after HA
failover changes. Contributed by Chris Nauroth. (cnauroth:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580173)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java
WebHDFS cannot create a file with spaces in the name after HA failover
changes.
---
Key: HDFS-6140
URL: https://issues.apache.org/jira/browse/HDFS-6140
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 3.0.0, 2.4.0
Reporter: Chris Nauroth
Assignee: Chris Nauroth
Fix For: 3.0.0, 2.4.0
Attachments: HDFS-6140.1.patch
Attempting to create a file with spaces in the name through WebHDFS now fails
with an {{IllegalArgumentException}} while attempting to construct a URI.
This had been fixed earlier in HDFS-4944. The failure now is a regression
introduced by HDFS-6100.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6138) User Guide for how to use viewfs with federation
[ https://issues.apache.org/jira/browse/HDFS-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944114#comment-13944114 ] Hudson commented on HDFS-6138: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1734 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1734/]) HDFS-6138. Add a user guide for how to use viewfs with federation. Contributed by sanjay and szetszwo (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1580013) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ViewFs.apt.vm * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/resources/images/viewfs_TypicalMountTable.png * /hadoop/common/trunk/hadoop-project/src/site/site.xml User Guide for how to use viewfs with federation Key: HDFS-6138 URL: https://issues.apache.org/jira/browse/HDFS-6138 Project: Hadoop HDFS Issue Type: Improvement Components: documentation Affects Versions: 2.0.0-alpha Reporter: Sanjay Radia Assignee: Sanjay Radia Priority: Minor Fix For: 2.4.0 Attachments: Viewfs Guide.pdf, c7426_20111214.patch, c7426_20111215.patch, c7426_20111215b.patch, c7426_20111218.patch, c7426_20111220.patch, c7426_20111220_site.tar.gz, c7426_20140320.patch, c7426_20140320b.patch, c7426_20140321.patch, c7426_20140321b.patch, viewfs_TypicalMountTable.png -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HDFS-6142) StandbyException wrapped to InvalidToken exception
Ding Yuan created HDFS-6142:
---
Summary: StandbyException wrapped to InvalidToken exception
Key: HDFS-6142
URL: https://issues.apache.org/jira/browse/HDFS-6142
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 2.2.0
Reporter: Ding Yuan
The following code in
org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSecretManager.java:
{noformat}
public byte[] retrievePassword(
DelegationTokenIdentifier identifier) throws InvalidToken {
try {
// this check introduces inconsistency in the authentication to a
// HA standby NN. non-token auths are allowed into the namespace which
// decides whether to throw a StandbyException. tokens are a bit
// different in that a standby may be behind and thus not yet know
// of all tokens issued by the active NN. the following check does
// not allow ANY token auth, however it should allow known tokens in
namesystem.checkOperation(OperationCategory.READ);
} catch (StandbyException se) {
// FIXME: this is a hack to get around changing method signatures by
// tunneling a non-InvalidToken exception as the cause which the
// RPC server will unwrap before returning to the client
InvalidToken wrappedStandby = new InvalidToken(StandbyException);
wrappedStandby.initCause(se);
throw wrappedStandby;
}
return super.retrievePassword(identifier);
}
{noformat}
A StandbyException from namesystem.checkOperation is wrapped to InvalidToken
exception. The comment suggests that the RPC server will unwrap it to
StandbyException before sending back to the client, but this may not be the
case for every code path. For example, in datanode's DataXceiver.copyBlock, it
will call checkAccess which eventually might call retrievePassword, but when
copyBlock catches an InvalidToken exception, it would simply send to the client
that exception without unwrapping it.
I am not exactly sure about the possible consequence, but it seems client
treats StandbyException (which is perhaps much more serious) very different
from InvalidToken exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
