[
https://issues.apache.org/jira/browse/HDFS-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122697#comment-14122697
]
sam liu commented on HDFS-7002:
-------------------------------
Failed to rolling upgrade hdfs from 2.2.0 to 2.4.1, according to
http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html#Upgrading_Non-Federated_Clusters.
[A] Cluster Info:
Name Nodes: host2(active), host3(standby)
QJM Nodes: host1, host4, host5
Data Nodes: host4, host5
[B] Issue Reproduce Steps:
1. stop standy namenode on host3
2. replace hadoop-2.2.0 binary with hadoop-2.4.1 binary
3. failed to execute command 'hdfs namenode -initializeSharedEdits' with
following exception, accordint to last comment of HDFS-5550.
14/09/05 16:31:49 INFO namenode.AclConfigFlag: ACLs enabled? false
14/09/05 16:31:49 INFO common.Storage: Lock on /hadoop/hdfs/name/in_use.lock
acquired by nodename 155436@host3
14/09/05 16:31:49 WARN namenode.FSNamesystem: Encountered exception loading
fsimage
java.io.IOException:
File system image contains an old layout version -47.
An upgrade to version -56 is required.
Please restart NameNode with the "-rollingUpgrade started" option if a rolling
upgraded is already started; or restart NameNode with the "-upgrade" option to
start a new upgrade.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:224)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:891)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:638)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:980)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1340)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)
14/09/05 16:31:49 ERROR namenode.NameNode: Could not initialize shared edits dir
4. failed to execute command 'hdfs namenode -rollingUpgrade started'
14/09/05 16:32:22 INFO namenode.EditLogInputStream: Fast-forwarding stream
'http://host4:8480/getJournal?jid=BICluster&segmentTxId=2570&storageInfo=-56%3A1686370894%3A0%3ACID-70b084bc-5e08-402e-b2e5-f4461d7dc69c'
to transaction ID 2570
14/09/05 16:32:22 ERROR namenode.EditLogInputStream: caught exception
initializing
http://host4:8480/getJournal?jid=BICluster&segmentTxId=2570&storageInfo=-56%3A1686370894%3A0%3ACID-70b084bc-5e08-402e-b2e5-f4461d7dc69c
org.apache.hadoop.hdfs.server.namenode.TransferFsImage$HttpGetFailedException:
Fetch of
http://host4:8480/getJournal?jid=BICluster&segmentTxId=2570&storageInfo=-56%3A1686370894%3A0%3ACID-70b084bc-5e08-402e-b2e5-f4461d7dc69c
failed with status code 403
Response message:
This node has storage info
'-47:1686370894:0:CID-70b084bc-5e08-402e-b2e5-f4461d7dc69c' but the requesting
node expected '-56:1686370894:0:CID-70b084bc-5e08-402e-b2e5-f4461d7dc69c'
at
org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:472)
at
org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:460)
at
java.security.AccessController.doPrivileged(AccessController.java:366)
at javax.security.auth.Subject.doAs(Subject.java:572)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1558)
at
org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:448)
at
org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:442)
at
org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java:459)
at
org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:145)
at
org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOpImpl(EditLogFileInputStream.java:196)
at
org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOp(EditLogFileInputStream.java:254)
at
org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
at
org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
at
org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
at
org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
at
org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
at
org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
at
org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:180)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:133)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:805)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:665)
> Failed to rolling upgrade hdfs from 2.2.0 to 2.4.1
> --------------------------------------------------
>
> Key: HDFS-7002
> URL: https://issues.apache.org/jira/browse/HDFS-7002
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: journal-node, namenode, qjm
> Affects Versions: 2.2.0, 2.4.1
> Reporter: sam liu
> Priority: Blocker
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
