[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946894#comment-13946894
]
Tsz Wo Nicholas Sze commented on HDFS-6130:
---
... where you get 1.3.0?
It is the current
[branch-1|http://svn.apache.org/viewvc/hadoop/common/branches/branch-1/].
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
Attachments: fsimage.tar.gz
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13947034#comment-13947034
]
Tsz Wo Nicholas Sze commented on HDFS-6130:
---
I tried the new steps with current branch-1. Cannot reproduce the NPE.
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
Attachments: fsimage.tar.gz
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13947073#comment-13947073
]
Tsz Wo Nicholas Sze commented on HDFS-6130:
---
I tried it with 1.0.4
(http://archive.apache.org/dist/hadoop/core/hadoop-1.0.4/). Still cannot
reproduce the NPE.
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
Attachments: fsimage.tar.gz
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13947103#comment-13947103
]
Haohui Mai commented on HDFS-6130:
--
This is similar to HDFS-5988. The attached fsimage has image version -32, where
the local name optimization (version -38) was unavailable at that time.
HDFS-5988 has fixed the code path for fsimage that contains the local name
optimization, that is, fsimage that is newer than the version -38, but not for
the earlier fsimage. Due to the exact reason of HDFS-5988, the upgraded fsimage
will be corrupted.
One can create a file inside a directory in the old cluster, and then upgrade
to trunk to reproduce this bug.
I'll post a patch shortly.
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
Attachments: fsimage.tar.gz
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13947110#comment-13947110
]
Haohui Mai commented on HDFS-6130:
--
Raising the priority to blocker since it leads to data loss during upgrades.
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
Attachments: fsimage.tar.gz
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945628#comment-13945628
]
Tsz Wo Nicholas Sze commented on HDFS-6130:
---
Apache release also has this issue. Apache 1.0.4 upgrade to the trunk, you
can reproduce this issue.
Hi Fengdong, I just have tried it but cannot reproduce the NPE. There were a
log of changes since Apache 1.0.4. I was using 1.3.0 in my test. Could you
also try it?
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945642#comment-13945642
]
Tsz Wo Nicholas Sze commented on HDFS-6130:
---
I believe that this is a duplicate of HDFS-5988.
Hi [~wheat9], the stack trace posted here is indeed different from the one
posted in HDFS-6021 (a dup of HDFS-5988). So it seems that this is a different
issue. In this bug, FSImageFormatPBINode somehow passes a null inode to
FSDirectory. Could you take a look?
- Stack trace posted here
{noformat}
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
...
{noformat}
- Stack trace posted in HDFS-6021 (a dup of HDFS-5988)
{noformat}
2014-02-26 17:03:11,755 FATAL [main] namenode.NameNode
(NameNode.java:main(1351)) - Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:227)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:169)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:225)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:802)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:792)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:624)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:593)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:331)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:251)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:882)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:641)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:435)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:491)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.init(NameNode.java:647)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.init(NameNode.java:632)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1280)
...
{noformat}
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945653#comment-13945653
]
Haohui Mai commented on HDFS-6130:
--
It would be very helpful if the corresponding fsimage is available.
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945943#comment-13945943
]
Fengdong Yu commented on HDFS-6130:
---
Thanks [~szetszwo]!
[~wheat9], do you want only fsimage or both image and edit log? I'll reproduce
today using 1.3.0 and the latest trunk, then I'll keep the corresponding
fsimage and edit logs.
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945988#comment-13945988
]
Haohui Mai commented on HDFS-6130:
--
Can you create a checkpoint so that upgrading from the checkpointed fsimage
will triggered the bug?
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945992#comment-13945992
]
Fengdong Yu commented on HDFS-6130:
---
OK, no problem, I can using rollingUpgrade -prepare to create check point.
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946073#comment-13946073
]
Fengdong Yu commented on HDFS-6130:
---
hi [~szetszwo], where you get 1.3.0?
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946089#comment-13946089
]
Fengdong Yu commented on HDFS-6130:
---
please ignore my create check point method, that's wrong.
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946201#comment-13946201
]
Fengdong Yu commented on HDFS-6130:
---
[~wheat9],
fsimage was uploaded.
please read my following steps carefully before fix the bug.
1)There is no HA enabled during these steps.
2)all test files are all less than one block size
a. start hadoop-1.0.4 hdfs
b. put one files on the hdfs
c. stop hdfs.
d. start dfs with upgrade option to the lastest trunk
e. put more than ten files on the hdfs
f. stop hdfs
g. start hdfs (NPE here)
NOTE. if put a few files(such as one file) at step e, there is no NPE at step g.
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
Attachments: fsimage.tar.gz
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6130) NPE during namenode upgrade from old release
[
https://issues.apache.org/jira/browse/HDFS-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13943939#comment-13943939
]
Fengdong Yu commented on HDFS-6130:
---
update:
I miss a step between step 2 and step3. Add as step2.1, otherwise, all upgrades
succeed.
step2.1:
{code}
hdfs dfs -put test.data /
{code}
So, after upgrade from Apache1.x to the trunk, we MUST writer HDFS before ha
enabled in the next step.
I don't find any unit tests cover this scenrio.
NPE during namenode upgrade from old release
Key: HDFS-6130
URL: https://issues.apache.org/jira/browse/HDFS-6130
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.4.0
Reporter: Fengdong Yu
I want upgrade an old cluster(0.20.2-cdh3u1) to trunk instance,
I can upgrade successfully if I don't configurage HA, but if HA enabled,
there is NPE when I run ' hdfs namenode -initializeSharedEdits'
{code}
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache on namenode is
enabled
14/03/20 15:06:41 INFO namenode.FSNamesystem: Retry cache will use 0.03 of
total heap and retry cache entry expiry time is 60 millis
14/03/20 15:06:41 INFO util.GSet: Computing capacity for map
NameNodeRetryCache
14/03/20 15:06:41 INFO util.GSet: VM type = 64-bit
14/03/20 15:06:41 INFO util.GSet: 0.02999329447746% max memory 896 MB =
275.3 KB
14/03/20 15:06:41 INFO util.GSet: capacity = 2^15 = 32768 entries
14/03/20 15:06:41 INFO namenode.AclConfigFlag: ACLs enabled? false
14/03/20 15:06:41 INFO common.Storage: Lock on
/data/hadoop/data1/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO common.Storage: Lock on
/data/hadoop/data2/dfs/name/in_use.lock acquired by nodename
7326@10-150-170-176
14/03/20 15:06:42 INFO namenode.FSImage: No edit log streams selected.
14/03/20 15:06:42 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.
14/03/20 15:06:42 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.isReservedName(FSDirectory.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.addToParent(FSImageFormatPBINode.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeDirectorySection(FSImageFormatPBINode.java:162)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:168)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:120)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:881)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:704)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:642)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:653)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initializeSharedEdits(NameNode.java:912)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1276)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1360)
14/03/20 15:06:42 INFO util.ExitUtil: Exiting with status 1
14/03/20 15:06:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at 10-150-170-176/10.150.170.176
/
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
