[jira] [Commented] (HDFS-6696) Name node cannot start if the path of a file under construction contains .snapshot
[
https://issues.apache.org/jira/browse/HDFS-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073603#comment-14073603
]
Jing Zhao commented on HDFS-6696:
-
The patch looks good to me. The new unit tests also pass in my local machine
after applying the binary changes. +1 pending Jenkins.
Name node cannot start if the path of a file under construction contains
.snapshot
Key: HDFS-6696
URL: https://issues.apache.org/jira/browse/HDFS-6696
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Andrew Wang
Priority: Blocker
Attachments: hdfs-6696.001.patch, hdfs-6696.002.patch,
hdfs-6696.003.patch
Using {{-renameReserved}} to rename .snapshot in a pre-hdfs-snapshot
feature fsimage during upgrade only works, if there is nothing under
construction under the renamed directory. I am not sure whether it takes
care of edits containing .snapshot properly.
The workaround is to identify these directories and rename, then do
{{saveNamespace}} before performing upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6696) Name node cannot start if the path of a file under construction contains .snapshot
[
https://issues.apache.org/jira/browse/HDFS-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073814#comment-14073814
]
Hadoop QA commented on HDFS-6696:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12657664/hdfs-6696.003.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7456//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7456//console
This message is automatically generated.
Name node cannot start if the path of a file under construction contains
.snapshot
Key: HDFS-6696
URL: https://issues.apache.org/jira/browse/HDFS-6696
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Andrew Wang
Priority: Blocker
Attachments: hdfs-6696.001.patch, hdfs-6696.002.patch,
hdfs-6696.003.patch
Using {{-renameReserved}} to rename .snapshot in a pre-hdfs-snapshot
feature fsimage during upgrade only works, if there is nothing under
construction under the renamed directory. I am not sure whether it takes
care of edits containing .snapshot properly.
The workaround is to identify these directories and rename, then do
{{saveNamespace}} before performing upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6696) Name node cannot start if the path of a file under construction contains .snapshot
[
https://issues.apache.org/jira/browse/HDFS-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073846#comment-14073846
]
Andrew Wang commented on HDFS-6696:
---
I ran the failed tests successfully locally, so I think we're good. Thanks for
reviewing Jing, I'll commit this shortly to all the branches.
Name node cannot start if the path of a file under construction contains
.snapshot
Key: HDFS-6696
URL: https://issues.apache.org/jira/browse/HDFS-6696
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Andrew Wang
Priority: Blocker
Attachments: hdfs-6696.001.patch, hdfs-6696.002.patch,
hdfs-6696.003.patch
Using {{-renameReserved}} to rename .snapshot in a pre-hdfs-snapshot
feature fsimage during upgrade only works, if there is nothing under
construction under the renamed directory. I am not sure whether it takes
care of edits containing .snapshot properly.
The workaround is to identify these directories and rename, then do
{{saveNamespace}} before performing upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6696) Name node cannot start if the path of a file under construction contains .snapshot
[
https://issues.apache.org/jira/browse/HDFS-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072384#comment-14072384
]
Andrew Wang commented on HDFS-6696:
---
Forgot to make this patch available...review would still be appreciated, since
this is a blocker.
Name node cannot start if the path of a file under construction contains
.snapshot
Key: HDFS-6696
URL: https://issues.apache.org/jira/browse/HDFS-6696
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Andrew Wang
Priority: Blocker
Attachments: hdfs-6696.001.patch
Using {{-renameReserved}} to rename .snapshot in a pre-hdfs-snapshot
feature fsimage during upgrade only works, if there is nothing under
construction under the renamed directory. I am not sure whether it takes
care of edits containing .snapshot properly.
The workaround is to identify these directories and rename, then do
{{saveNamespace}} before performing upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6696) Name node cannot start if the path of a file under construction contains .snapshot
[
https://issues.apache.org/jira/browse/HDFS-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072395#comment-14072395
]
Hadoop QA commented on HDFS-6696:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12657038/hdfs-6696.001.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7447//console
This message is automatically generated.
Name node cannot start if the path of a file under construction contains
.snapshot
Key: HDFS-6696
URL: https://issues.apache.org/jira/browse/HDFS-6696
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Andrew Wang
Priority: Blocker
Attachments: hdfs-6696.001.patch
Using {{-renameReserved}} to rename .snapshot in a pre-hdfs-snapshot
feature fsimage during upgrade only works, if there is nothing under
construction under the renamed directory. I am not sure whether it takes
care of edits containing .snapshot properly.
The workaround is to identify these directories and rename, then do
{{saveNamespace}} before performing upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6696) Name node cannot start if the path of a file under construction contains .snapshot
[
https://issues.apache.org/jira/browse/HDFS-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072567#comment-14072567
]
Hadoop QA commented on HDFS-6696:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12657475/hdfs-6696.002.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7449//console
This message is automatically generated.
Name node cannot start if the path of a file under construction contains
.snapshot
Key: HDFS-6696
URL: https://issues.apache.org/jira/browse/HDFS-6696
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Andrew Wang
Priority: Blocker
Attachments: hdfs-6696.001.patch, hdfs-6696.002.patch
Using {{-renameReserved}} to rename .snapshot in a pre-hdfs-snapshot
feature fsimage during upgrade only works, if there is nothing under
construction under the renamed directory. I am not sure whether it takes
care of edits containing .snapshot properly.
The workaround is to identify these directories and rename, then do
{{saveNamespace}} before performing upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6696) Name node cannot start if the path of a file under construction contains .snapshot
[
https://issues.apache.org/jira/browse/HDFS-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069235#comment-14069235
]
Mit Desai commented on HDFS-6696:
-
[~andrew.wang], we were trying to upgrade 0.21.11 to 2.4.0
Name node cannot start if the path of a file under construction contains
.snapshot
Key: HDFS-6696
URL: https://issues.apache.org/jira/browse/HDFS-6696
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Andrew Wang
Priority: Blocker
Using {{-renameReserved}} to rename .snapshot in a pre-hdfs-snapshot
feature fsimage during upgrade only works, if there is nothing under
construction under the renamed directory. I am not sure whether it takes
care of edits containing .snapshot properly.
The workaround is to identify these directories and rename, then do
{{saveNamespace}} before performing upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6696) Name node cannot start if the path of a file under construction contains .snapshot
[
https://issues.apache.org/jira/browse/HDFS-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063977#comment-14063977
]
Andrew Wang commented on HDFS-6696:
---
Thanks for finding this Kihwal, lemme take a look.
Name node cannot start if the path of a file under construction contains
.snapshot
Key: HDFS-6696
URL: https://issues.apache.org/jira/browse/HDFS-6696
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Andrew Wang
Priority: Blocker
Using {{-renameReserved}} to rename .snapshot in a pre-hdfs-snapshot
feature fsimage during upgrade only works, if there is nothing under
construction under the renamed directory. I am not sure whether it takes
care of edits containing .snapshot properly.
The workaround is to identify these directories and rename, then do
{{saveNamespace}} before performing upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6696) Name node cannot start if the path of a file under construction contains .snapshot
[
https://issues.apache.org/jira/browse/HDFS-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064180#comment-14064180
]
Kihwal Lee commented on HDFS-6696:
--
{panel}
java.io.FileNotFoundException: File does not exist: /xxx/yyy/.snapshot/zzz/aaa
at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:65)
at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:55)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$Loader.loadFilesUnderConstruction(FSImageFormat.java:937)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$Loader.load(FSImageFormat.java:424)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:230)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:902)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:888)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:711)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:649)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:359)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:259)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:894)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:641)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:526)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:582)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.init(NameNode.java:747)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.init(NameNode.java:731)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1381)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1447)
{panel}
Name node cannot start if the path of a file under construction contains
.snapshot
Key: HDFS-6696
URL: https://issues.apache.org/jira/browse/HDFS-6696
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Andrew Wang
Priority: Blocker
Using {{-renameReserved}} to rename .snapshot in a pre-hdfs-snapshot
feature fsimage during upgrade only works, if there is nothing under
construction under the renamed directory. I am not sure whether it takes
care of edits containing .snapshot properly.
The workaround is to identify these directories and rename, then do
{{saveNamespace}} before performing upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6696) Name node cannot start if the path of a file under construction contains .snapshot
[
https://issues.apache.org/jira/browse/HDFS-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064206#comment-14064206
]
Andrew Wang commented on HDFS-6696:
---
Poking around on JIRA, snapshots were added in 2.1.0, and -renameReserved was
added in 2.4.0. Thus we had a few releases where -renameReserved wasn't
available at all. I don't think this current behavior is a regression from 2.4
either, since -renameReserved hasn't changed much.
All that considered, should this be marked as a blocker? I'll still work on it
ASAP, but I know [~kasha] is itching to roll a 2.5.
Name node cannot start if the path of a file under construction contains
.snapshot
Key: HDFS-6696
URL: https://issues.apache.org/jira/browse/HDFS-6696
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Andrew Wang
Priority: Blocker
Using {{-renameReserved}} to rename .snapshot in a pre-hdfs-snapshot
feature fsimage during upgrade only works, if there is nothing under
construction under the renamed directory. I am not sure whether it takes
care of edits containing .snapshot properly.
The workaround is to identify these directories and rename, then do
{{saveNamespace}} before performing upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6696) Name node cannot start if the path of a file under construction contains .snapshot
[
https://issues.apache.org/jira/browse/HDFS-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064331#comment-14064331
]
Andrew Wang commented on HDFS-6696:
---
Hey [~kihwal], what Hadoop version were you starting with and upgrading to? I
did some tests upgrading from a 1.2.1 image to trunk which I think repros this,
but I'd like to verify that my WIP fix works for your situation.
Name node cannot start if the path of a file under construction contains
.snapshot
Key: HDFS-6696
URL: https://issues.apache.org/jira/browse/HDFS-6696
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Andrew Wang
Priority: Blocker
Using {{-renameReserved}} to rename .snapshot in a pre-hdfs-snapshot
feature fsimage during upgrade only works, if there is nothing under
construction under the renamed directory. I am not sure whether it takes
care of edits containing .snapshot properly.
The workaround is to identify these directories and rename, then do
{{saveNamespace}} before performing upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
