[jira] [Commented] (HDFS-988) saveNamespace race can corrupt the edits log
[ https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048278#comment-13048278 ] Hadoop QA commented on HDFS-988: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12482272/hdfs-988-7.patch against trunk revision 1134869. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 36 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/775//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/775//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/775//console This message is automatically generated. > saveNamespace race can corrupt the edits log > > > Key: HDFS-988 > URL: https://issues.apache.org/jira/browse/HDFS-988 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20-append, 0.21.0, 0.22.0 >Reporter: dhruba borthakur >Assignee: Eli Collins >Priority: Blocker > Fix For: 0.20-append, 0.22.0 > > Attachments: 988-fixups.txt, HDFS-988_fix_synchs.patch, > hdfs-988-2.patch, hdfs-988-3.patch, hdfs-988-4.patch, hdfs-988-5.patch, > hdfs-988-6.patch, hdfs-988-7.patch, hdfs-988-b22-1.patch, hdfs-988.txt, > saveNamespace.txt, saveNamespace_20-append.patch > > > The adminstrator puts the namenode is safemode and then issues the > savenamespace command. This can corrupt the edits log. The problem is that > when the NN enters safemode, there could still be pending logSycs occuring > from other threads. Now, the saveNamespace command, when executed, would save > a edits log with partial writes. I have seen this happen on 0.20. > https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-988) saveNamespace race can corrupt the edits log
[ https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048301#comment-13048301 ] Hudson commented on HDFS-988: - Integrated in Hadoop-Hdfs-22-branch #66 (See [https://builds.apache.org/job/Hadoop-Hdfs-22-branch/66/]) HDFS-988. saveNamespace race can corrupt the edits log. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1134860 Files : * /hadoop/hdfs/branches/branch-0.22/src/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * /hadoop/hdfs/branches/branch-0.22/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java * /hadoop/hdfs/branches/branch-0.22/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/hdfs/branches/branch-0.22/src/test/hdfs/org/apache/hadoop/hdfs/TestSafeMode.java * /hadoop/hdfs/branches/branch-0.22/CHANGES.txt * /hadoop/hdfs/branches/branch-0.22/src/test/hdfs/org/apache/hadoop/hdfs/DFSTestUtil.java > saveNamespace race can corrupt the edits log > > > Key: HDFS-988 > URL: https://issues.apache.org/jira/browse/HDFS-988 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20-append, 0.21.0, 0.22.0 >Reporter: dhruba borthakur >Assignee: Eli Collins >Priority: Blocker > Fix For: 0.20-append, 0.22.0 > > Attachments: 988-fixups.txt, HDFS-988_fix_synchs.patch, > hdfs-988-2.patch, hdfs-988-3.patch, hdfs-988-4.patch, hdfs-988-5.patch, > hdfs-988-6.patch, hdfs-988-7.patch, hdfs-988-b22-1.patch, hdfs-988.txt, > saveNamespace.txt, saveNamespace_20-append.patch > > > The adminstrator puts the namenode is safemode and then issues the > savenamespace command. This can corrupt the edits log. The problem is that > when the NN enters safemode, there could still be pending logSycs occuring > from other threads. Now, the saveNamespace command, when executed, would save > a edits log with partial writes. I have seen this happen on 0.20. > https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-988) saveNamespace race can corrupt the edits log
[ https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048406#comment-13048406 ] Eli Collins commented on HDFS-988: -- Marking issue as fixed, we now have patches checked into branch-20-append, branch-22 and trunk. > saveNamespace race can corrupt the edits log > > > Key: HDFS-988 > URL: https://issues.apache.org/jira/browse/HDFS-988 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20-append, 0.21.0, 0.22.0 >Reporter: dhruba borthakur >Assignee: Eli Collins >Priority: Blocker > Fix For: 0.20-append, 0.22.0 > > Attachments: 988-fixups.txt, HDFS-988_fix_synchs.patch, > hdfs-988-2.patch, hdfs-988-3.patch, hdfs-988-4.patch, hdfs-988-5.patch, > hdfs-988-6.patch, hdfs-988-7.patch, hdfs-988-b22-1.patch, hdfs-988.txt, > saveNamespace.txt, saveNamespace_20-append.patch > > > The adminstrator puts the namenode is safemode and then issues the > savenamespace command. This can corrupt the edits log. The problem is that > when the NN enters safemode, there could still be pending logSycs occuring > from other threads. Now, the saveNamespace command, when executed, would save > a edits log with partial writes. I have seen this happen on 0.20. > https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-988) saveNamespace race can corrupt the edits log
[ https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050133#comment-13050133 ] Hudson commented on HDFS-988: - Integrated in Hadoop-Hdfs-trunk-Commit #746 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/746/]) > saveNamespace race can corrupt the edits log > > > Key: HDFS-988 > URL: https://issues.apache.org/jira/browse/HDFS-988 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20-append, 0.21.0, 0.22.0 >Reporter: dhruba borthakur >Assignee: Eli Collins >Priority: Blocker > Fix For: 0.20-append, 0.22.0 > > Attachments: 988-fixups.txt, HDFS-988_fix_synchs.patch, > hdfs-988-2.patch, hdfs-988-3.patch, hdfs-988-4.patch, hdfs-988-5.patch, > hdfs-988-6.patch, hdfs-988-7.patch, hdfs-988-b22-1.patch, hdfs-988.txt, > saveNamespace.txt, saveNamespace_20-append.patch > > > The adminstrator puts the namenode is safemode and then issues the > savenamespace command. This can corrupt the edits log. The problem is that > when the NN enters safemode, there could still be pending logSycs occuring > from other threads. Now, the saveNamespace command, when executed, would save > a edits log with partial writes. I have seen this happen on 0.20. > https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-988) saveNamespace race can corrupt the edits log
[ https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080642#comment-13080642 ] Tsz Wo (Nicholas), SZE commented on HDFS-988: - Hi Eli and Todd, it seems that this issue has introduced a deadlock; see HDFS-2229. > saveNamespace race can corrupt the edits log > > > Key: HDFS-988 > URL: https://issues.apache.org/jira/browse/HDFS-988 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20-append, 0.21.0, 0.22.0 >Reporter: dhruba borthakur >Assignee: Eli Collins >Priority: Blocker > Fix For: 0.20-append, 0.22.0 > > Attachments: 988-fixups.txt, HDFS-988_fix_synchs.patch, > hdfs-988-2.patch, hdfs-988-3.patch, hdfs-988-4.patch, hdfs-988-5.patch, > hdfs-988-6.patch, hdfs-988-7.patch, hdfs-988-b22-1.patch, hdfs-988.txt, > saveNamespace.txt, saveNamespace_20-append.patch > > > The adminstrator puts the namenode is safemode and then issues the > savenamespace command. This can corrupt the edits log. The problem is that > when the NN enters safemode, there could still be pending logSycs occuring > from other threads. Now, the saveNamespace command, when executed, would save > a edits log with partial writes. I have seen this happen on 0.20. > https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-988) saveNamespace race can corrupt the edits log
[ https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096254#comment-13096254 ] Suresh Srinivas commented on HDFS-988: -- +1 for the 20-security patch. > saveNamespace race can corrupt the edits log > > > Key: HDFS-988 > URL: https://issues.apache.org/jira/browse/HDFS-988 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20-append, 0.21.0, 0.22.0 >Reporter: dhruba borthakur >Assignee: Eli Collins >Priority: Blocker > Fix For: 0.20-append, 0.20.205.0, 0.22.0 > > Attachments: 988-fixups.txt, HDFS-988.20-security.patch, > HDFS-988_fix_synchs.patch, hdfs-988-2.patch, hdfs-988-3.patch, > hdfs-988-4.patch, hdfs-988-5.patch, hdfs-988-6.patch, hdfs-988-7.patch, > hdfs-988-b22-1.patch, hdfs-988.txt, saveNamespace.txt, > saveNamespace_20-append.patch > > > The adminstrator puts the namenode is safemode and then issues the > savenamespace command. This can corrupt the edits log. The problem is that > when the NN enters safemode, there could still be pending logSycs occuring > from other threads. Now, the saveNamespace command, when executed, would save > a edits log with partial writes. I have seen this happen on 0.20. > https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-988) saveNamespace race can corrupt the edits log
[ https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096261#comment-13096261 ] Suresh Srinivas commented on HDFS-988: -- I committed the patch to 0.20-security branch. > saveNamespace race can corrupt the edits log > > > Key: HDFS-988 > URL: https://issues.apache.org/jira/browse/HDFS-988 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20-append, 0.21.0, 0.22.0 >Reporter: dhruba borthakur >Assignee: Eli Collins >Priority: Blocker > Fix For: 0.20-append, 0.20.205.0, 0.22.0 > > Attachments: 988-fixups.txt, HDFS-988.20-security.patch, > HDFS-988_fix_synchs.patch, hdfs-988-2.patch, hdfs-988-3.patch, > hdfs-988-4.patch, hdfs-988-5.patch, hdfs-988-6.patch, hdfs-988-7.patch, > hdfs-988-b22-1.patch, hdfs-988.txt, saveNamespace.txt, > saveNamespace_20-append.patch > > > The adminstrator puts the namenode is safemode and then issues the > savenamespace command. This can corrupt the edits log. The problem is that > when the NN enters safemode, there could still be pending logSycs occuring > from other threads. Now, the saveNamespace command, when executed, would save > a edits log with partial writes. I have seen this happen on 0.20. > https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-988) saveNamespace race can corrupt the edits log
[ https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096262#comment-13096262 ] Suresh Srinivas commented on HDFS-988: -- +1 for the patch. > saveNamespace race can corrupt the edits log > > > Key: HDFS-988 > URL: https://issues.apache.org/jira/browse/HDFS-988 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20-append, 0.21.0, 0.22.0 >Reporter: dhruba borthakur >Assignee: Eli Collins >Priority: Blocker > Fix For: 0.20-append, 0.20.205.0, 0.22.0 > > Attachments: 988-fixups.txt, HDFS-988.20-security.patch, > HDFS-988_fix_synchs.patch, hdfs-988-2.patch, hdfs-988-3.patch, > hdfs-988-4.patch, hdfs-988-5.patch, hdfs-988-6.patch, hdfs-988-7.patch, > hdfs-988-b22-1.patch, hdfs-988.txt, saveNamespace.txt, > saveNamespace_20-append.patch > > > The adminstrator puts the namenode is safemode and then issues the > savenamespace command. This can corrupt the edits log. The problem is that > when the NN enters safemode, there could still be pending logSycs occuring > from other threads. Now, the saveNamespace command, when executed, would save > a edits log with partial writes. I have seen this happen on 0.20. > https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
