[jira] [Commented] (HDFS-6601) Issues in finalizing rolling upgrade when there is a layout version change
[ https://issues.apache.org/jira/browse/HDFS-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047138#comment-14047138 ] Hudson commented on HDFS-6601: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1816 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1816/]) HDFS-6601. Issues in finalizing rolling upgrade when there is a layout version change. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1606371) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java > Issues in finalizing rolling upgrade when there is a layout version change > -- > > Key: HDFS-6601 > URL: https://issues.apache.org/jira/browse/HDFS-6601 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.5.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Blocker > Fix For: 3.0.0, 2.5.0 > > Attachments: HDFS-6601.patch > > > After HDFS-6545, we have noticed a couple of issues. > - The storage dir's VERSION file is not properly updated. This becomes a > problem when there is a layout version change. We can have the finalization > do {{storage.writeAll()}} > - {{OP_ROLLING_UPGRADE_FINALIZE}} cannot be replayed, once the corresponding > {{OP_ROLLING_UPGRADE_START}} is consumed and a new fsimage is created (e.g. > rollback image). On restart, NN terminates complaining it can't finalize > something that it didn't start. We can make NN ignore > {{OP_ROLLING_UPGRADE_FINALIZE}} if no rolling upgrade is in progress. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6601) Issues in finalizing rolling upgrade when there is a layout version change
[ https://issues.apache.org/jira/browse/HDFS-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047128#comment-14047128 ] Hudson commented on HDFS-6601: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1789 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1789/]) HDFS-6601. Issues in finalizing rolling upgrade when there is a layout version change. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1606371) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java > Issues in finalizing rolling upgrade when there is a layout version change > -- > > Key: HDFS-6601 > URL: https://issues.apache.org/jira/browse/HDFS-6601 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.5.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Blocker > Fix For: 3.0.0, 2.5.0 > > Attachments: HDFS-6601.patch > > > After HDFS-6545, we have noticed a couple of issues. > - The storage dir's VERSION file is not properly updated. This becomes a > problem when there is a layout version change. We can have the finalization > do {{storage.writeAll()}} > - {{OP_ROLLING_UPGRADE_FINALIZE}} cannot be replayed, once the corresponding > {{OP_ROLLING_UPGRADE_START}} is consumed and a new fsimage is created (e.g. > rollback image). On restart, NN terminates complaining it can't finalize > something that it didn't start. We can make NN ignore > {{OP_ROLLING_UPGRADE_FINALIZE}} if no rolling upgrade is in progress. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6601) Issues in finalizing rolling upgrade when there is a layout version change
[ https://issues.apache.org/jira/browse/HDFS-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047097#comment-14047097 ] Hudson commented on HDFS-6601: -- FAILURE: Integrated in Hadoop-Yarn-trunk #598 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/598/]) HDFS-6601. Issues in finalizing rolling upgrade when there is a layout version change. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1606371) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java > Issues in finalizing rolling upgrade when there is a layout version change > -- > > Key: HDFS-6601 > URL: https://issues.apache.org/jira/browse/HDFS-6601 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.5.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Blocker > Fix For: 3.0.0, 2.5.0 > > Attachments: HDFS-6601.patch > > > After HDFS-6545, we have noticed a couple of issues. > - The storage dir's VERSION file is not properly updated. This becomes a > problem when there is a layout version change. We can have the finalization > do {{storage.writeAll()}} > - {{OP_ROLLING_UPGRADE_FINALIZE}} cannot be replayed, once the corresponding > {{OP_ROLLING_UPGRADE_START}} is consumed and a new fsimage is created (e.g. > rollback image). On restart, NN terminates complaining it can't finalize > something that it didn't start. We can make NN ignore > {{OP_ROLLING_UPGRADE_FINALIZE}} if no rolling upgrade is in progress. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6601) Issues in finalizing rolling upgrade when there is a layout version change
[ https://issues.apache.org/jira/browse/HDFS-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046891#comment-14046891 ] Hudson commented on HDFS-6601: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5796 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5796/]) HDFS-6601. Issues in finalizing rolling upgrade when there is a layout version change. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1606371) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java > Issues in finalizing rolling upgrade when there is a layout version change > -- > > Key: HDFS-6601 > URL: https://issues.apache.org/jira/browse/HDFS-6601 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.5.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Blocker > Fix For: 3.0.0, 2.5.0 > > Attachments: HDFS-6601.patch > > > After HDFS-6545, we have noticed a couple of issues. > - The storage dir's VERSION file is not properly updated. This becomes a > problem when there is a layout version change. We can have the finalization > do {{storage.writeAll()}} > - {{OP_ROLLING_UPGRADE_FINALIZE}} cannot be replayed, once the corresponding > {{OP_ROLLING_UPGRADE_START}} is consumed and a new fsimage is created (e.g. > rollback image). On restart, NN terminates complaining it can't finalize > something that it didn't start. We can make NN ignore > {{OP_ROLLING_UPGRADE_FINALIZE}} if no rolling upgrade is in progress. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6601) Issues in finalizing rolling upgrade when there is a layout version change
[ https://issues.apache.org/jira/browse/HDFS-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046205#comment-14046205 ] Jing Zhao commented on HDFS-6601: - The patch looks good to me. +1. Thanks [~kihwal]! > Issues in finalizing rolling upgrade when there is a layout version change > -- > > Key: HDFS-6601 > URL: https://issues.apache.org/jira/browse/HDFS-6601 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.5.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Blocker > Attachments: HDFS-6601.patch > > > After HDFS-6545, we have noticed a couple of issues. > - The storage dir's VERSION file is not properly updated. This becomes a > problem when there is a layout version change. We can have the finalization > do {{storage.writeAll()}} > - {{OP_ROLLING_UPGRADE_FINALIZE}} cannot be replayed, once the corresponding > {{OP_ROLLING_UPGRADE_START}} is consumed and a new fsimage is created (e.g. > rollback image). On restart, NN terminates complaining it can't finalize > something that it didn't start. We can make NN ignore > {{OP_ROLLING_UPGRADE_FINALIZE}} if no rolling upgrade is in progress. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6601) Issues in finalizing rolling upgrade when there is a layout version change
[ https://issues.apache.org/jira/browse/HDFS-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045990#comment-14045990 ] Kihwal Lee commented on HDFS-6601: -- [~jingzhao] Would you review the patch? We've tested the NN rolling upgrades scenario involving layout version change with this patch. > Issues in finalizing rolling upgrade when there is a layout version change > -- > > Key: HDFS-6601 > URL: https://issues.apache.org/jira/browse/HDFS-6601 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.5.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee >Priority: Blocker > Attachments: HDFS-6601.patch > > > After HDFS-6545, we have noticed a couple of issues. > - The storage dir's VERSION file is not properly updated. This becomes a > problem when there is a layout version change. We can have the finalization > do {{storage.writeAll()}} > - {{OP_ROLLING_UPGRADE_FINALIZE}} cannot be replayed, once the corresponding > {{OP_ROLLING_UPGRADE_START}} is consumed and a new fsimage is created (e.g. > rollback image). On restart, NN terminates complaining it can't finalize > something that it didn't start. We can make NN ignore > {{OP_ROLLING_UPGRADE_FINALIZE}} if no rolling upgrade is in progress. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6601) Issues in finalizing rolling upgrade when there is a layout version change
[ https://issues.apache.org/jira/browse/HDFS-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042886#comment-14042886 ] Hadoop QA commented on HDFS-6601: - {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12652284/HDFS-6601.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7227//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7227//console This message is automatically generated. > Issues in finalizing rolling upgrade when there is a layout version change > -- > > Key: HDFS-6601 > URL: https://issues.apache.org/jira/browse/HDFS-6601 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.5.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee > Attachments: HDFS-6601.patch > > > After HDFS-6545, we have noticed a couple of issues. > - The storage dir's VERSION file is not properly updated. This becomes a > problem when there is a layout version change. We can have the finalization > do {{storage.writeAll()}} > - {{OP_ROLLING_UPGRADE_FINALIZE}} cannot be replayed, once the corresponding > {{OP_ROLLING_UPGRADE_START}} is consumed and a new fsimage is created (e.g. > rollback image). On restart, NN terminates complaining it can't finalize > something that it didn't start. We can make NN ignore > {{OP_ROLLING_UPGRADE_FINALIZE}} if no rolling upgrade is in progress. -- This message was sent by Atlassian JIRA (v6.2#6252)