[jira] [Commented] (HDFS-6545) Finalizing rolling upgrade can make NN unavailable for a long duration
[
https://issues.apache.org/jira/browse/HDFS-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14035587#comment-14035587
]
Hudson commented on HDFS-6545:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #587 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/587/])
HDFS-6545. Finalizing rolling upgrade can make NN unavailable for a long
duration. Contributed by Kihwal Lee. (kihwal:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1603239)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
Finalizing rolling upgrade can make NN unavailable for a long duration
--
Key: HDFS-6545
URL: https://issues.apache.org/jira/browse/HDFS-6545
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 3.0.0, 2.5.0
Attachments: HDFS-6545.patch, HDFS-6545.v2.patch
In {{FSNamesystem#finalizeRollingUpgrade()}}, {{saveNamespace()}} is directly
called. For name nodes with a big name space, it can take minutes to save a
new fsimage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6545) Finalizing rolling upgrade can make NN unavailable for a long duration
[
https://issues.apache.org/jira/browse/HDFS-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14035705#comment-14035705
]
Hudson commented on HDFS-6545:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #1778 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1778/])
HDFS-6545. Finalizing rolling upgrade can make NN unavailable for a long
duration. Contributed by Kihwal Lee. (kihwal:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1603239)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
Finalizing rolling upgrade can make NN unavailable for a long duration
--
Key: HDFS-6545
URL: https://issues.apache.org/jira/browse/HDFS-6545
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 3.0.0, 2.5.0
Attachments: HDFS-6545.patch, HDFS-6545.v2.patch
In {{FSNamesystem#finalizeRollingUpgrade()}}, {{saveNamespace()}} is directly
called. For name nodes with a big name space, it can take minutes to save a
new fsimage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6545) Finalizing rolling upgrade can make NN unavailable for a long duration
[
https://issues.apache.org/jira/browse/HDFS-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14035814#comment-14035814
]
Hudson commented on HDFS-6545:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1805 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1805/])
HDFS-6545. Finalizing rolling upgrade can make NN unavailable for a long
duration. Contributed by Kihwal Lee. (kihwal:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1603239)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
Finalizing rolling upgrade can make NN unavailable for a long duration
--
Key: HDFS-6545
URL: https://issues.apache.org/jira/browse/HDFS-6545
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 3.0.0, 2.5.0
Attachments: HDFS-6545.patch, HDFS-6545.v2.patch
In {{FSNamesystem#finalizeRollingUpgrade()}}, {{saveNamespace()}} is directly
called. For name nodes with a big name space, it can take minutes to save a
new fsimage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6545) Finalizing rolling upgrade can make NN unavailable for a long duration
[
https://issues.apache.org/jira/browse/HDFS-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034076#comment-14034076
]
Jing Zhao commented on HDFS-6545:
-
The patch looks good to me. +1. Thanks for working on this, [~kihwal].
Finalizing rolling upgrade can make NN unavailable for a long duration
--
Key: HDFS-6545
URL: https://issues.apache.org/jira/browse/HDFS-6545
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Attachments: HDFS-6545.patch, HDFS-6545.v2.patch
In {{FSNamesystem#finalizeRollingUpgrade()}}, {{saveNamespace()}} is directly
called. For name nodes with a big name space, it can take minutes to save a
new fsimage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6545) Finalizing rolling upgrade can make NN unavailable for a long duration
[
https://issues.apache.org/jira/browse/HDFS-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034136#comment-14034136
]
Kihwal Lee commented on HDFS-6545:
--
Thanks for the review [~jingzhao]. I've committed this to trunk and branch-2.
Finalizing rolling upgrade can make NN unavailable for a long duration
--
Key: HDFS-6545
URL: https://issues.apache.org/jira/browse/HDFS-6545
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 3.0.0, 2.5.0
Attachments: HDFS-6545.patch, HDFS-6545.v2.patch
In {{FSNamesystem#finalizeRollingUpgrade()}}, {{saveNamespace()}} is directly
called. For name nodes with a big name space, it can take minutes to save a
new fsimage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6545) Finalizing rolling upgrade can make NN unavailable for a long duration
[
https://issues.apache.org/jira/browse/HDFS-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034149#comment-14034149
]
Hudson commented on HDFS-6545:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #5719 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/5719/])
HDFS-6545. Finalizing rolling upgrade can make NN unavailable for a long
duration. Contributed by Kihwal Lee. (kihwal:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1603239)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
Finalizing rolling upgrade can make NN unavailable for a long duration
--
Key: HDFS-6545
URL: https://issues.apache.org/jira/browse/HDFS-6545
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 3.0.0, 2.5.0
Attachments: HDFS-6545.patch, HDFS-6545.v2.patch
In {{FSNamesystem#finalizeRollingUpgrade()}}, {{saveNamespace()}} is directly
called. For name nodes with a big name space, it can take minutes to save a
new fsimage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6545) Finalizing rolling upgrade can make NN unavailable for a long duration
[
https://issues.apache.org/jira/browse/HDFS-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14032962#comment-14032962
]
Kihwal Lee commented on HDFS-6545:
--
I think we can skip {{saveNamespace()}}. Presence of finalize op in a
consumable edit segment should not cause any trouble, since any subsequent
rolling upgrade will create a rollback image with txid past the op. The only
potential inconvenience is that if ANN or SBN needs to be restarted before the
new version saving a checkpoint for some reason, it may need -rollingUpgrade
started start-up option if there was a layout version change. But this can
also happen on SBN with the current code. If SBN was not up when ANN is
finalizing an upgrade, SBN may need to be started with -rollingUpgrade
started in order for it to read the rollback image saved by the old version.
Or it will need to be bootstrapped fresh in order to download the new fsimage
created only on ANN through {{saveNamespace()}}.
In summary, removing {{saveNamespace()}} from {{finalizeRollingUpgrade()}}
seems reasonable.
Finalizing rolling upgrade can make NN unavailable for a long duration
--
Key: HDFS-6545
URL: https://issues.apache.org/jira/browse/HDFS-6545
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Kihwal Lee
Priority: Critical
In {{FSNamesystem#finalizeRollingUpgrade()}}, {{saveNamespace()}} is directly
called. For name nodes with a big name space, it can take minutes to save a
new fsimage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6545) Finalizing rolling upgrade can make NN unavailable for a long duration
[
https://issues.apache.org/jira/browse/HDFS-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14033128#comment-14033128
]
Jing Zhao commented on HDFS-6545:
-
Yeah, looks like we do not need to do an extra checkpoint while finalizing
rolling upgrade. The current patch looks good to me. One minor comment is that
for HA setup we do not need to call logSync since rollEditLog already covers
the sync part.
Finalizing rolling upgrade can make NN unavailable for a long duration
--
Key: HDFS-6545
URL: https://issues.apache.org/jira/browse/HDFS-6545
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Kihwal Lee
Priority: Critical
Attachments: HDFS-6545.patch
In {{FSNamesystem#finalizeRollingUpgrade()}}, {{saveNamespace()}} is directly
called. For name nodes with a big name space, it can take minutes to save a
new fsimage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6545) Finalizing rolling upgrade can make NN unavailable for a long duration
[
https://issues.apache.org/jira/browse/HDFS-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14033225#comment-14033225
]
Hadoop QA commented on HDFS-6545:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650655/HDFS-6545.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7140//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7140//console
This message is automatically generated.
Finalizing rolling upgrade can make NN unavailable for a long duration
--
Key: HDFS-6545
URL: https://issues.apache.org/jira/browse/HDFS-6545
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Kihwal Lee
Priority: Critical
Attachments: HDFS-6545.patch
In {{FSNamesystem#finalizeRollingUpgrade()}}, {{saveNamespace()}} is directly
called. For name nodes with a big name space, it can take minutes to save a
new fsimage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
