[jira] [Commented] (HDFS-6135) In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump when rolling back
[
https://issues.apache.org/jira/browse/HDFS-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13946570#comment-13946570
]
Hudson commented on HDFS-6135:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #1712 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1712/])
HDFS-6135. In HDFS upgrade with HA setup, JournalNode cannot handle layout
version bump when rolling back. Contributed by Jing Zhao. (jing9:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1581070)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JNStorage.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/StorageInfo.java
> In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump
> when rolling back
> --
>
> Key: HDFS-6135
> URL: https://issues.apache.org/jira/browse/HDFS-6135
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: journal-node
>Affects Versions: 3.0.0
>Reporter: Jing Zhao
>Assignee: Jing Zhao
>Priority: Blocker
> Fix For: 2.4.0
>
> Attachments: HDFS-6135.000.patch, HDFS-6135.001.patch,
> HDFS-6135.002.patch, HDFS-6135.test.txt
>
>
> While doing HDFS upgrade with HA setup, if the layoutversion gets changed in
> the upgrade, the rollback may trigger the following exception in JournalNodes
> (suppose the new software bumped the layoutversion from -55 to -56):
> {code}
> 14/03/21 01:01:53 FATAL namenode.NameNode: Exception in namenode join
> org.apache.hadoop.hdfs.qjournal.client.QuorumException: Could not check if
> roll back possible for one or more JournalNodes. 1 exceptions thrown:
> Unexpected version of storage directory /grid/1/tmp/journal/mycluster.
> Reported: -56. Expecting = -55.
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.setLayoutVersion(StorageInfo.java:203)
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.setFieldsFromProperties(StorageInfo.java:156)
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.readProperties(StorageInfo.java:135)
> at
> org.apache.hadoop.hdfs.qjournal.server.JNStorage.analyzeStorage(JNStorage.java:202)
> at
> org.apache.hadoop.hdfs.qjournal.server.JNStorage.(JNStorage.java:73)
> at
> org.apache.hadoop.hdfs.qjournal.server.Journal.(Journal.java:142)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNode.getOrCreateJournal(JournalNode.java:87)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNode.canRollBack(JournalNode.java:304)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.canRollBack(JournalNodeRpcServer.java:228)
> {code}
> Looks like for rollback JN with old software cannot handle future
> layoutversion brought by new software.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6135) In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump when rolling back
[
https://issues.apache.org/jira/browse/HDFS-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13946542#comment-13946542
]
Hudson commented on HDFS-6135:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1737 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1737/])
HDFS-6135. In HDFS upgrade with HA setup, JournalNode cannot handle layout
version bump when rolling back. Contributed by Jing Zhao. (jing9:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1581070)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JNStorage.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/StorageInfo.java
> In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump
> when rolling back
> --
>
> Key: HDFS-6135
> URL: https://issues.apache.org/jira/browse/HDFS-6135
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: journal-node
>Affects Versions: 3.0.0
>Reporter: Jing Zhao
>Assignee: Jing Zhao
>Priority: Blocker
> Fix For: 2.4.0
>
> Attachments: HDFS-6135.000.patch, HDFS-6135.001.patch,
> HDFS-6135.002.patch, HDFS-6135.test.txt
>
>
> While doing HDFS upgrade with HA setup, if the layoutversion gets changed in
> the upgrade, the rollback may trigger the following exception in JournalNodes
> (suppose the new software bumped the layoutversion from -55 to -56):
> {code}
> 14/03/21 01:01:53 FATAL namenode.NameNode: Exception in namenode join
> org.apache.hadoop.hdfs.qjournal.client.QuorumException: Could not check if
> roll back possible for one or more JournalNodes. 1 exceptions thrown:
> Unexpected version of storage directory /grid/1/tmp/journal/mycluster.
> Reported: -56. Expecting = -55.
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.setLayoutVersion(StorageInfo.java:203)
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.setFieldsFromProperties(StorageInfo.java:156)
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.readProperties(StorageInfo.java:135)
> at
> org.apache.hadoop.hdfs.qjournal.server.JNStorage.analyzeStorage(JNStorage.java:202)
> at
> org.apache.hadoop.hdfs.qjournal.server.JNStorage.(JNStorage.java:73)
> at
> org.apache.hadoop.hdfs.qjournal.server.Journal.(Journal.java:142)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNode.getOrCreateJournal(JournalNode.java:87)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNode.canRollBack(JournalNode.java:304)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.canRollBack(JournalNodeRpcServer.java:228)
> {code}
> Looks like for rollback JN with old software cannot handle future
> layoutversion brought by new software.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6135) In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump when rolling back
[
https://issues.apache.org/jira/browse/HDFS-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13946410#comment-13946410
]
Hudson commented on HDFS-6135:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #520 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/520/])
HDFS-6135. In HDFS upgrade with HA setup, JournalNode cannot handle layout
version bump when rolling back. Contributed by Jing Zhao. (jing9:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1581070)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JNStorage.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/StorageInfo.java
> In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump
> when rolling back
> --
>
> Key: HDFS-6135
> URL: https://issues.apache.org/jira/browse/HDFS-6135
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: journal-node
>Affects Versions: 3.0.0
>Reporter: Jing Zhao
>Assignee: Jing Zhao
>Priority: Blocker
> Fix For: 2.4.0
>
> Attachments: HDFS-6135.000.patch, HDFS-6135.001.patch,
> HDFS-6135.002.patch, HDFS-6135.test.txt
>
>
> While doing HDFS upgrade with HA setup, if the layoutversion gets changed in
> the upgrade, the rollback may trigger the following exception in JournalNodes
> (suppose the new software bumped the layoutversion from -55 to -56):
> {code}
> 14/03/21 01:01:53 FATAL namenode.NameNode: Exception in namenode join
> org.apache.hadoop.hdfs.qjournal.client.QuorumException: Could not check if
> roll back possible for one or more JournalNodes. 1 exceptions thrown:
> Unexpected version of storage directory /grid/1/tmp/journal/mycluster.
> Reported: -56. Expecting = -55.
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.setLayoutVersion(StorageInfo.java:203)
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.setFieldsFromProperties(StorageInfo.java:156)
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.readProperties(StorageInfo.java:135)
> at
> org.apache.hadoop.hdfs.qjournal.server.JNStorage.analyzeStorage(JNStorage.java:202)
> at
> org.apache.hadoop.hdfs.qjournal.server.JNStorage.(JNStorage.java:73)
> at
> org.apache.hadoop.hdfs.qjournal.server.Journal.(Journal.java:142)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNode.getOrCreateJournal(JournalNode.java:87)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNode.canRollBack(JournalNode.java:304)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.canRollBack(JournalNodeRpcServer.java:228)
> {code}
> Looks like for rollback JN with old software cannot handle future
> layoutversion brought by new software.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6135) In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump when rolling back
[
https://issues.apache.org/jira/browse/HDFS-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13945749#comment-13945749
]
Hudson commented on HDFS-6135:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #5392 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/5392/])
HDFS-6135. In HDFS upgrade with HA setup, JournalNode cannot handle layout
version bump when rolling back. Contributed by Jing Zhao. (jing9:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1581070)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JNStorage.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/StorageInfo.java
> In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump
> when rolling back
> --
>
> Key: HDFS-6135
> URL: https://issues.apache.org/jira/browse/HDFS-6135
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: journal-node
>Affects Versions: 3.0.0
>Reporter: Jing Zhao
>Assignee: Jing Zhao
>Priority: Blocker
> Fix For: 2.4.0
>
> Attachments: HDFS-6135.000.patch, HDFS-6135.001.patch,
> HDFS-6135.002.patch, HDFS-6135.test.txt
>
>
> While doing HDFS upgrade with HA setup, if the layoutversion gets changed in
> the upgrade, the rollback may trigger the following exception in JournalNodes
> (suppose the new software bumped the layoutversion from -55 to -56):
> {code}
> 14/03/21 01:01:53 FATAL namenode.NameNode: Exception in namenode join
> org.apache.hadoop.hdfs.qjournal.client.QuorumException: Could not check if
> roll back possible for one or more JournalNodes. 1 exceptions thrown:
> Unexpected version of storage directory /grid/1/tmp/journal/mycluster.
> Reported: -56. Expecting = -55.
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.setLayoutVersion(StorageInfo.java:203)
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.setFieldsFromProperties(StorageInfo.java:156)
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.readProperties(StorageInfo.java:135)
> at
> org.apache.hadoop.hdfs.qjournal.server.JNStorage.analyzeStorage(JNStorage.java:202)
> at
> org.apache.hadoop.hdfs.qjournal.server.JNStorage.(JNStorage.java:73)
> at
> org.apache.hadoop.hdfs.qjournal.server.Journal.(Journal.java:142)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNode.getOrCreateJournal(JournalNode.java:87)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNode.canRollBack(JournalNode.java:304)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.canRollBack(JournalNodeRpcServer.java:228)
> {code}
> Looks like for rollback JN with old software cannot handle future
> layoutversion brought by new software.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6135) In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump when rolling back
[
https://issues.apache.org/jira/browse/HDFS-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13945614#comment-13945614
]
Hadoop QA commented on HDFS-6135:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12636293/HDFS-6135.002.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6476//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6476//console
This message is automatically generated.
> In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump
> when rolling back
> --
>
> Key: HDFS-6135
> URL: https://issues.apache.org/jira/browse/HDFS-6135
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Jing Zhao
>Assignee: Jing Zhao
>Priority: Blocker
> Attachments: HDFS-6135.000.patch, HDFS-6135.001.patch,
> HDFS-6135.002.patch, HDFS-6135.test.txt
>
>
> While doing HDFS upgrade with HA setup, if the layoutversion gets changed in
> the upgrade, the rollback may trigger the following exception in JournalNodes
> (suppose the new software bumped the layoutversion from -55 to -56):
> {code}
> 14/03/21 01:01:53 FATAL namenode.NameNode: Exception in namenode join
> org.apache.hadoop.hdfs.qjournal.client.QuorumException: Could not check if
> roll back possible for one or more JournalNodes. 1 exceptions thrown:
> Unexpected version of storage directory /grid/1/tmp/journal/mycluster.
> Reported: -56. Expecting = -55.
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.setLayoutVersion(StorageInfo.java:203)
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.setFieldsFromProperties(StorageInfo.java:156)
> at
> org.apache.hadoop.hdfs.server.common.StorageInfo.readProperties(StorageInfo.java:135)
> at
> org.apache.hadoop.hdfs.qjournal.server.JNStorage.analyzeStorage(JNStorage.java:202)
> at
> org.apache.hadoop.hdfs.qjournal.server.JNStorage.(JNStorage.java:73)
> at
> org.apache.hadoop.hdfs.qjournal.server.Journal.(Journal.java:142)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNode.getOrCreateJournal(JournalNode.java:87)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNode.canRollBack(JournalNode.java:304)
> at
> org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.canRollBack(JournalNodeRpcServer.java:228)
> {code}
> Looks like for rollback JN with old software cannot handle future
> layoutversion brought by new software.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6135) In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump when rolling back
[
https://issues.apache.org/jira/browse/HDFS-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13944706#comment-13944706
]
Hadoop QA commented on HDFS-6135:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12636271/HDFS-6135.000.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.shell.TestHdfsTextCommand
org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
org.apache.hadoop.hdfs.server.namenode.TestNameNodeJspHelper
org.apache.hadoop.hdfs.server.namenode.TestAclConfigFlag
org.apache.hadoop.hdfs.TestClose
org.apache.hadoop.hdfs.TestShortCircuitLocalRead
org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
org.apache.hadoop.hdfs.TestFSInputChecker
org.apache.hadoop.hdfs.server.namenode.TestBackupNode
org.apache.hadoop.hdfs.TestDataTransferProtocol
org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat
org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol
org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
org.apache.hadoop.hdfs.server.namenode.TestFSNamesystemMBean
org.apache.hadoop.hdfs.TestDFSClientFailover
org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters
org.apache.hadoop.tools.TestJMXGet
org.apache.hadoop.hdfs.TestDFSShell
org.apache.hadoop.hdfs.server.namenode.TestFSImage
org.apache.hadoop.hdfs.security.TestDelegationToken
org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
org.apache.hadoop.fs.permission.TestStickyBit
org.apache.hadoop.hdfs.TestFileConcurrentReader
org.apache.hadoop.hdfs.server.datanode.TestCachingStrategy
org.apache.hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
org.apache.hadoop.fs.TestFcHdfsCreateMkdir
org.apache.hadoop.hdfs.TestCrcCorruption
org.apache.hadoop.hdfs.TestAppendDifferentChecksum
org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
org.apache.hadoop.fs.viewfs.TestViewFileSystemHdfs
org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser
org.apache.hadoop.hdfs.server.namenode.TestSequentialBlockId
org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
org.apache.hadoop.hdfs.TestDFSPermission
org.apache.hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot
org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
org.apache.hadoop.hdfs.TestListFilesInFileContext
org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
org.apache.hadoop.fs.viewfs.TestViewFsDefaultValue
org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics
