[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14642823#comment-14642823 ] Sangjin Lee commented on HDFS-7788: --- The backport to 2.6.0 is pretty trivial but the test FS image needs to be recreated for 2.6.0. Also it'd be good to rename the variable (HADOOP_2_7_ZER0_BLOCK_SIZE_TGZ) in TestFSImage for 2.6. I'll post a suggested backport patch for 2.6.0. Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Rushabh S Shah Priority: Blocker Labels: 2.6.1-candidate Fix For: 2.7.0 Attachments: HDFS-7788-binary.patch, rushabh.patch Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14332192#comment-14332192 ] Hudson commented on HDFS-7788: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #112 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/112/]) HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Rushabh S Shah Priority: Blocker Fix For: 2.7.0 Attachments: HDFS-7788-binary.patch, rushabh.patch Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14332220#comment-14332220 ] Hudson commented on HDFS-7788: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2062 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2062/]) HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) * hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Rushabh S Shah Priority: Blocker Fix For: 2.7.0 Attachments: HDFS-7788-binary.patch, rushabh.patch Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330285#comment-14330285 ] Hudson commented on HDFS-7788: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2043 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2043/]) HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java * hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Rushabh S Shah Priority: Blocker Fix For: 2.7.0 Attachments: HDFS-7788-binary.patch, rushabh.patch Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330306#comment-14330306 ] Hudson commented on HDFS-7788: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #102 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/102/]) HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java * hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Rushabh S Shah Priority: Blocker Fix For: 2.7.0 Attachments: HDFS-7788-binary.patch, rushabh.patch Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330147#comment-14330147 ] Hudson commented on HDFS-7788: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #111 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/111/]) HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) * hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Rushabh S Shah Priority: Blocker Fix For: 2.7.0 Attachments: HDFS-7788-binary.patch, rushabh.patch Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330177#comment-14330177 ] Hudson commented on HDFS-7788: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #845 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/845/]) HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Rushabh S Shah Priority: Blocker Fix For: 2.7.0 Attachments: HDFS-7788-binary.patch, rushabh.patch Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[
https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329010#comment-14329010
]
Kihwal Lee commented on HDFS-7788:
--
Although the precommit posted a +1, the test result is missing. Judging from
the run-time, it ran many tests. So I ran hdfs tests locally overnight. Two
test cases failed.
{panel}
TestDFSHAAdminMiniCluster.testFencer:163 expected:0 but was:-1
TestDFSUpgradeFromImage.testUpgradeFromRel1BBWImage:619-upgradeAndVerify:597-verifyFileSystem:225-verifyDir:210-dfsOpenFileWithRetries:174
ยป IO
{panel}
I just reran the tests and they are passing.
+1 The change looks good.
Post-2.6 namenode may not start up with an image containing inodes created
with an old release.
---
Key: HDFS-7788
URL: https://issues.apache.org/jira/browse/HDFS-7788
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Rushabh S Shah
Priority: Blocker
Attachments: HDFS-7788-binary.patch, rushabh.patch
Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify
arbitrarily small preferred block size for a file including 0. This was
normally done by faulty clients or failed creates, but it was possible.
Until 2.5, reading a fsimage containing inodes with 0 byte preferred block
size was allowed. So if a fsimage contained such an inode, the namenode would
come up fine. In 2.6, the preferred block size is required be 0. Because
of this change, the image that worked with 2.5 may not work with 2.6.
If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is
under this risk even if it worked fine with 2.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329025#comment-14329025 ] Kihwal Lee commented on HDFS-7788: -- After a git pull, TestDFSHAAdminMiniCluster#testFencer failing consistently. It looks like HDFS-7813, unrelated this change. I will review HDFS-7813. Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Rushabh S Shah Priority: Blocker Attachments: HDFS-7788-binary.patch, rushabh.patch Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329036#comment-14329036 ] Hudson commented on HDFS-7788: -- FAILURE: Integrated in Hadoop-trunk-Commit #7161 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7161/]) HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) * hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Rushabh S Shah Priority: Blocker Fix For: 2.7.0 Attachments: HDFS-7788-binary.patch, rushabh.patch Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329067#comment-14329067 ] Rushabh S Shah commented on HDFS-7788: -- Thanks Kihwal for reviewing and committting !! Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Rushabh S Shah Priority: Blocker Fix For: 2.7.0 Attachments: HDFS-7788-binary.patch, rushabh.patch Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328103#comment-14328103 ] Kihwal Lee commented on HDFS-7788: -- The patch was applied fine by the precommit. https://builds.apache.org/job/PreCommit-HDFS-Build/9621 Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Assignee: Rushabh S Shah Priority: Blocker Attachments: HDFS-7788-binary.patch, rushabh.patch Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[
https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328124#comment-14328124
]
Hadoop QA commented on HDFS-7788:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699735/rushabh.patch
against trunk revision d49ae72.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9621//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9621//console
This message is automatically generated.
Post-2.6 namenode may not start up with an image containing inodes created
with an old release.
---
Key: HDFS-7788
URL: https://issues.apache.org/jira/browse/HDFS-7788
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Rushabh S Shah
Priority: Blocker
Attachments: HDFS-7788-binary.patch, rushabh.patch
Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify
arbitrarily small preferred block size for a file including 0. This was
normally done by faulty clients or failed creates, but it was possible.
Until 2.5, reading a fsimage containing inodes with 0 byte preferred block
size was allowed. So if a fsimage contained such an inode, the namenode would
come up fine. In 2.6, the preferred block size is required be 0. Because
of this change, the image that worked with 2.5 may not work with 2.6.
If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is
under this risk even if it worked fine with 2.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[
https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326721#comment-14326721
]
Hadoop QA commented on HDFS-7788:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699564/HDFS-7788-binary.patch
against trunk revision 9a3e292.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9614//console
This message is automatically generated.
Post-2.6 namenode may not start up with an image containing inodes created
with an old release.
---
Key: HDFS-7788
URL: https://issues.apache.org/jira/browse/HDFS-7788
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Assignee: Rushabh S Shah
Priority: Blocker
Attachments: HDFS-7788-binary.patch
Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify
arbitrarily small preferred block size for a file including 0. This was
normally done by faulty clients or failed creates, but it was possible.
Until 2.5, reading a fsimage containing inodes with 0 byte preferred block
size was allowed. So if a fsimage contained such an inode, the namenode would
come up fine. In 2.6, the preferred block size is required be 0. Because
of this change, the image that worked with 2.5 may not work with 2.6.
If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is
under this risk even if it worked fine with 2.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post-2.6 namenode may not start up with an image containing inodes created with an old release.
[ https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318668#comment-14318668 ] Kihwal Lee commented on HDFS-7788: -- One option is to make namenode automatically convert such inodes to have the min block size as their preferred block size. Post-2.6 namenode may not start up with an image containing inodes created with an old release. --- Key: HDFS-7788 URL: https://issues.apache.org/jira/browse/HDFS-7788 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Priority: Blocker Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible. Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be 0. Because of this change, the image that worked with 2.5 may not work with 2.6. If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7788) Post 2.6 namenode may not start up with an image containing inodes created with an old release.
[
https://issues.apache.org/jira/browse/HDFS-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318655#comment-14318655
]
Kihwal Lee commented on HDFS-7788:
--
The stack trace:
{panel}
2015-02-09 19:02:28,361 \[main\] FATAL namenode.NameNode: Failed to start
namenode.java.lang.IllegalArgumentException: Illagal value:
PREFERRED_BLOCK_SIZE = 0
MIN = 1
at
org.apache.hadoop.hdfs.util.LongBitFormat.combine(LongBitFormat.java:58)
at
org.apache.hadoop.hdfs.server.namenode.INodeFile$HeaderFormat.toLong(INodeFile.java:106)
at
org.apache.hadoop.hdfs.server.namenode.INodeFile.init(INodeFile.java:128)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeFile(FSImageFormatPBINode.java:291)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINode(FSImageFormatPBINode.java:266)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeSection(FSImageFormatPBINode.java:221)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:254)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:180)
at
org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:226)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:926)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:910)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:729)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:665)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:374)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:268)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1053)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:757)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:538)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:597)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.init(NameNode.java:764)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.init(NameNode.java:748)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1441)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1507)
2015-02-09 19:02:28,381 \[main\] INFO util.ExitUtil: Exiting with status 1
2015-02-09 19:02:28,384 \[Thread-1\] INFO namenode.NameNode: SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at
/
{panel}
Post 2.6 namenode may not start up with an image containing inodes created
with an old release.
---
Key: HDFS-7788
URL: https://issues.apache.org/jira/browse/HDFS-7788
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Kihwal Lee
Priority: Blocker
Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify
arbitrarily small preferred block size for a file including 0. This was
normally done by faulty clients or failed creates, but it was possible.
Until 2.5, reading a fsimage containing inodes with 0 byte preferred block
size was allowed. So if a fsimage contained such an inode, the namenode would
come up fine. In 2.6, the preferred block size is required be 0. Because
of this change, the image that worked with 2.5 may not work with 2.6.
If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is
under this risk even if it worked fine with 2.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
