[
https://issues.apache.org/jira/browse/HBASE-16644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537332#comment-15537332
]
Hadoop QA commented on HBASE-16644:
-----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m 21s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
56s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} branch-1.3 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} branch-1.3 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} branch-1.3 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} branch-1.3 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 38s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 104m 26s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 161m 26s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat |
| | hadoop.hbase.regionserver.TestFailedAppendAndSync |
| | hadoop.hbase.mapreduce.TestMultiTableSnapshotInputFormat |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-09-30 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831158/HBASE-16644.branch-1.3.patch
|
| JIRA Issue | HBASE-16644 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 87a2819b8894 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/hbase.sh |
| git revision | branch-1.3 / 9f36408 |
| Default Java | 1.7.0_111 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_101
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/3787/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/3787/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3787/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3787/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Errors when reading legit HFile' Trailer on branch 1.3
> ------------------------------------------------------
>
> Key: HBASE-16644
> URL: https://issues.apache.org/jira/browse/HBASE-16644
> Project: HBase
> Issue Type: Bug
> Components: HFile
> Affects Versions: 1.3.0, 1.4.0
> Reporter: Mikhail Antonov
> Assignee: Mikhail Antonov
> Priority: Critical
> Fix For: 1.3.0
>
> Attachments: HBASE-16644.branch-1.3.patch,
> HBASE-16644.branch-1.3.patch
>
>
> There seems to be a regression in branch 1.3 where we can't read HFile
> trailer(getting "CorruptHFileException: Problem reading HFile Trailer") on
> some HFiles that could be successfully read on 1.2.
> I've seen this error manifesting in two ways so far.
> {code}Caused by: org.apache.hadoop.hbase.io.hfile.CorruptHFileException:
> Problem reading HFile Trailer from file <file name>
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1164)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:259)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:427)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:528)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:518)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:652)
> at
> org.apache.hadoop.hbase.regionserver.HStore.access$000(HStore.java:117)
> at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:519)
> at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:516)
> ... 6 more
> Caused by: java.io.IOException: Invalid HFile block magic:
> \x00\x00\x04\x00\x00\x00\x00\x00
> at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:155)
> at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:167)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock.<init>(HFileBlock.java:344)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1735)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1558)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlock(HFileBlock.java:1397)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlockWithBlockType(HFileBlock.java:1405)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.<init>(HFileReaderV2.java:156)
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:485)
> {code}
> and second
> {code}
> Caused by: org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem
> reading HFile Trailer from file <file path>
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1164)
> at
> org.apache.hadoop.hbase.io.HalfStoreFileReader.<init>(HalfStoreFileReader.java:104)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:256)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:427)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:528)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:518)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:652)
> at
> org.apache.hadoop.hbase.regionserver.HStore.access$000(HStore.java:117)
> at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:519)
> at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:516)
> ... 6 more
> Caused by: java.io.IOException: Premature EOF from inputStream (read returned
> -1, was trying to read 10083 necessary bytes and 24 extra bytes, successfully
> read 1072
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock.readWithExtra(HFileBlock.java:737)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1459)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1712)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1558)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlock(HFileBlock.java:1397)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlockWithBlockType(HFileBlock.java:1405)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.<init>(HFileReaderV2.java:156)
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:485)
> {code}
> In my case this problem was reproducible by running `hbase hfile -m -f`
> command. There seem to be two changes in behavior introduced by HBASE-15477
> (cc [~stack]).
> One is that HFileBlock#getOnDiskSizeWithHeader always assumes that checksum
> verification is true (this behavior results in onDiskSizeWithHeader being
> calculated differently in 1.2 and 1.3 for some cases).
> Second is that in HFileBlock constructor we always attempt to retrive
> checksum-related fields from the header even if no checksum verification is
> going on.
> Attached is the patch which when applied allows 1.3 to read again the same
> HFiles.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
