[
https://issues.apache.org/jira/browse/HBASE-11620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081516#comment-14081516
]
Hadoop QA commented on HBASE-11620:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12658969/11620-v2.txt
against trunk revision .
ATTACHMENT ID: 12658969
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestIOFencing
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
org.apache.hadoop.hbase.regionserver.TestRegionReplicas
org.apache.hadoop.hbase.master.TestMasterOperationsForRegionReplicas
org.apache.hadoop.hbase.client.TestReplicasClient
org.apache.hadoop.hbase.master.TestRestartCluster
org.apache.hadoop.hbase.TestRegionRebalancing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10248//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10248//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10248//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10248//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10248//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10248//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10248//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10248//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10248//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10248//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10248//console
This message is automatically generated.
> Record the class name of Writer in WAL header so that only proper Reader can
> open the WAL file
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-11620
> URL: https://issues.apache.org/jira/browse/HBASE-11620
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.98.4
> Reporter: Ted Yu
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.99.0, 0.98.5, 2.0.0
>
> Attachments: 11620-v1.txt, 11620-v2.txt, 11620-v3.txt
>
>
> Reported by Kiran in this thread: "HBase file encryption, inconsistencies
> observed and data loss"
> After step 4 ( i.e disabling of WAL encryption, removing
> SecureProtobufReader/Writer and restart), read of encrypted WAL fails mainly
> due to EOF exception at Basedecoder. This is not considered as error and
> these WAL are being moved to /oldWALs.
> Following is observed in log files:
> {code}
> 2014-07-30 19:44:29,254 INFO [RS_LOG_REPLAY_OPS-HOST-16:15264-1]
> wal.HLogSplitter: Splitting hlog:
> hdfs://HOST-16:18020/hbase/WALs/HOST-16,15264,1406725441997-splitting/HOST-16%2C15264%2C1406725441997.1406725444017,
> length=172
> 2014-07-30 19:44:29,254 INFO [RS_LOG_REPLAY_OPS-HOST-16:15264-1]
> wal.HLogSplitter: DistributedLogReplay = false
> 2014-07-30 19:44:29,313 INFO [RS_LOG_REPLAY_OPS-HOST-16:15264-1]
> util.FSHDFSUtils: Recovering lease on dfs file
> hdfs://HOST-16:18020/hbase/WALs/HOST-16,15264,1406725441997-splitting/HOST-16%2C15264%2C1406725441997.1406725444017
> 2014-07-30 19:44:29,315 INFO [RS_LOG_REPLAY_OPS-HOST-16:15264-1]
> util.FSHDFSUtils: recoverLease=true, attempt=0 on
> file=hdfs://HOST-16:18020/hbase/WALs/HOST-16,15264,1406725441997-splitting/HOST-16%2C15264%2C1406725441997.1406725444017
> after 1ms
> 2014-07-30 19:44:29,429 DEBUG [RS_LOG_REPLAY_OPS-HOST-16:15264-1-Writer-0]
> wal.HLogSplitter: Writer thread
> Thread[RS_LOG_REPLAY_OPS-HOST-16:15264-1-Writer-0,5,main]: starting
> 2014-07-30 19:44:29,429 DEBUG [RS_LOG_REPLAY_OPS-HOST-16:15264-1-Writer-1]
> wal.HLogSplitter: Writer thread
> Thread[RS_LOG_REPLAY_OPS-HOST-16:15264-1-Writer-1,5,main]: starting
> 2014-07-30 19:44:29,430 DEBUG [RS_LOG_REPLAY_OPS-HOST-16:15264-1-Writer-2]
> wal.HLogSplitter: Writer thread
> Thread[RS_LOG_REPLAY_OPS-HOST-16:15264-1-Writer-2,5,main]: starting
> 2014-07-30 19:44:29,591 ERROR [RS_LOG_REPLAY_OPS-HOST-16:15264-1]
> codec.BaseDecoder: Partial cell read caused by EOF: java.io.IOException:
> Premature EOF from inputStream
> 2014-07-30 19:44:29,592 INFO [RS_LOG_REPLAY_OPS-HOST-16:15264-1]
> wal.HLogSplitter: Finishing writing output logs and closing down.
> 2014-07-30 19:44:29,592 INFO [RS_LOG_REPLAY_OPS-HOST-16:15264-1]
> wal.HLogSplitter: Waiting for split writer threads to finish
> 2014-07-30 19:44:29,592 INFO [RS_LOG_REPLAY_OPS-HOST-16:15264-1]
> wal.HLogSplitter: Split writers finished
> 2014-07-30 19:44:29,592 INFO [RS_LOG_REPLAY_OPS-HOST-16:15264-1]
> wal.HLogSplitter: Processed 0 edits across 0 regions; log
> file=hdfs://HOST-16:18020/hbase/WALs/HOST-16,15264,1406725441997-splitting/HOST-16%2C15264%2C1406725441997.1406725444017
> is corrupted = false progress failed = false
> {code}
> To fix this, we need to propagate EOF exception to HLogSplitter. Any
> suggestions on the fix?
> -------- (end of quote from Kiran)
> In BaseDecoder#rethrowEofException() :
> {code}
> if (!isEof) throw ioEx;
> LOG.error("Partial cell read caused by EOF: " + ioEx);
> EOFException eofEx = new EOFException("Partial cell read");
> eofEx.initCause(ioEx);
> throw eofEx;
> {code}
> throwing EOFException would not propagate the "Partial cell read" condition
> to HLogSplitter which doesn't treat EOFException as an error.
> I think IOException should be thrown above - HLogSplitter#getNextLogLine()
> would translate the IOEx to CorruptedLogFileException.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
