[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257263#comment-13257263 ] Hudson commented on HBASE-5782: --- Integrated in HBase-TRUNK-security #175 (See [https://builds.apache.org/job/HBase-TRUNK-security/175/]) HBASE-5782 Edits can be appended out of seqid order since HBASE-4487 (Revision 1327673) Result = FAILURE larsh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt, > 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch, > hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257105#comment-13257105 ] Lars Hofhansl commented on HBASE-5782: -- Yes. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt, > 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch, > hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257100#comment-13257100 ] Zhihong Yu commented on HBASE-5782: --- Do we need to log another issue for trunk which would finish Todd's work ? > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt, > 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch, > hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257005#comment-13257005 ] Hudson commented on HBASE-5782: --- Integrated in HBase-0.94 #129 (See [https://builds.apache.org/job/HBase-0.94/129/]) HBASE-5782 Edits can be appended out of seqid order since HBASE-4487 (Revision 1327672) Result = FAILURE larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt, > 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch, > hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256993#comment-13256993 ] Hudson commented on HBASE-5782: --- Integrated in HBase-TRUNK #2783 (See [https://builds.apache.org/job/HBase-TRUNK/2783/]) HBASE-5782 Edits can be appended out of seqid order since HBASE-4487 (Revision 1327673) Result = FAILURE larsh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt, > 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch, > hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256978#comment-13256978 ] Hudson commented on HBASE-5782: --- Integrated in HBase-0.94-security #15 (See [https://builds.apache.org/job/HBase-0.94-security/15/]) HBASE-5782 Edits can be appended out of seqid order since HBASE-4487 (Revision 1327672) Result = SUCCESS larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt, > 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch, > hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256968#comment-13256968 ] stack commented on HBASE-5782: -- Sorry. Dumb. The tool calls system.exit. Let me fix in another issue. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt, > 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch, > hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256933#comment-13256933 ] Lars Hofhansl commented on HBASE-5782: -- Now note, that TestHLog still does not run anything locally (neither in 0.94 or trunk), there is something with that specific test it seems. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt, > 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch, > hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256788#comment-13256788 ] stack commented on HBASE-5782: -- You will need to pull in HLogPerformanceEvaluation. Copy it whole (don't do the hbase-5792 because it got mod'd a few times subsequent to commit). You could also just commit the unit test to trunk and not to 0.94; that should be fine long as we hold to committing patches to trunk first. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt, > 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch, > hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256756#comment-13256756
]
Lars Hofhansl commented on HBASE-5782:
--
For some reason I cannot run TestHLog locally. I always get:
{code}
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
{code}
I can run all other tests that I tried (including some others marked with
@LargeTests). Verified again with HLogPerformanceEvaluation manually.
Should we put HBASE-5792 in 0.94 as well, so that we use this test?
> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: Lars Hofhansl
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt,
> 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch,
> hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment.
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256236#comment-13256236 ] Hadoop QA commented on HBASE-5782: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523078/5782.unittest.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestHLog Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1562//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1562//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1562//console This message is automatically generated. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt, > 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch, > hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256220#comment-13256220 ] Hadoop QA commented on HBASE-5782: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523074/5782-v3.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1560//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1560//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1560//console This message is automatically generated. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782-v3.txt, > 5782.txt, 5782.unfinished-stack.txt, 5782.unittest.txt, HBASE-5782.patch, > hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256202#comment-13256202 ] Lars Hofhansl commented on HBASE-5782: -- I can work on a test, unless you like to Stack, or maybe you have started on it anyway. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256200#comment-13256200 ] Lars Hofhansl commented on HBASE-5782: -- Yes, we need a test for this. What scared me most about this bug was that no test caught it, and this one of *the* core parts of HBase. This test would basically just call code from HLogPerformanceEvaluation, right? > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256187#comment-13256187 ] Zhihong Yu commented on HBASE-5782: --- +1 on such a test which would prevent regression. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256185#comment-13256185 ] stack commented on HBASE-5782: -- Want me to make a test that does simple three threads with just a few edits ... say 1k... and then verifies all in order and all edits written so we notice regression? > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256176#comment-13256176 ] ramkrishna.s.vasudevan commented on HBASE-5782: --- I'm +1 on Lar's patch. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256094#comment-13256094 ] Lars Hofhansl commented on HBASE-5782: -- I'm +1 on committing on my patch for 0.94.0. We can then either revisit for 0.94.1+ or 0.96. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256087#comment-13256087 ] stack commented on HBASE-5782: -- @Lars As to your patch being 'slower' when fewer threads, I think you can't do such a compare. W/o your patch, we are broke. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256085#comment-13256085 ] stack commented on HBASE-5782: -- I tried to reproduce what JD is seeing on cluster using same sized keys and values but Lars' patch completes before Todds. My test run may be too small I did thread dumps during Lars and Todd runs. Both seem to be down in sync mostly, down here 'org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.waitForAckedSeqno(DFSClient.java:3789)' otherwise hung up on sync points around wal append/sync. Lets go w/ the Lars patch because minimal changes. As per Todd, lets file an issue to clean up this stuff with his patch as seed. From J-D work, any grease lightening we can apply around hlog append makes for a big difference in overall write throughput. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256072#comment-13256072 ] Lars Hofhansl commented on HBASE-5782: -- Did some more tests with HLogPerformanceEvaluation (and my patch): 10 threads, 10 iterations: with patch: 42s, without patch: 41s 5 threads, 20 iterations: with patch: 46s, without patch: 44s 2 threads, 20 iterations: with patch: 46s, without patch: 44s So for fewer threads my patch is slightly slower. So... What do we do? :) > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256061#comment-13256061 ] Jean-Daniel Cryans commented on HBASE-5782: --- For my part I tested on a cluster with 1kb values, huge batches, and I don't see any improvement with Lars' patch. Todd's peaks 20% higher. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256058#comment-13256058 ] Lars Hofhansl commented on HBASE-5782: -- Thanks Stack! I'll run a test with fewer threads too, just to make sure, the fact that both of our patches are faster probably means that we did a lot of unnecessary sync'ing before...? 20% performance increase is pretty damn awesome. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256044#comment-13256044 ] stack commented on HBASE-5782: -- Ok on lars patch into 0.94. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256043#comment-13256043 ] stack commented on HBASE-5782: -- I made the hlog perf tool work on hdfs and ran some basic tests. Both Todd an Lars' patches seem faster than what we have currently. Running w/o a fix on hdfs w/ current trunk I have to disable verify because it fails (verify happens after we print out test timings). $ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation -conf /home/stack/hadoop-conf/core-site.xml -path hdfs://sv4r11s38:7000/tmp -threads 100 -roll 1 12/04/17 22:58:28 INFO wal.HLogPerformanceEvaluation: Summary: threads=100, iterations=1 took 100.630s 9937.395ops/s 12/04/17 23:00:33 INFO wal.HLogPerformanceEvaluation: Summary: threads=100, iterations=1 took 94.945s 10532.413ops/s Todd patch on hdfs: $ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation -conf /home/stack/hadoop-conf/core-site.xml -path hdfs://sv4r11s38:7000/tmp -threads 100 -roll 1 -verify 12/04/17 22:53:35 INFO wal.HLogPerformanceEvaluation: Summary: threads=100, iterations=1 took 81.202s 12314.967ops/s Lars patch: 12/04/17 23:07:08 INFO wal.HLogPerformanceEvaluation: Summary: threads=100, iterations=1 took 76.800s 13020.833ops/s For Todd and Lars, both pass verify which checks that seqids are ordered and that we wrote as much as we think we did. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256030#comment-13256030 ] Todd Lipcon commented on HBASE-5782: I think for 0.94.0 we should commit Lars's, and file another JIRA to clean up / rewrite this stuff to be less circuitous. Could definitely do with a cleanup. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255974#comment-13255974 ] Lars Hofhansl commented on HBASE-5782: -- @Stack: I ran it too. It works fine. The "interesting" thing I find that it is faster *with* the patch! And that scares me. I ran with 100 threads, 1 iterations. With the patch it took 29s, without it took 43s. This is on a 6 core machine with hyper threading. Now, the HLogPerformanceEvaluation does not work with -nosync and -verify, right? (presumably because no final sync is issued). Wouldn't mind to get some other numbers from you as well if you had some time. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255959#comment-13255959 ] stack commented on HBASE-5782: -- @Lars But it own't matter right since the map we are getting from is not under our new flush lock? I think its harmless. We will undercount whats been flushed I believe; we'll not overcount (and so possible lose data)? I added log rolling and tested your patch using HLogPerformanceEvaluation. It 'works' at least. If you want me to compare before and after, just say. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255939#comment-13255939 ] Lars Hofhansl commented on HBASE-5782: -- Ah, I see. Yes, doneUpto line should be pulled into the synchronized block. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255937#comment-13255937 ] stack commented on HBASE-5782: -- bq. We won't write more into the log (once we take the pendingWrites they are gone Is that so? We don't get the pendingWrites until we are under the flush lock but we've taken doneUpTo before we go under the lock. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255911#comment-13255911 ] Lars Hofhansl commented on HBASE-5782: -- @Stack: the doneUpTo is fine, because it is only used to set syncedTillHere. We won't write more into the log (once we take the pendingWrites they are gone), but we might sync too much unnecessarily. Will do the (performance) testing now. I don't have a cluster at my disposal atm, so I'll do it with a local HDFS instance. If we'd rather finish Todd's for 0.94 that'd be nice. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255885#comment-13255885 ] stack commented on HBASE-5782: -- Can we try and make Todd's work? It does some nice cleanup. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255877#comment-13255877
]
stack commented on HBASE-5782:
--
Looking at Lars patch.
On you 1. and 2. above, apparently the append is also expensive according to
Dhruba. Just saying.
Also on "...might lead to sync be issued multiple time when only one was
necessary (it seems the same race condition existed before)."
Yes, this we have always had.
I'd say kill this stuff... it looks like rubbish to me:
{code}
+ syncBatchSize.addAndGet(doneUpto - this.syncedTillHere);
{code}
Its not read by anyone, looks like the math can go wonky, and when it is read,
its set back to zero which is probably unexpected. Kill it I'd say.
I think this is ok:
{code}
+ this.syncedTillHere = Math.max(this.syncedTillHere, doneUpto);
{code}
but this is racy
{code}
long doneUpto = this.unflushedEntries.get();
{code}
It could be low in number; i.e. we could be putting into hdfs more edits than
the current value of unflushedEntries if we read after an edit has been added
to the queue but before the above is updated. Is that ok? Its ok if this is a
little sloppy especially if it under reports?
On tactic for 0.94, sure on doing this for 0.94 though I like Todds fix better.
The verification tool will help you figure if this slows stuff much and if we
are writing out of order. Let me know if you want me to run it for you. Let
me add in log rolling too as per Todd suggestion.
> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt,
> 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment.
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255834#comment-13255834 ] Zhihong Yu commented on HBASE-5782: --- HLogPerformanceEvaluation verification is needed as well. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255832#comment-13255832 ] Todd Lipcon commented on HBASE-5782: Seems reasonable to me. Did yours pass tests, etc? > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255828#comment-13255828 ] Lars Hofhansl commented on HBASE-5782: -- Anybody opposed to do my patch for 0.94 and do a rewrite in trunk? Todd? Stack? Ted? > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255784#comment-13255784 ] Jean-Daniel Cryans commented on HBASE-5782: --- bq. The pendingWrites are appended strictly in order, so there is a very short race that might lead to sync be issued multiple time when only one was necessary (it seems the same race condition existed before). A short race is better than what we currently do in 0.92 (and before) where everything syncs everything. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255770#comment-13255770 ] Lars Hofhansl commented on HBASE-5782: -- Yep. It's crucial that syncTillHere is updated last. The pendingWrites are appended strictly in order, so there is a very short race that might lead to sync be issued multiple time when only one was necessary (it seems the same race condition existed before). So I think it is safe and low risk. The question now is: Do this for 0.94 and then a more elaborate rewrite in trunk, or do a more risky rewrite for 0.94? > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255760#comment-13255760 ] Hadoop QA commented on HBASE-5782: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522975/5782-lars-v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1551//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1551//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1551//console This message is automatically generated. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255718#comment-13255718 ] Todd Lipcon commented on HBASE-5782: Don't think you're smoking something. It looks reasonable, though I think there are still races around the update of doneUpTo, syncedTilHere, etc. These races may be "safe" in that they only result in too-low values (and thus extra needless syncs) but I'm not 100% sure of it. Maybe we can extend the verification test case to run as a stress test, including rolls, and run it overnight or something? > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-lars-v2.txt, 5782-sketch.txt, 5782.txt, > 5782.unfinished-stack.txt, HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255395#comment-13255395 ] Todd Lipcon commented on HBASE-5782: Looks like some test timeouts... will investigate tomorrow. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-sketch.txt, 5782.txt, 5782.unfinished-stack.txt, > HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255381#comment-13255381 ] Hadoop QA commented on HBASE-5782: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522913/hbase-5782.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1549//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1549//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1549//console This message is automatically generated. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-sketch.txt, 5782.txt, 5782.unfinished-stack.txt, > HBASE-5782.patch, hbase-5782.txt > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255344#comment-13255344 ] stack commented on HBASE-5782: -- @Ram Read over the HLog comments. Its got stuff on why we want sequenceids in order and where we have dependency on their being ordered, at least they are notes on how we used to think. I was wondering too about ordering today. If we didn't have to have order, then it would make stuff like running a regionserver with N WALs a bit easier, and we don't try to guarantee sequence order when replicating. But I'm wary undoing order though without our giving the issue a bunch of thought first (Your patch above makes me nervous). On the patch, Todds' seems way superior to me. His is more radical, removing what seems to be a confusing sequenceid double, and its more clear whats going on. Oh, and thanks to you fellas for finding this one. Its a good one. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-sketch.txt, 5782.txt, 5782.unfinished-stack.txt, > HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255329#comment-13255329 ] ramkrishna.s.vasudevan commented on HBASE-5782: --- @Lars Correct me if am wrong. Currently the edits are ordered atleast in the memory so that the store file that is persisted is having the proper sequence number. But it is when applying it to the WAL there is a change and the same needs to be taken care while replaying. Is there any special consideration needed that my edits have to ordered even in the WAL file? What is the design consideration behind this? May be am missing something here. @Stack This patch is one similar to the one i had in mind. Synchronize getPendingWrites and the 'writer.append(e);'. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-sketch.txt, 5782.txt, 5782.unfinished-stack.txt, > HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255308#comment-13255308 ] Lars Hofhansl commented on HBASE-5782: -- @Ram: It looks like your patch should work for this scenario, but I'd be generally wary about edits not being in order in the WAL. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-sketch.txt, 5782.txt, 5782.unfinished-stack.txt, > HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255299#comment-13255299 ] ramkrishna.s.vasudevan commented on HBASE-5782: --- Thanks to all for working on this. @JD I agree that LogSyncer thread is not the real problem, it can happen even without that. My initial thought was also to introduce some locks but finally thought that the expected parallelism cannot be obtained. That is why i went with the fix while replayingtheRecoveredEdits. I will go thro the current patch also. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-sketch.txt, 5782.txt, 5782.unfinished-stack.txt, > HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255239#comment-13255239 ] Hadoop QA commented on HBASE-5782: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522890/5782.unfinished-stack.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestLogRollingNoCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1548//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1548//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1548//console This message is automatically generated. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-sketch.txt, 5782.txt, 5782.unfinished-stack.txt, > HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255211#comment-13255211
]
Zhihong Yu commented on HBASE-5782:
---
{code}
+ synchronized (flushLock) {
+List pending;
- // write out all accumulated Entries to hdfs.
- for (Entry e : pending) {
-writer.append(e);
+synchronized (this) {
{code}
Is the second synchronized needed ?
> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-sketch.txt, 5782.txt, 5782.unfinished-stack.txt,
> HBASE-5782.patch
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment.
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255201#comment-13255201
]
stack commented on HBASE-5782:
--
Not sure I follow but I do know this patch more ambitious than what I was at
+ You remove the 'other' sequence numbering system, unflushedEntries? That
looks good.
+ Are asserts on by default? We disabled them a while back I believe? You run
w/ asserts? (Yeah, thats a good thing to test -- should you use your guava test
instead?)
+ Its ugly we call it hlogFlush but internal we do appends (thats not your
change)
+ I agree that the reset of the the pending writes linked list needs to be done
under the synchronization held by hlogFlush
+ I like how you do pushback of edits if we failin hlogFlush.
+ On this thing:
{code}
+ // TODO: restore metric syncBatchSize.addAndGet(doneUpto -
this.syncedTillHere);
{code}
Its not used anywhere and it the math looked dodgy... then when you read it it
gets set to zero so I'm not so sure it is of any use.
> Edits can be appended out of seqid order since HBASE-4487
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 5782-sketch.txt, 5782.txt, HBASE-5782.patch
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment.
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255192#comment-13255192 ] Hadoop QA commented on HBASE-5782: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522884/5782-sketch.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause mvn compile goal to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1547//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1547//console This message is automatically generated. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-sketch.txt, 5782.txt, HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255191#comment-13255191 ] Lars Hofhansl commented on HBASE-5782: -- Sketched patch looks good, making sure that we do sync' a batch of write before the previous batch is done. I was thinking about something similar, but still allowing multiple threads to write, just that a thread cannot start writing before the previous batch is confirmed sync'ed. I guess we actually wouldn't get more parallelism out that than with your approach. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782-sketch.txt, 5782.txt, HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255172#comment-13255172 ] Jean-Daniel Cryans commented on HBASE-5782: --- bq. It seems even without HLog.appendNoSync() is is possible that one threads flushes an entire batch of pending write before an thread that started earlier can get to it. We didn't have pending writes before, it was inside the sequence file writer, and we append under lock. Managing those pending writes is what's giving us trouble. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782.txt, HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255168#comment-13255168 ] Lars Hofhansl commented on HBASE-5782: -- Is this really only a problem because of HBASE-4528 and HBASE-4487? It seems even without HLog.appendNoSync() is is possible that one threads flushes an entire batch of pending write before an thread that started earlier can get to it. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782.txt, HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Edits can be appended out of seqid order since HBASE-4487
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255135#comment-13255135 ] Hadoop QA commented on HBASE-5782: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522867/5782.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1545//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1545//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1545//console This message is automatically generated. > Edits can be appended out of seqid order since HBASE-4487 > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782.txt, HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
