[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500108#comment-13500108 ] Cosmin Lehene commented on HBASE-2645: -- Wow, I think it was one of the longest lived blockers. Glad to see it fixed. Thanks :) HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645-addendum.txt, 2645_hacking.txt, 2645.txt, 2645v2.txt, 2645v3.txt, 2645v4.txt, hdfs_1.0_editswriter_recoverlease.txt, hdfs_trunk_editswriter_recoverlease.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499395#comment-13499395 ] Hudson commented on HBASE-2645: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #266 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/266/]) HBASE-2645 HLog writer can do 1-2 sync operations after lease has been recovered for split process (Revision 1410651) HBASE-2645 HLog writer can do 1-2 sync operations after lease has been recovered for split process (Revision 1410631) Result = SUCCESS stack : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java stack : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645-addendum.txt, 2645_hacking.txt, 2645.txt, 2645v2.txt, 2645v3.txt, 2645v4.txt, hdfs_1.0_editswriter_recoverlease.txt, hdfs_trunk_editswriter_recoverlease.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499186#comment-13499186 ] stack commented on HBASE-2645: -- Yeah, looks like the reenabled test works. It just is taking a while to complete because the mocked regionserver thread is taking its time dying. I made HDFS-4203 for failing faster (after Todd helped out on what is going on). Let me run it a few times local to makes sure it good. HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645_hacking.txt, 2645.txt, 2645v2.txt, 2645v3.txt, hdfs_1.0_editswriter_recoverlease.txt, hdfs_trunk_editswriter_recoverlease.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499246#comment-13499246
]
Hadoop QA commented on HBASE-2645:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12553853/2645v4.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
99 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 24 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3355//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3355//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3355//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3355//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3355//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3355//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3355//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3355//console
This message is automatically generated.
HLog writer can do 1-2 sync operations after lease has been recovered for
split process.
Key: HBASE-2645
URL: https://issues.apache.org/jira/browse/HBASE-2645
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 0.90.4
Reporter: Cosmin Lehene
Assignee: Todd Lipcon
Priority: Blocker
Fix For: 0.96.0
Attachments: 2645_hacking.txt, 2645.txt, 2645v2.txt, 2645v3.txt,
2645v4.txt, hdfs_1.0_editswriter_recoverlease.txt,
hdfs_trunk_editswriter_recoverlease.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt
TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing.
This test starts a thread that writes one edit to the log, syncs and counts.
During this, a HLog.splitLog operation is started. splitLog recovers the log
lease before reading the log, so that the original regionserver could not
wake up and write after the split process started.
The test compares the number of edits reported by the split process and by
the writer thread. Writer thread (called zombie in the test) should report =
than the splitLog (sync() might raise after the last edit gets written and
the edit won't get counted by zombie thread). However it appears that the
zombie counts 1-2 more edits. So it looks like it can sync without a lease.
This might be a hdfs-0.20 related issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499262#comment-13499262
]
Ted Yu commented on HBASE-2645:
---
Might be a typo:
{code}
+ZOMBIE = User.getCurrent().getName() + -robber;
{code}
HLog writer can do 1-2 sync operations after lease has been recovered for
split process.
Key: HBASE-2645
URL: https://issues.apache.org/jira/browse/HBASE-2645
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 0.90.4
Reporter: Cosmin Lehene
Assignee: Todd Lipcon
Priority: Blocker
Fix For: 0.96.0
Attachments: 2645_hacking.txt, 2645.txt, 2645v2.txt, 2645v3.txt,
2645v4.txt, hdfs_1.0_editswriter_recoverlease.txt,
hdfs_trunk_editswriter_recoverlease.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt
TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing.
This test starts a thread that writes one edit to the log, syncs and counts.
During this, a HLog.splitLog operation is started. splitLog recovers the log
lease before reading the log, so that the original regionserver could not
wake up and write after the split process started.
The test compares the number of edits reported by the split process and by
the writer thread. Writer thread (called zombie in the test) should report =
than the splitLog (sync() might raise after the last edit gets written and
the edit won't get counted by zombie thread). However it appears that the
zombie counts 1-2 more edits. So it looks like it can sync without a lease.
This might be a hdfs-0.20 related issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499275#comment-13499275 ] stack commented on HBASE-2645: -- Doesn't seem used. Applied addendum to address above comment. HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645-addendum.txt, 2645_hacking.txt, 2645.txt, 2645v2.txt, 2645v3.txt, 2645v4.txt, hdfs_1.0_editswriter_recoverlease.txt, hdfs_trunk_editswriter_recoverlease.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493440#comment-13493440 ] stack commented on HBASE-2645: -- Hmmm... this patch sortof works but I want to learn more about this 40 second hang in writer. Running more tests. HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645_hacking.txt, 2645.txt, 2645v2.txt, 2645v3.txt, hdfs_1.0_editswriter_recoverlease.txt, hdfs_trunk_editswriter_recoverlease.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491291#comment-13491291 ] stack commented on HBASE-2645: -- Been playing w/ this one. I can't reproduce in hdfs on trunk or on 1.0 branch (rough patches attached). If after calling recoverFileLease, I open the file and keep reading to the end, counting edits, usually I do it once and the number of edits is constant, even if I open the file again and read. Other times I see the number of edits climb w/ each opening of the file, often over seconds. HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645.txt, 2645v2.txt, 2645v3.txt, hdfs_1.0_editswriter_recoverlease.txt, hdfs_trunk_editswriter_recoverlease.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491727#comment-13491727 ] stack commented on HBASE-2645: -- Adding logging, I can see the 'master' return from the recover file lease call against the namenode. We then go to open the 'wal' to read the edits out of it splitting. Meantime, I see that the writer thread is still progressing adding edits until it stalls in a sync call. This means the read will miss edits (lots in this test case). HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645.txt, 2645v2.txt, 2645v3.txt, hdfs_1.0_editswriter_recoverlease.txt, hdfs_trunk_editswriter_recoverlease.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13490226#comment-13490226 ] Hudson commented on HBASE-2645: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #248 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/248/]) HBASE-2645 HLog writer can do 1-2 sync operations after lease has been recovered for split process; REVERT -- TEST FAILS (Revision 1404875) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645.txt, 2645v2.txt, 2645v3.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489455#comment-13489455 ] Hudson commented on HBASE-2645: --- Integrated in HBase-TRUNK #3508 (See [https://builds.apache.org/job/HBase-TRUNK/3508/]) HBASE-2645 HLog writer can do 1-2 sync operations after lease has been recovered for split process; REVERT -- TEST FAILS (Revision 1404875) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645.txt, 2645v2.txt, 2645v3.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489030#comment-13489030 ] stack commented on HBASE-2645: -- [~ted_yu], your repeat of info in here, info that is public and available to all is of no help; its just spam in our mailboxes. Neither is your cut-off paste of a failure on your local machine w/ no context such as what your local machine is, if it fails always, etc. I committed the patch because it passed locally on repeated runs and passed hadoopqa. HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645.txt, 2645v2.txt, 2645v3.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489041#comment-13489041
]
Ted Yu commented on HBASE-2645:
---
Here is information about my environment:
{code}
java version 1.6.0_35
Java(TM) SE Runtime Environment (build 1.6.0_35-b10-428-11M3811)
Java HotSpot(TM) 64-Bit Server VM (build 20.10-b01-428, mixed mode)
Darwin T32 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012;
root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64
{code}
Initially I wasn't sure whether the test failure only happened with JDK 1.6
Later I saw the test failure on Jenkins where JDK 1.7 was used.
HLog writer can do 1-2 sync operations after lease has been recovered for
split process.
Key: HBASE-2645
URL: https://issues.apache.org/jira/browse/HBASE-2645
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 0.90.4
Reporter: Cosmin Lehene
Assignee: Todd Lipcon
Priority: Blocker
Fix For: 0.96.0
Attachments: 2645.txt, 2645v2.txt, 2645v3.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt
TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing.
This test starts a thread that writes one edit to the log, syncs and counts.
During this, a HLog.splitLog operation is started. splitLog recovers the log
lease before reading the log, so that the original regionserver could not
wake up and write after the split process started.
The test compares the number of edits reported by the split process and by
the writer thread. Writer thread (called zombie in the test) should report =
than the splitLog (sync() might raise after the last edit gets written and
the edit won't get counted by zombie thread). However it appears that the
zombie counts 1-2 more edits. So it looks like it can sync without a lease.
This might be a hdfs-0.20 related issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489252#comment-13489252
]
stack commented on HBASE-2645:
--
It fails for me too locally if I keep running it. If I do the below, it
usually passes but it can also fail:
{code}
$ for i in `seq 0 9`; do echo here=$i; mvn test
-Dtest=TestHLogSplit#testLogCannotBeWrittenOnceParsed -PlocalTests; mv
hbase-server/target/surefire-reports hbase-server/target/$i; done
/tmp/test.txt
{code}
I tried w/ 1.0.4 and it also fails.
Will revert for now. If a bug in HDFS, will file an issue.
HLog writer can do 1-2 sync operations after lease has been recovered for
split process.
Key: HBASE-2645
URL: https://issues.apache.org/jira/browse/HBASE-2645
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 0.90.4
Reporter: Cosmin Lehene
Assignee: Todd Lipcon
Priority: Blocker
Fix For: 0.96.0
Attachments: 2645.txt, 2645v2.txt, 2645v3.txt,
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt
TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing.
This test starts a thread that writes one edit to the log, syncs and counts.
During this, a HLog.splitLog operation is started. splitLog recovers the log
lease before reading the log, so that the original regionserver could not
wake up and write after the split process started.
The test compares the number of edits reported by the split process and by
the writer thread. Writer thread (called zombie in the test) should report =
than the splitLog (sync() might raise after the last edit gets written and
the edit won't get counted by zombie thread). However it appears that the
zombie counts 1-2 more edits. So it looks like it can sync without a lease.
This might be a hdfs-0.20 related issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489256#comment-13489256 ] Ted Yu commented on HBASE-2645: --- Thanks for the verification, Stack. Appreciate it. HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645.txt, 2645v2.txt, 2645v3.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487935#comment-13487935
]
Ted Yu commented on HBASE-2645:
---
{code}
r1403977 | stack | 2012-10-30 21:48:37 -0700 (Tue, 30 Oct 2012) | 1 line
HLog writer can do 1-2 sync operations after lease has been recovered for split
process.
{code}
I don't see JIRA number in the checkin message.
Second, in
https://builds.apache.org/job/PreCommit-HBASE-Build/3197//testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testLogCannotBeWrittenOnceParsed/
:
{code}
Error Message
The log file could have at most 1 extra log entry, but can't have less. Zombie
could write 6559 and logfile had only 6520 in
logfile=hdfs://localhost:44865/hbase/t1/bbb/recovered.edits/001
Stacktrace
java.lang.AssertionError: The log file could have at most 1 extra log entry,
but can't have less. Zombie could write 6559 and logfile had only 6520 in
logfile=hdfs://localhost:44865/hbase/t1/bbb/recovered.edits/001
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit.testLogCannotBeWrittenOnceParsed(TestHLogSplit.java:190)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
{code}
HLog writer can do 1-2 sync operations after lease has been recovered for
split process.
Key: HBASE-2645
URL: https://issues.apache.org/jira/browse/HBASE-2645
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 0.90.4
Reporter: Cosmin Lehene
Assignee: Todd Lipcon
Priority: Blocker
Fix For: 0.96.0
Attachments: 2645.txt, 2645v2.txt, 2645v3.txt
TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing.
This test starts a thread that writes one edit to the log, syncs and counts.
During this, a HLog.splitLog operation is started. splitLog recovers the log
lease before reading the log, so that the original regionserver could not
wake up and write after the split process started.
The test compares the number of edits reported by the split process and by
the writer thread. Writer thread (called zombie in the test) should report =
than the splitLog (sync() might raise after the last edit gets written and
the edit won't get counted by zombie thread). However it appears that the
zombie counts 1-2 more edits. So it looks like it can sync without a lease.
This might be a hdfs-0.20 related issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13488324#comment-13488324 ] Ted Yu commented on HBASE-2645: --- Another test failure: https://builds.apache.org/job/PreCommit-HBASE-Build/3204/testReport/junit/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testLogCannotBeWrittenOnceParsed/ HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645.txt, 2645v2.txt, 2645v3.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487328#comment-13487328
]
Hadoop QA commented on HBASE-2645:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551421/2645v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
83 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestShell
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3186//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3186//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3186//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3186//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3186//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3186//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3186//console
This message is automatically generated.
HLog writer can do 1-2 sync operations after lease has been recovered for
split process.
Key: HBASE-2645
URL: https://issues.apache.org/jira/browse/HBASE-2645
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 0.90.4
Reporter: Cosmin Lehene
Assignee: Todd Lipcon
Priority: Blocker
Fix For: 0.96.0
Attachments: 2645.txt, 2645v2.txt
TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing.
This test starts a thread that writes one edit to the log, syncs and counts.
During this, a HLog.splitLog operation is started. splitLog recovers the log
lease before reading the log, so that the original regionserver could not
wake up and write after the split process started.
The test compares the number of edits reported by the split process and by
the writer thread. Writer thread (called zombie in the test) should report =
than the splitLog (sync() might raise after the last edit gets written and
the edit won't get counted by zombie thread). However it appears that the
zombie counts 1-2 more edits. So it looks like it can sync without a lease.
This might be a hdfs-0.20 related issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487344#comment-13487344
]
Ted Yu commented on HBASE-2645:
---
Partial change from another JIRA seemed to get into patch v2:
{code}
+define_test create should be able to set table options do
{code}
HLog writer can do 1-2 sync operations after lease has been recovered for
split process.
Key: HBASE-2645
URL: https://issues.apache.org/jira/browse/HBASE-2645
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 0.90.4
Reporter: Cosmin Lehene
Assignee: Todd Lipcon
Priority: Blocker
Fix For: 0.96.0
Attachments: 2645.txt, 2645v2.txt
TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing.
This test starts a thread that writes one edit to the log, syncs and counts.
During this, a HLog.splitLog operation is started. splitLog recovers the log
lease before reading the log, so that the original regionserver could not
wake up and write after the split process started.
The test compares the number of edits reported by the split process and by
the writer thread. Writer thread (called zombie in the test) should report =
than the splitLog (sync() might raise after the last edit gets written and
the edit won't get counted by zombie thread). However it appears that the
zombie counts 1-2 more edits. So it looks like it can sync without a lease.
This might be a hdfs-0.20 related issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487549#comment-13487549
]
Hadoop QA commented on HBASE-2645:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12551477/2645v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3194//console
This message is automatically generated.
HLog writer can do 1-2 sync operations after lease has been recovered for
split process.
Key: HBASE-2645
URL: https://issues.apache.org/jira/browse/HBASE-2645
Project: HBase
Issue Type: Bug
Components: Filters
Affects Versions: 0.90.4
Reporter: Cosmin Lehene
Assignee: Todd Lipcon
Priority: Blocker
Fix For: 0.96.0
Attachments: 2645.txt, 2645v2.txt, 2645v3.txt
TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing.
This test starts a thread that writes one edit to the log, syncs and counts.
During this, a HLog.splitLog operation is started. splitLog recovers the log
lease before reading the log, so that the original regionserver could not
wake up and write after the split process started.
The test compares the number of edits reported by the split process and by
the writer thread. Writer thread (called zombie in the test) should report =
than the splitLog (sync() might raise after the last edit gets written and
the edit won't get counted by zombie thread). However it appears that the
zombie counts 1-2 more edits. So it looks like it can sync without a lease.
This might be a hdfs-0.20 related issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13070384#comment-13070384 ] vamshi commented on HBASE-2645: --- Hi Todd,may be in this place this question is irrelevant. But please let me know whether we can implement a distributed hashing in HBase for fast lookup/ scanning purpose?? i want to implement scalable data structure i.e DHT in Hbase, for that how can i proceed? Thank you. HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.94.0 TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12918185#action_12918185 ] ryan rawson commented on HBASE-2645: Dhruba can you please commit the proper patch to 0.20-append so we can close this issue? In the mean time we'll move it to 0.92 since many people are running cdh3b2. HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: io Affects Versions: 0.92.0 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.90.0 TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12902144#action_12902144 ] Jean-Daniel Cryans commented on HBASE-2645: --- Are we actually going to have this in time for 0.90? If not, punt? HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: io Affects Versions: 0.90.0 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.90.0 TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12875686#action_12875686 ] Andrew Purtell commented on HBASE-2645: --- We need for it to pass on the 0.20-append branch. HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: io Affects Versions: 0.21.0 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.21.0 TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12875689#action_12875689 ] Todd Lipcon commented on HBASE-2645: Right, the linked HDFS JIRA is slated for append 20 branch. Dhruba will hopefully commit it there after he gets in the rest of the patches. HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: io Affects Versions: 0.21.0 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.21.0 TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12875455#action_12875455 ] Todd Lipcon commented on HBASE-2645: This passes on my current snapshot of CDH3b2 available in my maven repo. If you edit pom.xml to enable snapshots for my apache.org repository, and change the hadoop version to 0.20.2-CDH3b2-SNAPSHOT you can verify. HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: io Affects Versions: 0.21.0 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.21.0 TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
