[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159794#comment-13159794 ] Hudson commented on HBASE-4838: --- Integrated in HBase-0.92 #163 (See [https://builds.apache.org/job/HBase-0.92/163/]) HBASE-4838 addendum, missing file larsh : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MultiVersionConsistencyControl.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/ReadWriteConsistencyControl.java Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v3.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13158053#comment-13158053 ] stack commented on HBASE-4838: -- I'm good on it going in. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v3.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13158086#comment-13158086 ] Lars Hofhansl commented on HBASE-4838: -- Whoops, missing adding MultiVersionConsistencyControl.java and removing ReadWriteConsistencyControl.java. Fixed in addendum patch. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v3.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13158159#comment-13158159 ] Hudson commented on HBASE-4838: --- Integrated in HBase-0.92-security #19 (See [https://builds.apache.org/job/HBase-0.92-security/19/]) HBASE-4838 addendum, missing file larsh : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MultiVersionConsistencyControl.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/ReadWriteConsistencyControl.java Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v3.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13157537#comment-13157537 ] stack commented on HBASE-4838: -- Wahoo!! +1 on commit to trunk and 0.92 Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v3.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13157538#comment-13157538 ] stack commented on HBASE-4838: -- The testClosing is same as I'm seeing on ubuntu? Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v3.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13157554#comment-13157554 ] Lars Hofhansl commented on HBASE-4838: -- @Stack: not sure. I do not have that run's output anymore. But when I run it again (multiple times) it passes (as long as I keep moving the mouse around - see HBASE-4874) And this is in trunk already, only for 0.92. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v3.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13157625#comment-13157625 ] Lars Hofhansl commented on HBASE-4838: -- So everybody's fine if I committed this now? Or should I wait until Monday to get more folks to look at it? Personally, I think this can go in. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v3.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13157653#comment-13157653 ] Jonathan Hsieh commented on HBASE-4838: --- @Lars: +1 lgtm. I didn't do a deep code review but I applied v3 and tested the TestAcidGuarantees ran it 20 times, and also ran the failures enumerated in HBASE-2856 they all pass. (Wow, the diff between v1 and v3 is pretty subtle.) Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v3.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13157673#comment-13157673 ] Lars Hofhansl commented on HBASE-4838: -- Thanks Jon. v3 passes all tests (see above), I'll do a bit more sanity checking, and commit this some time tomorrow. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v3.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13157345#comment-13157345 ] Lars Hofhansl commented on HBASE-4838: -- Finally found the problem!! It was indeed a problem in HalfStoreFileReader (as Todd and Stack have suggested all along). HalfStoreFileReader did not have a getScanner(final boolean cacheBlocks, final boolean pread, final boolean isCompaction) method, and hence the super method was called returning a normal reader. bq. Reference.java and HalfStoreFileReader.java are identical between 0.92 and trunk I don't know what I was comparing here (must have been late in the night), but they are different and that difference was exactly the problem. Could have saved myself about 16 hours of debugging. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13157347#comment-13157347 ] Ted Yu commented on HBASE-4838: --- Superb, Lars. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v2.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13157349#comment-13157349 ] Lars Hofhansl commented on HBASE-4838: -- Running all tests now. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt, 4838-v3.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[
https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13157362#comment-13157362
]
Lars Hofhansl commented on HBASE-4838:
--
Looking pretty good...
{noformat}
Results :
Failed tests: testClosing(org.apache.hadoop.hbase.client.TestHCM)
testWorkerAbort(org.apache.hadoop.hbase.master.TestDistributedLogSplitting):
expected:1 but was:0
Tests run: 1065, Failures: 2, Errors: 0, Skipped: 7
{noformat}
testClosing is unrelated.
And a 2nd run for for distributed log splitting:
{noformat}
Running org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 115.653 sec
Results :
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
{noformat}
Port 2856 (TestAcidGuarantee is failing) to 0.92
Key: HBASE-4838
URL: https://issues.apache.org/jira/browse/HBASE-4838
Project: HBase
Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.92.0
Attachments: 4838-v1.txt, 4838-v3.txt
Moving back port into a separate issue (as suggested by JonH), because this
not trivial.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[
https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13157000#comment-13157000
]
Lars Hofhansl commented on HBASE-4838:
--
I pinpointed the difference to the compactions of the daughters (again with
just 2 keys):
in 0.92 (with this patch) I see this for the 1st daughter region (which is
compacted last):
{noformat}
2011-11-24 22:08:51,324 INFO
[RegionServer:2;localhost,42385,1322201325234-smallCompactions-1322201331230]
regionserver.HRegion(1012): Starting compaction on testFamily in region
testFilterAcrossMutlipleRegions,,1322201330936.0db66f8aabdf138dbbcf6c04f857c284.
2011-11-24 22:08:51,332 INFO
[RegionServer:2;localhost,42385,1322201325234-smallCompactions-1322201331230]
regionserver.Store(725): Starting compaction of 1 file(s) in testFamily of
testFilterAcrossMutlipleRegions,,1322201330936.0db66f8aabdf138dbbcf6c04f857c284.
into
tmpdir=hdfs://localhost:52206/user/lars/testFilterAcrossMutlipleRegions/0db66f8aabdf138dbbcf6c04f857c284/.tmp,
seqid=3, totalSize=662.0
2011-11-24 22:08:51,333 DEBUG
[RegionServer:2;localhost,42385,1322201325234-smallCompactions-1322201331230]
regionserver.Store(1174): Compacting
hdfs://localhost:52206/user/lars/testFilterAcrossMutlipleRegions/0db66f8aabdf138dbbcf6c04f857c284/testFamily/85a0a11b15a248c69e09e44e0e9e052e.4e293f99103a49243c16eb104996554b-hdfs://localhost:52206/user/lars/testFilterAcrossMutlipleRegions/4e293f99103a49243c16eb104996554b/testFamily/85a0a11b15a248c69e09e44e0e9e052e-bottom,
keycount=2, bloomtype=NONE, size=662.0
2011-11-24 22:08:51,388 INFO
[RegionServer:2;localhost,42385,1322201325234-smallCompactions-1322201331230]
regionserver.Store(1322): Renaming compacted file at
hdfs://localhost:52206/user/lars/testFilterAcrossMutlipleRegions/0db66f8aabdf138dbbcf6c04f857c284/.tmp/7e7f4acb121e4696bd3c7d64e26a66b9
to
hdfs://localhost:52206/user/lars/testFilterAcrossMutlipleRegions/0db66f8aabdf138dbbcf6c04f857c284/testFamily/7e7f4acb121e4696bd3c7d64e26a66b9
2011-11-24 22:08:51,402 INFO
[RegionServer:2;localhost,42385,1322201325234-smallCompactions-1322201331230]
regionserver.Store(746): Completed major compaction of 1 file(s) in testFamily
of
testFilterAcrossMutlipleRegions,,1322201330936.0db66f8aabdf138dbbcf6c04f857c284.
into 7e7f4acb121e4696bd3c7d64e26a66b9, size=662.0; total size for store is
662.0
{noformat}
in trunk I see this for the 1st daughter region:
{noformat}
2011-11-24 22:15:18,205 INFO
[RegionServer:0;localhost,46427,1322201712357-smallCompactions-1322201718071]
regionserver.HRegion(1097): Starting compaction on testFamily in region
testFilterAcrossMutlipleRegions,,1322201717807.2bdeac6934712efdd694ec44ae48d1b2.
2011-11-24 22:15:18,206 INFO
[RegionServer:0;localhost,46427,1322201712357-smallCompactions-1322201718071]
regionserver.Store(797): Starting compaction of 1 file(s) in testFamily of
testFilterAcrossMutlipleRegions,,1322201717807.2bdeac6934712efdd694ec44ae48d1b2.
into
tmpdir=hdfs://localhost:37213/user/lars/testFilterAcrossMutlipleRegions/2bdeac6934712efdd694ec44ae48d1b2/.tmp,
seqid=3, totalSize=718.0
2011-11-24 22:15:18,206 DEBUG
[RegionServer:0;localhost,46427,1322201712357-smallCompactions-1322201718071]
regionserver.Store(1255): Compacting
hdfs://localhost:37213/user/lars/testFilterAcrossMutlipleRegions/2bdeac6934712efdd694ec44ae48d1b2/testFamily/64908313825b4c0599b86c26b33797e3.215be88f57f1ca63b6ead035b39c4d2e-hdfs://localhost:37213/user/lars/testFilterAcrossMutlipleRegions/215be88f57f1ca63b6ead035b39c4d2e/testFamily/64908313825b4c0599b86c26b33797e3-bottom,
keycount=2, bloomtype=NONE, size=718.0
2011-11-24 22:15:18,211 INFO
[RegionServer:0;localhost,46427,1322201712357-smallCompactions-1322201718071]
regionserver.Store(818): Completed major compaction of 1 file(s) in testFamily
of
testFilterAcrossMutlipleRegions,,1322201717807.2bdeac6934712efdd694ec44ae48d1b2.
into none, size=none; total size for store is 0.0
{noformat}
The keys in both cases are aaa and aab and the split key is aaa, so the 1st
region (''-'aaa') should indeed be empty after compaction. In trunk it is
correctly compacted to an empty file.
In 0.92 it somehow wrote out the entire file again (so the keys are found in
the store files for both regions).
Port 2856 (TestAcidGuarantee is failing) to 0.92
Key: HBASE-4838
URL: https://issues.apache.org/jira/browse/HBASE-4838
Project: HBase
Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.92.0
Attachments: 4838-v1.txt
Moving back port into a separate issue (as suggested by JonH), because this
not trivial.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13156319#comment-13156319 ] Lars Hofhansl commented on HBASE-4838: -- With the above scenario what I found is this: o the table is populated with only two KV: aaa and aab. o after the split there two regions: ['', aaa) and [aaa,'') x the client scanner first tries the 1st region o then it tries the 2nd region The X is where the difference is. In trunk (and unpatched 0.92), the region's internal scanner finds no KVs (as it should) and returns an empty result to the client scanner, which then proceeds to the next region. In 0.92 with this patch, the region's internal scanner actually finds both aaa and aab in the 1st region (which is wrong), and then again the 2nd region (which is correct). I don't know, yet, why this is happening, though. Maybe the scanner picks up the wrong store files, or there a problem with flushes or compactions. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13156350#comment-13156350 ] Todd Lipcon commented on HBASE-4838: Maybe a problem with the HalfHFile references? After a compaction of the split daughters, does the doubling persist? Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13156357#comment-13156357 ] stack commented on HBASE-4838: -- Yeah, look see if TRUNK has a fix in Reference or HalfStoreFileReader. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13156369#comment-13156369 ] Lars Hofhansl commented on HBASE-4838: -- Reference.java and HalfStoreFileReader.java are identical between 0.92 and trunk (and neither Reference nor HalfStoreFileReader appear in this patch), so that is likely not the cause. I also verified now that it picks up the correct store file (judged by the filename), which means the content of the store file is not correct. I thought maybe it had to do with ignoring the version counts in the ColumnTrackers, but that does not appear to be the problem. ... going to have to shelve this for a bit to work on some other stuff. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13155394#comment-13155394 ] Lars Hofhansl commented on HBASE-4838: -- Still working on this. I have narrowed down the scenario (also happens with just 2 cells that are split), but don't have the root cause, yet. I looked through the patch multiple times and I pretty sure the changes are correct (in the sense that they represent the patch), there must be an assumption about some behavior in trunk that is not present in 0.92. Port 2856 (TestAcidGuarantee is failing) to 0.92 Key: HBASE-4838 URL: https://issues.apache.org/jira/browse/HBASE-4838 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4838-v1.txt Moving back port into a separate issue (as suggested by JonH), because this not trivial. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4838) Port 2856 (TestAcidGuarantee is failing) to 0.92
[
https://issues.apache.org/jira/browse/HBASE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13155596#comment-13155596
]
Lars Hofhansl commented on HBASE-4838:
--
This still fails:
{noformat}
testFilterAcrossMultipleRegions(org.apache.hadoop.hbase.client.TestFromClientSid
e) Time elapsed: 12.233 sec FAILURE!
java.lang.AssertionError: expected:17576 but was:28064
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.hadoop.hbase.client.TestFromClientSide.assertRowCount(Test
FromClientSide.java:528)
at org.apache.hadoop.hbase.client.TestFromClientSide.testFilterAcrossMul
tipleRegions(TestFromClientSide.java:436)
{noformat}
I went through the entire diff between 0.92 with this patch and trunk. There is
nothing that sticks out that could be causing this.
This problem be easily reproduced by reducing the number of rows generated in
HBaseTestingUtility.loadTable(...) to (note: replace 'z' with 'a' and 'b'):
{noformat}
public int loadTable(final HTable t, final byte[] f) throws IOException {
t.setAutoFlush(false);
byte[] k = new byte[3];
int rowCount = 0;
for (byte b1 = 'a'; b1 = 'a'; b1++) {
for (byte b2 = 'a'; b2 = 'a'; b2++) {
for (byte b3 = 'a'; b3 = 'b'; b3++) {
k[0] = b1;
k[1] = b2;
k[2] = b3;
Put put = new Put(k);
put.add(f, null, k);
t.put(put);
rowCount++;
}
}
}
t.flushCommits();
return rowCount;
}
{noformat}
this will only generate two rows for this test, which makes it easier to debug.
The same test then fails with:
{noformat}
java.lang.AssertionError: expected:2 but was:4
...
{noformat}
Somehow it is scanning more rows after a split that it is supposed to repeating
some of the same rows. If somebody else wanted have a look at it, I could use
another pair of eyes.
Port 2856 (TestAcidGuarantee is failing) to 0.92
Key: HBASE-4838
URL: https://issues.apache.org/jira/browse/HBASE-4838
Project: HBase
Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.92.0
Attachments: 4838-v1.txt
Moving back port into a separate issue (as suggested by JonH), because this
not trivial.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
