[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228206#comment-13228206
]
Lars Hofhansl commented on HBASE-5569:
--------------------------------------
A bit more context:
One case:
{quote}
2012-03-12 21:48:49,497 INFO [Thread-260] regionserver.Store(796): Added
/home/jenkins/jenkins-slave/workspace/HBase-TRUNK/trunk/target/test-data/e923fe0e-3b3e-4c67-89ec-4cac8c991955/TestIncrementtestMultiRowMutationMultiThreads/testtable/446f80b650aa093734c2dff4b9581ff8/colfamily11/e0930b6c478c4a5db9eceaead90bc80e,
entries=7, sequenceid=75545, filesize=1.0k
2012-03-12 21:48:49,522 INFO [Thread-260] regionserver.HRegion(1552): Finished
memstore flush of ~87.4k/89544, currentsize=20.8k/21320 for region
testtable,,1331588915162.446f80b650aa093734c2dff4b9581ff8. in 63ms,
sequenceid=75545, compaction requested=true
2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
Storescanner.peek() is changed where before =
rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
2012-03-12 21:48:49,523 DEBUG [Thread-211]
regionserver.TestAtomicOperation$2(390): []
Exception in thread "Thread-211" junit.framework.AssertionFailedError at
junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.fail(Assert.java:56)
at
org.apache.hadoop.hbase.regionserver.TestAtomicOperation$2.run(TestAtomicOperation.java:392)
{quote}
Another case:
{quote}
2012-03-13 01:34:06,655 INFO [Thread-212] regionserver.Store(748): Flushed ,
sequenceid=56173, memsize=1.8k, into tmp file
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/48f0d653-d644-41be-80ff-90e726af10d4/TestIncrementtestMultiRowMutationMultiThreads/testtable/00fad569500db871769b9d5951b3ed16/.tmp/a0e1d5df9b5344c19ddbc7b11e0cd9d2
2012-03-13 01:34:06,656 DEBUG [Thread-212] regionserver.Store(773): Renaming
flushed file at
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/48f0d653-d644-41be-80ff-90e726af10d4/TestIncrementtestMultiRowMutationMultiThreads/testtable/00fad569500db871769b9d5951b3ed16/.tmp/a0e1d5df9b5344c19ddbc7b11e0cd9d2
to
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/48f0d653-d644-41be-80ff-90e726af10d4/TestIncrementtestMultiRowMutationMultiThreads/testtable/00fad569500db871769b9d5951b3ed16/colfamily11/a0e1d5df9b5344c19ddbc7b11e0cd9d2
2012-03-13 01:34:06,661 INFO [Thread-212] regionserver.Store(796): Added
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-data/48f0d653-d644-41be-80ff-90e726af10d4/TestIncrementtestMultiRowMutationMultiThreads/testtable/00fad569500db871769b9d5951b3ed16/colfamily11/a0e1d5df9b5344c19ddbc7b11e0cd9d2,
entries=11, sequenceid=56173, filesize=1.2k
2012-03-13 01:34:06,674 DEBUG [Thread-287] regionserver.StoreScanner(499):
Storescanner.peek() is changed where before =
rowB/colfamily11:qual1/56043/DeleteColumn/vlen=0,and after =
rowB/colfamily11:qual1/54931/Put/vlen=6
2012-03-13 01:34:06,674 DEBUG [Thread-287]
regionserver.TestAtomicOperation$2(390):
[rowA/colfamily11:qual1/56043/Put/vlen=6,
rowB/colfamily11:qual1/54931/Put/vlen=6]
Exception in thread "Thread-287" junit.framework.AssertionFailedError at
junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.fail(Assert.java:56)
at
org.apache.hadoop.hbase.regionserver.TestAtomicOperation$2.run(TestAtomicOperation.java:392)
2012-03-13 01:34:06,675 INFO [Thread-212] regionserver.HRegion(1552): Finished
memstore flush of ~380.5k/389664, currentsize=28.5k/29192 for region
testtable,,1331602436835.00fad569500db871769b9d5951b3ed16. in 44ms,
sequenceid=56173, compaction requested=true
{quote}
So it seems this is related to flushing (test test flushes frequently - 1/s -
precisely to exercise this scenario)
> TestAtomicOperation.testMultiRowMutationMultiThreads fails occasionally
> -----------------------------------------------------------------------
>
> Key: HBASE-5569
> URL: https://issues.apache.org/jira/browse/HBASE-5569
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Priority: Minor
>
> What I pieced together so far is that it is the *scanning* side that has
> problems sometimes.
> Every time I see a assertion failure in the log I see this before:
> {quote}
> 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
> Storescanner.peek() is changed where before =
> rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
> rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
> {quote}
> The order of if the Put and Delete is sometimes reversed.
> The test threads should always see exactly one KV, if the "before" was the
> Put the thread see 0 KVs, if the "before" was the Delete the threads see 2
> KVs.
> This debug message comes from StoreScanner to checkReseek. It seems we still
> some consistency issue with scanning sometimes :(
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
