[
https://issues.apache.org/jira/browse/HBASE-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736265#action_12736265
]
Andrew Purtell commented on HBASE-1715:
---------------------------------------
I'm running with this local patch:
{code}
Index: src/java/org/apache/hadoop/hbase/regionserver/HRegion.java
===================================================================
--- src/java/org/apache/hadoop/hbase/regionserver/HRegion.java (revision 798568
)
+++ src/java/org/apache/hadoop/hbase/regionserver/HRegion.java (working copy)
@@ -1250,6 +1250,7 @@
byte [] family = entry.getKey();
checkFamily(family);
List<KeyValue> puts = entry.getValue();
+ checkFamilyValues(family, puts);
if (updateKeys(puts, now)) {
put(family, puts, writeToWAL);
}
@@ -1262,6 +1263,22 @@
}
}
+ private void checkFamilyValues(byte[] family, List<KeyValue> puts)
+ throws IOException {
+ for (KeyValue kv: puts) {
+ if (Bytes.compareTo(
+ kv.getBuffer(),
+ kv.getFamilyOffset(),
+ kv.getFamilyLength(),
+ family,
+ 0,
+ family.length) != 0) {
+ throw new IOException("KV in edit set has wrong family: want=" +
+ Bytes.toStringBinary(family) + ", got=" +
+ Bytes.toStringBinary(kv.getFamily()));
+ }
+ }
+ }
//TODO, Think that gets/puts and deletes should be refactored a bit so that
//the getting of the lock happens before, so that you would just pass it into
{code}
and am watching the client logs. I may be wrong, but I don't think the family
of every KV is sanity checked elsewhere. Just about all of the ~250 regions of
my table have the error at hand, so I think the occurrence frequency here is
high enough that if there is some client problem it should show up.
> compaction failure in ScanWildcardColumnTracker.checkColumn
> -----------------------------------------------------------
>
> Key: HBASE-1715
> URL: https://issues.apache.org/jira/browse/HBASE-1715
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1715-v2.patch, HBASE-1715-v3.patch,
> HBASE-1715.patch
>
>
> I have at least one region that won't compact.
> {noformat}
> hbase> status 'detailed'
> [...]
> content,0b41cdd2ee9b36b0674ad423089800ba,1248257816633
> stores=3, storefiles=113, storefileSizeMB=1213, memstoreSizeMB=0,
> storefileIndexSizeMB=0
> [...]
> hbase> major_compact 'content'
> [...]
> hbase> status 'detailed'
> [...]
> content,0b41cdd2ee9b36b0674ad423089800ba,1248257816633
> stores=3, storefiles=113, storefileSizeMB=1213, memstoreSizeMB=0,
> storefileIndexSizeMB=0
> [...]
> hbase> major_compact 'content,0b41cdd2ee9b36b0674ad423089800ba,1248257816633'
> [...]
> hbase> status 'detailed'
> [...]
> content,0b41cdd2ee9b36b0674ad423089800ba,1248257816633
> stores=3, storefiles=113, storefileSizeMB=1213, memstoreSizeMB=0,
> storefileIndexSizeMB=0
> [...]
> {noformat}
> {noformat}
> $ hadoop fs -ls /hbase/content/1226419153/content
> Found 37 items
> -rw-r--r-- 2 hadoop supergroup 86422089 2009-07-24 16:11
> /hbase/content/1226419153/content/1045181559873943545
> -rw-r--r-- 2 hadoop supergroup 7167852 2009-07-26 05:13
> /hbase/content/1226419153/content/1225211885117827793
> -rw-r--r-- 2 hadoop supergroup 2678 2009-07-24 20:24
> /hbase/content/1226419153/content/1275251824404920815
> -rw-r--r-- 2 hadoop supergroup 10288489 2009-07-26 02:16
> /hbase/content/1226419153/content/1386224161790061095
> -rw-r--r-- 2 hadoop supergroup 16111798 2009-07-24 23:43
> /hbase/content/1226419153/content/1454452327579297030
> -rw-r--r-- 2 hadoop supergroup 12650208 2009-07-26 23:31
> /hbase/content/1226419153/content/1534514745508157864
> -rw-r--r-- 2 hadoop supergroup 6994590 2009-07-27 10:45
> /hbase/content/1226419153/content/1706519919143970421
> -rw-r--r-- 2 hadoop supergroup 13449537 2009-07-26 20:53
> /hbase/content/1226419153/content/2186668356269910871
> -rw-r--r-- 2 hadoop supergroup 10397457 2009-07-26 22:15
> /hbase/content/1226419153/content/2292706332643873473
> -rw-r--r-- 2 hadoop supergroup 6481576 2009-07-27 05:59
> /hbase/content/1226419153/content/2438489739664745000
> -rw-r--r-- 2 hadoop supergroup 10810191 2009-07-25 19:48
> /hbase/content/1226419153/content/2538008511360230014
> -rw-r--r-- 2 hadoop supergroup 13877559 2009-07-27 18:26
> /hbase/content/1226419153/content/2772019170563217117
> -rw-r--r-- 2 hadoop supergroup 12796040 2009-07-26 10:14
> /hbase/content/1226419153/content/2902101782772083009
> -rw-r--r-- 2 hadoop supergroup 6541657 2009-07-27 01:06
> /hbase/content/1226419153/content/3113808018684114931
> -rw-r--r-- 2 hadoop supergroup 8655428 2009-07-25 18:17
> /hbase/content/1226419153/content/3817932211925236778
> -rw-r--r-- 2 hadoop supergroup 1445535 2009-07-27 16:10
> /hbase/content/1226419153/content/4286233593585189878
> -rw-r--r-- 2 hadoop supergroup 9139508 2009-07-27 02:11
> /hbase/content/1226419153/content/495340788313226264
> -rw-r--r-- 2 hadoop supergroup 3313459 2009-07-26 05:57
> /hbase/content/1226419153/content/5167064731599803595
> -rw-r--r-- 2 hadoop supergroup 9473393 2009-07-26 14:44
> /hbase/content/1226419153/content/5490426319631514899
> -rw-r--r-- 2 hadoop supergroup 9321224 2009-07-26 19:40
> /hbase/content/1226419153/content/5790825797519034907
> -rw-r--r-- 2 hadoop supergroup 12171283 2009-07-27 12:20
> /hbase/content/1226419153/content/6036401533248383324
> -rw-r--r-- 2 hadoop supergroup 824790136 2009-07-23 22:28
> /hbase/content/1226419153/content/6211942192349190964
> -rw-r--r-- 2 hadoop supergroup 9905606 2009-07-27 07:45
> /hbase/content/1226419153/content/6295275445036553977
> -rw-r--r-- 2 hadoop supergroup 23857510 2009-07-27 17:02
> /hbase/content/1226419153/content/6535266251812885635
> -rw-r--r-- 2 hadoop supergroup 9177439 2009-07-26 11:21
> /hbase/content/1226419153/content/6625735185665629662
> -rw-r--r-- 2 hadoop supergroup 6916543 2009-07-27 20:15
> /hbase/content/1226419153/content/6934569497672884872
> -rw-r--r-- 2 hadoop supergroup 8818427 2009-07-27 13:48
> /hbase/content/1226419153/content/7162767181372457089
> -rw-r--r-- 2 hadoop supergroup 12028925 2009-07-25 08:48
> /hbase/content/1226419153/content/7254368961746328584
> -rw-r--r-- 2 hadoop supergroup 14695089 2009-07-27 09:16
> /hbase/content/1226419153/content/728058506789102941
> -rw-r--r-- 2 hadoop supergroup 9154325 2009-07-27 03:43
> /hbase/content/1226419153/content/7462549804082617977
> -rw-r--r-- 2 hadoop supergroup 22295283 2009-07-27 21:46
> /hbase/content/1226419153/content/7498645200757573769
> -rw-r--r-- 2 hadoop supergroup 5414671 2009-07-27 15:27
> /hbase/content/1226419153/content/758918234053985701
> -rw-r--r-- 2 hadoop supergroup 11260015 2009-07-26 08:16
> /hbase/content/1226419153/content/7626050291010778604
> -rw-r--r-- 2 hadoop supergroup 11309116 2009-07-26 12:57
> /hbase/content/1226419153/content/7704085331915902813
> -rw-r--r-- 2 hadoop supergroup 6326623 2009-07-26 17:58
> /hbase/content/1226419153/content/7735737241739599956
> -rw-r--r-- 2 hadoop supergroup 832 2009-07-24 20:15
> /hbase/content/1226419153/content/8170327402351444879
> -rw-r--r-- 2 hadoop supergroup 13480887 2009-07-26 16:32
> /hbase/content/1226419153/content/890729481029340216
> {noformat}
> Relevant nonstandard options:
> {noformat}
> <property>
> <name>hbase.hregion.max.filesize</name>
> <value>1073741824</value>
> </property>
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
