[ https://issues.apache.org/jira/browse/HBASE-9338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13762125#comment-13762125 ]
Elliott Clark edited comment on HBASE-9338 at 9/9/13 6:25 PM: -------------------------------------------------------------- Here's the patch that I used to search for missed row keys. The row keys in the RowLogSearchJob are the ones that were referenced but not found. I found the keys in 5 different wal logs. (so it doesn't seem like it's an issue with reading the wal log) This was then checked like this: {code} hbase@a1805:/home/eclark$ hbase org.apache.hadoop.hbase.regionserver.wal.HLogPrettyPrinter -w "\\x8E<\\xF3\\xE1f\\x1Al\\x89d\\xD3\\xC1w5\\x9B\\x8FN" hdfs://a1805.halxg.cloudera.com:8020/hbase/oldWALs/a1809.halxg.cloudera.com%2C60020%2C1378540278223.1378540795093 Row: \x8E<\xF3\xE1f\x1Al\x89d\xD3\xC1w5\x9B\x8FN SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/hbase/jenkins-hbase-custom-branch-34/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.1.0-beta/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2013-09-09 11:19:43,435 WARN [main] conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS Sequence 2846225 from region 8b3a9c2a567f1eca40d25859e9e56fa2 in table IntegrationTestBigLinkedList at write timestamp: Sat Sep 07 00:59:58 PDT 2013 Action: row: \x8E<\xF3\xE1f\x1Al\x89d\xD3\xC1w5\x9B\x8FN column: meta:prev timestamp: Sat Sep 07 00:59:58 PDT 2013 Action: row: \x8E<\xF3\xE1f\x1Al\x89d\xD3\xC1w5\x9B\x8FN column: meta:count timestamp: Sat Sep 07 00:59:58 PDT 2013 Action: row: \x8E<\xF3\xE1f\x1Al\x89d\xD3\xC1w5\x9B\x8FN column: meta:client timestamp: Sat Sep 07 00:59:58 PDT 2013 {code} was (Author: eclark): Here's the patch that I used to search for missed row keys. The row keys in the RowLogSearchJob are the ones that were referenced but not found. I found the keys in 5 different wal logs. (os it doesn't seem like it's an issue with reading the wal log) This was then checked like this: {code} hbase@a1805:/home/eclark$ hbase org.apache.hadoop.hbase.regionserver.wal.HLogPrettyPrinter -w "\\x8E<\\xF3\\xE1f\\x1Al\\x89d\\xD3\\xC1w5\\x9B\\x8FN" hdfs://a1805.halxg.cloudera.com:8020/hbase/oldWALs/a1809.halxg.cloudera.com%2C60020%2C1378540278223.1378540795093 Row: \x8E<\xF3\xE1f\x1Al\x89d\xD3\xC1w5\x9B\x8FN SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/hbase/jenkins-hbase-custom-branch-34/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.1.0-beta/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2013-09-09 11:19:43,435 WARN [main] conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS Sequence 2846225 from region 8b3a9c2a567f1eca40d25859e9e56fa2 in table IntegrationTestBigLinkedList at write timestamp: Sat Sep 07 00:59:58 PDT 2013 Action: row: \x8E<\xF3\xE1f\x1Al\x89d\xD3\xC1w5\x9B\x8FN column: meta:prev timestamp: Sat Sep 07 00:59:58 PDT 2013 Action: row: \x8E<\xF3\xE1f\x1Al\x89d\xD3\xC1w5\x9B\x8FN column: meta:count timestamp: Sat Sep 07 00:59:58 PDT 2013 Action: row: \x8E<\xF3\xE1f\x1Al\x89d\xD3\xC1w5\x9B\x8FN column: meta:client timestamp: Sat Sep 07 00:59:58 PDT 2013 {code} > Test Big Linked List fails on Hadoop 2.1.0 > ------------------------------------------ > > Key: HBASE-9338 > URL: https://issues.apache.org/jira/browse/HBASE-9338 > Project: HBase > Issue Type: Bug > Components: test > Affects Versions: 0.96.0 > Reporter: Elliott Clark > Assignee: Elliott Clark > Priority: Blocker > Fix For: 0.96.0 > > Attachments: HBASE-9338-TESTING-2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira