[ https://issues.apache.org/jira/browse/HBASE-21621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16728026#comment-16728026 ]
Hudson commented on HBASE-21621: -------------------------------- Results for branch branch-2.0 [build #1191 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1191/]: (x) *{color:red}-1 overall{color}* ---- details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1191//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1191//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1191//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > Reversed scan does not return expected number of rows > ------------------------------------------------------ > > Key: HBASE-21621 > URL: https://issues.apache.org/jira/browse/HBASE-21621 > Project: HBase > Issue Type: Bug > Components: scan > Affects Versions: 3.0.0, 2.1.1 > Reporter: Nihal Jain > Assignee: Guanghao Zhang > Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.1.2, 2.0.4 > > Attachments: HBASE-21621.master.001.patch, > HBASE-21621.master.002.patch, HBASE-21621.master.UT.patch > > > *Steps to reproduce* > # Create a table and put some data into it (data should be big enough, say N > rows) > # Flush the table > # Scan the table with reversed set to true > *Expected Result* > N rows should be retrieved in reversed order > *Actual Result* > Less than expected number of rows is retrieved with following error in logs > {noformat} > 2018-12-19 21:55:32,944 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=39007] > regionserver.StoreScanner(1000): Switch to stream read (scanned=262214 bytes) > of cf > 2018-12-19 21:55:32,955 ERROR > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=39007] > ipc.RpcServer(471): Unexpected throwable object > java.lang.AssertionError: Key > \x00\x00\x00\x00\x00\x00\x00\x09/cf:a/1545236714675/Put/vlen=131072/seqid=4 > followed by a error order key > \x00\x00\x00\x00\x00\x00\x00\x0F/cf:a/1545236715545/Put/vlen=131072/seqid=8 > in cf cf in reversed scan > at > org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.checkScanOrder(ReversedStoreScanner.java:105) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:568) > at > org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:153) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:6598) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6762) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:6535) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3252) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3501) > at > org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42002) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304) > 2018-12-19 21:55:32,955 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=39007] > ipc.CallRunner(142): callId: 508 service: ClientService methodName: Scan > size: 47 connection: 127.0.0.1:48328 deadline: 1545236792955, > exception=java.io.IOException: Key > \x00\x00\x00\x00\x00\x00\x00\x09/cf:a/1545236714675/Put/vlen=131072/seqid=4 > followed by a error order key > \x00\x00\x00\x00\x00\x00\x00\x0F/cf:a/1545236715545/Put/vlen=131072/seqid=8 > in cf cf in reversed scan > 2018-12-19 21:55:33,060 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=39007] > ipc.CallRunner(142): callId: 511 service: ClientService methodName: Scan > size: 47 connection: 127.0.0.1:48328 deadline: 1545236792955, > exception=org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: > Expected nextCallSeq: 1 But the nextCallSeq got from client: 0; > request=scanner_id: 2421102592655360183 number_of_rows: 2147483647 > close_scanner: false next_call_seq: 0 client_handles_partials: true > client_handles_heartbeats: true track_scan_metrics: false renew: false > 2018-12-19 21:55:33,060 DEBUG [Time-limited test] > client.ScannerCallableWithReplicas(200): Scan with primary region returns > org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: > org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected > nextCallSeq: 1 But the nextCallSeq got from client: 0; request=scanner_id: > 2421102592655360183 number_of_rows: 2147483647 close_scanner: false > next_call_seq: 0 client_handles_partials: true client_handles_heartbeats: > true track_scan_metrics: false renew: false > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.checkScanNextCallSeq(RSRpcServices.java:3122) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3455) > at > org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42002) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304) > {noformat} > *Analysis/Issue* > From initial analysis it seems problem occurs when we switch read type -- This message was sent by Atlassian JIRA (v7.6.3#76005)