[ https://issues.apache.org/jira/browse/HBASE-21621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16727155#comment-16727155 ]
Guanghao Zhang commented on HBASE-21621: ---------------------------------------- Ping [~stack] [~Apache9]. I thought this should be merged to 2.1.2 and 2.0.4 release. > Reversed scan does not return expected number of rows > ------------------------------------------------------ > > Key: HBASE-21621 > URL: https://issues.apache.org/jira/browse/HBASE-21621 > Project: HBase > Issue Type: Bug > Components: scan > Affects Versions: 3.0.0, 2.1.1 > Reporter: Nihal Jain > Assignee: Guanghao Zhang > Priority: Critical > Attachments: HBASE-21621.master.001.patch, > HBASE-21621.master.002.patch, HBASE-21621.master.UT.patch > > > *Steps to reproduce* > # Create a table and put some data into it (data should be big enough, say N > rows) > # Flush the table > # Scan the table with reversed set to true > *Expected Result* > N rows should be retrieved in reversed order > *Actual Result* > Less than expected number of rows is retrieved with following error in logs > {noformat} > 2018-12-19 21:55:32,944 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=39007] > regionserver.StoreScanner(1000): Switch to stream read (scanned=262214 bytes) > of cf > 2018-12-19 21:55:32,955 ERROR > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=39007] > ipc.RpcServer(471): Unexpected throwable object > java.lang.AssertionError: Key > \x00\x00\x00\x00\x00\x00\x00\x09/cf:a/1545236714675/Put/vlen=131072/seqid=4 > followed by a error order key > \x00\x00\x00\x00\x00\x00\x00\x0F/cf:a/1545236715545/Put/vlen=131072/seqid=8 > in cf cf in reversed scan > at > org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.checkScanOrder(ReversedStoreScanner.java:105) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:568) > at > org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:153) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:6598) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6762) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:6535) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3252) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3501) > at > org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42002) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304) > 2018-12-19 21:55:32,955 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=39007] > ipc.CallRunner(142): callId: 508 service: ClientService methodName: Scan > size: 47 connection: 127.0.0.1:48328 deadline: 1545236792955, > exception=java.io.IOException: Key > \x00\x00\x00\x00\x00\x00\x00\x09/cf:a/1545236714675/Put/vlen=131072/seqid=4 > followed by a error order key > \x00\x00\x00\x00\x00\x00\x00\x0F/cf:a/1545236715545/Put/vlen=131072/seqid=8 > in cf cf in reversed scan > 2018-12-19 21:55:33,060 DEBUG > [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=39007] > ipc.CallRunner(142): callId: 511 service: ClientService methodName: Scan > size: 47 connection: 127.0.0.1:48328 deadline: 1545236792955, > exception=org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: > Expected nextCallSeq: 1 But the nextCallSeq got from client: 0; > request=scanner_id: 2421102592655360183 number_of_rows: 2147483647 > close_scanner: false next_call_seq: 0 client_handles_partials: true > client_handles_heartbeats: true track_scan_metrics: false renew: false > 2018-12-19 21:55:33,060 DEBUG [Time-limited test] > client.ScannerCallableWithReplicas(200): Scan with primary region returns > org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: > org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected > nextCallSeq: 1 But the nextCallSeq got from client: 0; request=scanner_id: > 2421102592655360183 number_of_rows: 2147483647 close_scanner: false > next_call_seq: 0 client_handles_partials: true client_handles_heartbeats: > true track_scan_metrics: false renew: false > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.checkScanNextCallSeq(RSRpcServices.java:3122) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3455) > at > org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42002) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304) > {noformat} > *Analysis/Issue* > From initial analysis it seems problem occurs when we switch read type -- This message was sent by Atlassian JIRA (v7.6.3#76005)