Hi Anoop, I receive this error in client side, and pretty sure the scan failed. I'm using default caching, so it should be 100, right? About scan time out period, I will try to set it higher, probably 1 hour.
BTW, I'm using hbase 0.96.0. Best regards, Henry -----Original Message----- From: Anoop John [mailto:anoop.hb...@gmail.com] Sent: Monday, September 29, 2014 3:13 PM To: user@hbase.apache.org Subject: Re: scan + filter failed with OutOfOrderScannerNextException Hi Even when the RS throws this Exception, the client side will start a new Scanner and retry. U just see this in log or the scan is failing altogether? What is the caching you use on Scan? When most of the rows are filtered out at server side, it takes more time to fetch and return the 'caching' number of rows back to client. By this time the client would have already timed out and sent a retry. We allow such a retry will cause not getting back all the results. This is a serious problem and this Exception way helps there. You can try increasing the scan time out period at client side. This should help. -Anoop- On Mon, Sep 29, 2014 at 12:35 PM, Henry Hung <ythu...@winbond.com> wrote: > Hi All, > > Is there a way to let scanner finish scanning all regions without > throwing this kind of error? I'm using scan with filter MUST_PASS_ALL, > and I observe that whenever the result data is smaller (let's say 10%) > compare to another filter with larger result (let's say 80%), it always > failed. > > org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of > OutOfOrderScannerNextException: was there a rpc timeout? > at > org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:3 > 91) > at > com.winbond.hbase.schema.mesperflogtime.MesPerfLogTimeResultSet.next(M > esPerfLogTimeResultSet.java:58) > at > com.winbond.hbase.schema.mesperflogtime.MesPerfLogTimeResultSet.next(M > esPerfLogTimeResultSet.java:1) at > com.winbond.hbase.executor.Program.main(Program.java:106) > Caused by: > org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: > org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: > Expected > nextCallSeq: 1 But the nextCallSeq got from client: 0; request=scanner_id: > 1594996183743944787 number_of_rows: 100 close_scanner: false next_call_seq: > 0 > at > org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer. > java:3007) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$ > 2.callBlockingMethod(ClientProtos.java:26929) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146) > at > org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructo > rAccessorImpl.java:39) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCo > nstructorAccessorImpl.java:27) at > java.lang.reflect.Constructor.newInstance(Constructor.java:513) > at > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteExcep > tion.java:106) > at > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteExce > ption.java:95) > at > org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(Proto > bufUtil.java:235) > at > org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.ja > va:197) > at > org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.ja > va:57) > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRe > tryingCaller.java:116) > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRe > tryingCaller.java:94) > at > org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:3 > 43) > ... 3 more > Caused by: > org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException): > org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: > Expected > nextCallSeq: 1 But the nextCallSeq got from client: 0; request=scanner_id: > 1594996183743944787 number_of_rows: 100 close_scanner: false next_call_seq: > 0 > at > org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer. > java:3007) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$ > 2.callBlockingMethod(ClientProtos.java:26929) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146) > at > org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851) > > at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1446) > at > org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.jav > a:1650) > at > org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation > .callBlockingMethod(RpcClient.java:1708) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$ > BlockingStub.scan(ClientProtos.java:27332) > at > org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.ja > va:167) > ... 7 more > > Best regards, > Henry > > ________________________________ > The privileged confidential information contained in this email is > intended for use only by the addressees as indicated by the original > sender of this email. If you are not the addressee indicated in this > email or are not responsible for delivery of the email to such a > person, please kindly reply to the sender indicating this fact and > delete all copies of it from your computer and network server > immediately. Your cooperation is highly appreciated. It is advised > that any unauthorized use of confidential information of Winbond is > strictly prohibited; and any information in this email irrelevant to > the official business of Winbond shall be deemed as neither given nor > endorsed by Winbond. > The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Winbond is strictly prohibited; and any information in this email irrelevant to the official business of Winbond shall be deemed as neither given nor endorsed by Winbond.
