Hi Ted,
Thanks for your information.
My application queries the HBase, and for some of the queries it just hang
there and throw exception after several minutes (5-8minutes). As a workaround,
I try to set the timeout to a shorter time, so my app won’t hang for minutes
but for several seconds. I tried to set both the time out to 1000 (1s). but it
still hang for several minutes.Is this expected?
Appreciate it if you know how I could fix the exception.
Caused by:
org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(java.io.IOException):
java.io.IOException: Could not seek StoreFileScanner[HFileScanner for reader
reader=hdfs://xxx/hbase/data/data/default/xxx/af7898973c510425fabb7c814ac8ba04/EOUT_T_SRD/125acceb75d84724a089701c590a4d3d,
compression=snappy, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
[cacheEvictOnClose=false] [cacheCompressed=false],
firstKey=addrv#34005240#US,_28409,_822|addre/F|rval#null|cust#1158923121468951849|addre#1095283883|1/EOUT_T_SRD:~/1430982000000/Put,
lastKey=addrv#38035AC7#US,_60449,_4684|addre/F|rval#null|cust#1335211720509289817|addre#697997140|1/EOUT_T_SRD:~/1430982000000/Put,
avgKeyLen=122, avgValueLen=187, entries=105492830, length=6880313695,
cur=null] to key addrv#34B97AEC#FR,_06110,_41 route des
breguieres|addre/F|rval#/EOUT_T_SRD:/LATEST_TIMESTAMP/DeleteFamily/vlen=0/mvcc=0
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:165)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:317)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:176)
at
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1847)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:3716)
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1890)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1876)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1853)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3090)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28861)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.io.IOException: Failed to read compressed block at 1253175503,
onDiskSizeWithoutHeader=66428, preReadHeaderSize=33, header.length=33, header
bytes:
DATABLKE\x00\x00&3\x00\x00\xC3\xC9\x00\x00\x00\x01r\xC4-\xDF\x01\x00\x00@\x00\x00\x00&P
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1451)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1314)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:355)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:253)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:494)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:515)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:238)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:153)
... 15 more
Caused by: java.io.IOException: Invalid HFile block magic:
\x00\x00\x00\x00\x00\x00\x00\x00
at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:154)
at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:165)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock.<init>(HFileBlock.java:239)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1448
Thanks,
Mike
From: Ted Yu [mailto:yuzhih...@gmail.com]
Sent: Monday, May 18, 2015 11:55 PM
To: user@hbase.apache.org
Cc: Fang, Mike; Dai, Kevin
Subject: Re: How to set Timeout for get/scan operations without impacting others
hbase.client.operation.timeout is used by HBaseAdmin operations, by
RegionReplicaFlushHandler and by various HTable operations (including Get).
hbase.rpc.timeout is for the RPC layer to define how long HBase client
applications take for a remote call to time out. It uses pings to check
connections but will eventually throw a TimeoutException.
FYI
On Sun, May 17, 2015 at 11:11 PM, Jianshi Huang
<jianshi.hu...@gmail.com<mailto:jianshi.hu...@gmail.com>> wrote:
Hi,
I need to set tight timeout for get/scan operations and I think HBase
Client already support it.
I found three related keys:
- hbase.client.operation.timeout
- hbase.rpc.timeout
- hbase.client.retries.number
What's the difference between hbase.client.operation.timeout and
hbase.rpc.timeout?
My understanding is that hbase.rpc.timeout has larger scope than hbase.
client.operation.timeout, so setting hbase.client.operation.timeout is
safer. Am I correct?
And any other property keys I can uses?
--
Jianshi Huang
LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/
