Hello J.G,
Thank you for your reply.
My hbase version is the newest : 0.20.0.
I have two tables, both having 10000 cells, one having 100 rows and 100
columns per row, another having 10000 rows and 1 column per row. I wanted
to test the speed scanning all the cells in 2 tables. When I scanned this 2
tables, at "scanner = table.getScanner(s);", the same exceptions in last
mail were both thrown . And following is the complete codes:
public class HbaseScannerTest {
// private final String TableNameA = "ttest1";
// private final String TableNameB = "ttest2";
private final String ColumnFamily = "cf_test";
private final String Column = "c_test";
private void scan(String tableName) {
HBaseConfiguration config = new HBaseConfiguration();
HTable table = null;
ResultScanner scanner = null;
try {
table = new HTable(config, tableName);
Scan s = new Scan();
s = s.addFamily(Bytes.toBytes(ColumnFamily));
scanner = table.getScanner(s);
for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
NavigableMap<byte[], byte[]> map =
rr.getFamilyMap(Bytes.toBytes(ColumnFamily));
Set<byte[]> set = map.keySet();
Iterator<byte[]> iter = set.iterator();
while(iter.hasNext()){
byte[] key = iter.next();
String strKey = Bytes.toString(key);
String strValue = Bytes.toString(map.get(key));
System.out.println("key:" + strKey + ", value:" + strValue);
}
}
} catch(Exception e){
e.printStackTrace();
} finally {
try{
if(scanner != null){
scanner.close();
}
if(table != null){
table.close();
}
}catch(Exception e){
e.printStackTrace();
}
}
}
public static void main(String[] args) {
if(args == null || args.length == 0){
System.out.println("param error.");
return;
}else{
HbaseScannerTest scannerTest = new HbaseScannerTest();
long t1 = System.currentTimeMillis();
scannerTest.scan(args[0]);
long t2 = System.currentTimeMillis();
System.out.println("*********scan " + args[0] + " need : " + (t2 - t1));
}
}
}
When the exceptions were thrown, the log in master was normal, and the logs
in other region servers were normal too, but in
/192.168.33.6:60020(sometimes another server) which pointed in the
exceptions, the log is
abnormal. I think I have put all the relational logs in last mail, because
the remained logs looks like unrelated at all. The following is the more
complete logs:
2009-09-18 11:08:37,050 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_CLOSE:
TestTable,0002215349,1251708925379: Overloaded
2009-09-18 11:08:37,051 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_CLOSE:
TestTable,0001302275,1251708861978: Overloaded
2009-09-18 11:08:37,051 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_CLOSE:
TestTable,0000884288,1251705503037: Overloaded
2009-09-18 11:08:37,051 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_CLOSE:
TestTable,0000065336,1251705454231: Overloaded
2009-09-18 11:08:37,051 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_CLOSE:
TestTable,0001062941,1251708879165: Overloaded
2009-09-18 11:08:37,051 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
MSG_REGION_CLOSE: TestTable,0002215349,1251708925379: Overloaded
2009-09-18 11:08:37,052 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed TestTable,0002215349,1251708925379
2009-09-18 11:08:37,052 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
MSG_REGION_CLOSE: TestTable,0001302275,1251708861978: Overloaded
2009-09-18 11:08:37,052 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed TestTable,0001302275,1251708861978
2009-09-18 11:08:37,053 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
MSG_REGION_CLOSE: TestTable,0000884288,1251705503037: Overloaded
2009-09-18 11:08:37,053 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed TestTable,0000884288,1251705503037
2009-09-18 11:08:37,053 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
MSG_REGION_CLOSE: TestTable,0000065336,1251705454231: Overloaded
2009-09-18 11:08:37,053 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed TestTable,0000065336,1251705454231
2009-09-18 11:08:37,053 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
MSG_REGION_CLOSE: TestTable,0001062941,1251708879165: Overloaded
2009-09-18 11:08:37,054 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed TestTable,0001062941,1251708879165
2009-09-18 11:08:49,109 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
TestTable,0000952473,1251709074516
2009-09-18 11:08:49,109 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
TestTable,0000595460,1251708629215
2009-09-18 11:08:49,109 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
webpage,http:\x2F\x2Fnews.163.com
\x2F09\x2F0803\x2F01\x2F5FOO155J0001124J.html1251254047232_16208,1251254433708
2009-09-18 11:08:49,109 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
TestTable,0002805208,1251709024537
2009-09-18 11:08:49,109 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN:
TestTable,0000952473,1251709074516
2009-09-18 11:08:49,148 INFO org.apache.hadoop.hbase.regionserver.HRegion:
region TestTable,0000952473,1251709074516/1184073320 available; sequence id
is 5922826
2009-09-18 11:08:49,148 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN:
TestTable,0000595460,1251708629215
2009-09-18 11:08:49,191 INFO org.apache.hadoop.hbase.regionserver.HRegion:
region TestTable,0000595460,1251708629215/1982730697 available; sequence id
is 5937085
2009-09-18 11:08:49,191 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN:
webpage,http:\x2F\x2Fnews.163.com
\x2F09\x2F0803\x2F01\x2F5FOO155J0001124J.html1251254047232_16208,1251254433708
2009-09-18 11:08:49,285 INFO org.apache.hadoop.hbase.regionserver.HRegion:
region
webpage,http:\x2F\x2Fnews.163.com\x2F09\x2F0803\x2F01\x2F5FOO155J0001124J.html1251254047232_16208,1251254433708/174964099
available; sequence id is 173787
2009-09-18 11:08:49,285 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN:
TestTable,0002805208,1251709024537
2009-09-18 11:08:49,355 INFO org.apache.hadoop.hbase.regionserver.HRegion:
region TestTable,0002805208,1251709024537/1263036978 available; sequence id
is 5922825
2009-09-18 11:18:04,909 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: compactions no longer
limited
2009-09-18 11:32:03,523 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer:
org.apache.hadoop.hbase.UnknownScannerException: Name: -1
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1905)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:913)
2009-09-18 11:32:03,529 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
handler 1 on 60020, call next(-1, 1) from 192.168.33.7:43810: error:
org.apache.hadoop.hbase.UnknownScannerException: Name: -1
org.apache.hadoop.hbase.UnknownScannerException: Name: -1
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1905)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:913)
Any suggestion?
LvZheng
2009/9/18 Jonathan Gray <[email protected]>
> How many rows are you scanning? The code where you are iterating through
> "..." is also relevant, can you post it? And it would be helpful if you
> could post more of the regionserver log file.
>
> Also, which version are you running?
>
>
> Zheng Lv wrote:
>
>> Hello Everyone,
>> I got some exceptions when I run the following code:
>>
>> HBaseConfiguration config = new HBaseConfiguration();
>> HTable table = null;
>> ResultScanner scanner = null;
>> try {
>> table = new HTable(config, tableName);
>> Scan s = new Scan();
>> s.addFamily(Bytes.toBytes(ColumnFamily));
>> scanner = table.getScanner(s);
>> ... ...
>>
>>
>>
>> And the exceptions are:
>>
>> java.io.IOException: Call to /192.168.33.6:60020 failed on local
>> exception:
>> java.io.EOFException
>> java.io.IOException: Call to /192.168.33.6:60020 failed on local
>> exception:
>> java.io.EOFException
>> java.io.IOException: Call to /192.168.33.6:60020 failed on local
>> exception:
>> java.io.EOFException
>> java.io.IOException: Call to /192.168.33.6:60020 failed on local
>> exception:
>> java.io.EOFException
>> java.io.IOException: Call to /192.168.33.6:60020 failed on local
>> exception:
>> java.io.EOFException
>> java.io.IOException: Call to /192.168.33.6:60020 failed on local
>> exception:
>> java.io.EOFException
>> java.io.IOException: Call to /192.168.33.6:60020 failed on local
>> exception:
>> java.io.EOFException
>> java.io.IOException: Call to /192.168.33.6:60020 failed on local
>> exception:
>> java.io.EOFException
>> java.io.IOException: Call to /192.168.33.6:60020 failed on local
>> exception:
>> java.io.EOFException
>> java.io.IOException: Call to /192.168.33.6:60020 failed on local
>> exception:
>> java.io.EOFException
>> at
>>
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:994)
>> at
>>
>> org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1878)
>> at
>>
>> org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1824)
>> at
>> org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:370)
>> at hbasetest.HbaseScannerTest.scan(HbaseScannerTest.java:33)
>> at hbasetest.HbaseScannerTest.main(HbaseScannerTest.java:70)
>>
>>
>>
>>
>> And the exceptions on 192.168.33.6 are:
>> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> at
>>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1905)
>> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650)
>> at
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:913)
>> 2009-09-17 16:51:20,958 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
>> handler 1 on 60020, call next(-1, 1) from 192.168.33.7:54782: error:
>> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> at
>>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1905)
>> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650)
>> at
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:913)
>> 2009-09-17 16:58:53,128 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: compactions no longer
>> limited
>> 2009-09-17 17:29:51,130 ERROR
>> org.apache.hadoop.hbase.regionserver.HRegionServer:
>> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> at
>>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1905)
>> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650)
>> at
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:913)
>> 2009-09-17 17:29:51,134 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
>> handler 5 on 60020, call next(-1, 1) from 192.168.33.7:57936: error:
>> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> org.apache.hadoop.hbase.UnknownScannerException: Name: -1
>> at
>>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1905)
>> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650)
>> at
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:913)
>>
>>
>> I think my codes are really simple, can you tell why these exception
>> were thrown?
>> Thanks a lot,
>> LvZheng
>>
>>
