the first thread can be found at:
http://permalink.gmane.org/gmane.comp.java.hadoop.hbase.user/10074
After some dig, it seems that the problem is caused by long pause
between two "scanner.next()" call.
In my case the program has to spent a relatively long while to
process one row, when it calls "scanner.next()" again, seems that the
returned Result will be null even if there should be more rows in the
tables. The rowcaching is set to 1.
I have checked some of the source code, seems there is some
mechanism which will call the close() method of the ClientScanner, but I am
still checking.
I don't know if there is a certain timeout on
ClientScanner/ScannerCallable after a row has been successfully returned,
seems that timeout cause my problem here.
Any reply is appreciated.
On Fri, Apr 23, 2010 at 11:10 AM, steven zhuang <
steven.zhuang.1...@gmail.com> wrote:
> hi,
> sorry I start another thread here. This mail is actually answer
> to a previous thread "multiple scanners on same table will cause problem?
> Scan results change among different tries.".
> the mail system kept saying that I am spamming, now it seems that
> it's right! :)
>
> here is my reply to people in that thread:
>
> I don't know if there is a limit on reads to a single row/region in
> HBase, but if there is, I might have exceeded that limit. :(
> in my case, there are hundreds of rows, with dozens of kilos of cells
> in a row(a 256 MB region may contain 10- rows). for each row, I started a
> thread on each CF, there are 8 of them, so there might be dozens of scanners
> on the same region.
> and, to Tim, I could not see your attached mail, my test code is
> pasted below, it just iterate on the rows and column families, output all
> the cells.
>
> private void doScan() throws Exception {
> if (null == CopyOfTestTTT234.table) {
> return;
> }
> Scan s = new Scan();
> s.setStartRow("aaa".getBytes());
> s.setStopRow("ccc".getBytes());
> s.setCaching(CopyOfTestTTT234.ROWCACHING); //it's 1 here.
> ResultScanner scanner = CopyOfTestTTT234.table.getScanner(s);
> while (true) {
> Result row = scanner.next();
> if(null==row) break;
> String rowKey = new String(row.getRow());
> NavigableMap<byte[], NavigableMap<byte[], byte[]>> fm = row
> .getNoVersionMap();
> while (fm.size() > 0) {
> Entry<byte[], NavigableMap<byte[], byte[]>> ee = fm
> .pollFirstEntry();
> String fName = new String(ee.getKey());
> NavigableMap<byte[], byte[]> ff = ee.getValue();
> while (ff.size() > 0) {
> Entry<byte[], byte[]> cell = ff.pollFirstEntry();
> String key = new String(cell.getKey());
> String val = new String(cell.getValue());
> System.out.println(Thread.currentThread().hashCode() + "\t"
> + rowKey + "\t" + fName + "\t" + key + "\t" + val);
> }
> }
> }
> }
>
