I am doing a mult-thread(100) scan test over hbase. If one request with given key-range matches a large number of correspoding rows in hbase, my request is waiting for this scan to complete. The throughput is really slow. For test purpose, I'd like to use LIMIT to reduce the time on scanning and transferring results back from hbase to increase the throughput. Do you think the "hbase.client.scan.max.result.size" or setMaxResultSize(in bytes) could help HBase to stop scan at the LIMIT before scanning complete corresponding rows?
As you mentioned that there is no query optimizer in HBase, I assume that region servers will not stop scanning the rows in this key-range in this case until it gets all the results and limit the results to max size which is sent to the client. If so, there is not much I can do to compare the throughput with that in relational databases like MySQL. Thanks, Cheers. > Limit clause in SQL Select statement makes sense because it allows query > optimizer to plan accordingly. > It does not make sense in HBase as since there is no query planner and or > optimization involved during > scanning HBase table. You can easily mimic this functionality on a client > side (I mean - limit). > > Best regards, > Vladimir Rodionov > Principal Platform Engineer > Carrier IQ, www.carrieriq.com > e-mail: vrodio...@carrieriq.com > > ________________________________________ > From: Weiping Qu [q...@informatik.uni-kl.de] > Sent: Monday, March 17, 2014 4:48 AM > To: dev@hbase.apache.org > Subject: Re: 答复: setMaxResultSize method in Scan > > Thanks. > > I'd like to assume that setMaxResultSize is equivalent to the SQL Limit > clause as expected which is specified each time a SQL statement is > executed . > Now through "hbase.client.scanner.max.result.size" can the limitation of > number of row returned only apply to all the scanner instances. > I am wondering why the setMaxResultSize is removed now. > >> No such method for Scan in 0.94.x. >> >> If you want to set the max result size for a scan, you can achieve this by >> setting the "hbase.client.scanner.max.result.size" configuration, the >> default for which is Long.MAX_VALUE (no limited) >> ________________________________________ >> 发件人: Weiping Qu [q...@informatik.uni-kl.de] >> 发送时间: 2014年3月17日 18:50 >> 收件人: dev@hbase.apache.org >> 主题: setMaxResultSize method in Scan >> >> Hello, >> >> I could not find the method setMaxResultSize(long m) >> (http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html) >> in my Scanclass (0.94.13 version). >> Can anyone help me? Thanks >> >> Weiping > > -- > Mit freundlichen Grü?en / Kind Regards > > *Weiping Qu* > > University of Kaiserslautern > Department of Computer Science > Heterogeneous Information Systems Group > P.O. Box 3049 > 67653 Kaiserslautern, Germany > > Email: qu (at) informatik.uni-kl.de > Phone: +49 631 205 3264 > Fax: +49 631 205 3299 > Room: 36/331 > > Confidentiality Notice: The information contained in this message, including > any attachments hereto, may be confidential and is intended to be read only > by the individual or entity to whom this message is addressed. If the reader > of this message is not the intended recipient or an agent or designee of the > intended recipient, please note that any review, use, disclosure or > distribution of this message or its attachments, in any form, is strictly > prohibited. If you have received this message in error, please immediately > notify the sender and/or notificati...@carrieriq.com and delete or destroy > any copy of this message and its attachments. -- Mit freundlichen Grü?en / Kind Regards *Weiping Qu* University of Kaiserslautern Department of Computer Science Heterogeneous Information Systems Group P.O. Box 3049 67653 Kaiserslautern, Germany Email: qu (at) informatik.uni-kl.de Phone: +49 631 205 3264 Fax: +49 631 205 3299 Room: 36/331
