Phoenix uses the regular hbase.client.scanner.caching property, but we default it to 1000. The first next call does most of the work - pulling over the batch to the client in parallel. Given you query has no where clause, your more or less measuring your network speed.
Thanks, James On Wed, Mar 12, 2014 at 1:37 PM, Vladimir Rodionov <[email protected]>wrote: > Your first call to ResultSet.next does all the job (loading all 10000 rows > from HBase) and probably not in a optimal way? One row at a time? > I do not know if default scan caching setting affects Phoenix or Phoenix > uses its own scan caching - this is question to Phoenix developers. > > -Vladimir > > > On Wed, Mar 12, 2014 at 8:21 AM, alex kamil <[email protected]> wrote: > >> same result >> >> Execute Query : 401ms >> >> meta.getColumnCount : 1ms >> >> Time taken for first getNext() : 22593ms >> >> Time taken to loop through : 22871ms >> >> Execute Query : 14ms >> >> meta.getColumnCount : 0ms >> >> Time taken for first getNext() : 21892ms >> >> Time taken to loop through : 22041ms >> >> >> On Tue, Mar 11, 2014 at 7:15 PM, James Taylor <[email protected]>wrote: >> >>> What does the time look like if you run your query twice in the same >>> JVM, as I suspect you may be hitting a one-time cost: >>> >>> for (int i =0; i < 2; i++) { >>> >>> Statement stmt = con.createStatement(); >>> >>> ResultSet resultSet =stmt.executeQuery(query); >>> >>> Object o = null; int i=0;boolean isFirst = true; >>> >>> long start = System.currentTimeMillis(); >>> >>> * while(resultSet.next())*{ >>> >>> if (isFirst){ >>> >>> System.out.println("*Time taken for first getNext() *: " + >>> (System.currentTimeMillis() - start)); >>> >>> isFirst = false; >>> >>> } >>> >>> for (i = 0; i < colmax; ++i) >>> >>> { >>> >>> o = resultSet.getObject(i + 1); >>> >>> if (o != null){ >>> >>> //rowKeys.add(o.toString()); >>> >>> } >>> >>> } >>> >>> } >>> >>> System.out.println("Time taken to loop through : " + >>> (System.currentTimeMillis() - start)); >>> >>> } >>> >>> >>> >>> >>> >>> On Tue, Mar 11, 2014 at 12:57 PM, alex kamil <[email protected]>wrote: >>> >>>> running the query on a table with 3 salt buckets (previously 128) >>>> brought down the time to single digits sec, what could be the possible >>>> explanation? >>>> >>>> >>>> On Tue, Mar 11, 2014 at 3:30 PM, alex kamil <[email protected]>wrote: >>>> >>>>> running the below query in sqlline takes less than 4 sec but when >>>>> calling from Java client the first resultSet.next() is much slower (>20 >>>>> sec), any ideas? >>>>> >>>>> >>>>> client code >>>>> >>>>> String JDBC_DRIVER = "jdbc:phoenix:myip"; >>>>> >>>>> String query = "SELECT ROWKEY FROM mytable LIMIT 10000"; >>>>> >>>>> //ROWKEY size=32bytes, mytable size=4M rows (128 regions on 3 servers >>>>> with 40GB heap) >>>>> >>>>> Class.forName("com.salesforce.phoenix.jdbc.PhoenixDriver"); >>>>> >>>>> DriverManager.getConnection(JDBC_DRIVER); >>>>> >>>>> Connection con = DriverManager.getConnection(JDBC_DRIVER); >>>>> >>>>> Statement stmt = con.createStatement(); >>>>> >>>>> ResultSet resultSet =stmt.executeQuery(query); >>>>> >>>>> Object o = null; int i=0;boolean isFirst = true; >>>>> >>>>> long start = System.currentTimeMillis(); >>>>> >>>>> * while(resultSet.next())*{ >>>>> >>>>> if (isFirst){ >>>>> >>>>> System.out.println("*Time taken for first getNext() *: " + >>>>> (System.currentTimeMillis() - start)); >>>>> >>>>> isFirst = false; >>>>> >>>>> } >>>>> >>>>> for (i = 0; i < colmax; ++i) >>>>> >>>>> { >>>>> >>>>> o = resultSet.getObject(i + 1); >>>>> >>>>> if (o != null){ >>>>> >>>>> //rowKeys.add(o.toString()); >>>>> >>>>> } >>>>> >>>>> } >>>>> >>>>> } >>>>> >>>>> System.out.println("Time taken to loop through : " + >>>>> (System.currentTimeMillis() - start)); >>>>> >>>>> >>>>> >>>>> >>>> >>> >> >
