1) What is the heap size and total memory on each node? 8GB, 8GB 2) How big is the cluster? 4 3) What are the read and range timeouts (in cassandra.yaml) on the C* nodes? 10 secs, 10 secs 4) What are the timeouts for the Astyanax client? 2 secs 5) Do you see GC pressure on the C* nodes? How long does GC for new gen and old gen take? occurs every 5 secs dont see huge gc pressure, <50ms 6) Does any node crash with OOM error when you try AllRowsReader? No
From: Mohammed Guller [mailto:moham...@glassbeam.com] Sent: Friday, January 16, 2015 7:30 PM To: user@cassandra.apache.org Subject: RE: Retrieving all row keys of a CF A few questions: 1) What is the heap size and total memory on each node? 2) How big is the cluster? 3) What are the read and range timeouts (in cassandra.yaml) on the C* nodes? 4) What are the timeouts for the Astyanax client? 5) Do you see GC pressure on the C* nodes? How long does GC for new gen and old gen take? 6) Does any node crash with OOM error when you try AllRowsReader? Mohammed From: Ravi Agrawal [mailto:ragra...@clearpoolgroup.com] Sent: Friday, January 16, 2015 4:14 PM To: user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: Re: Retrieving all row keys of a CF Hi, I and Ruchir tried query using AllRowsReader recipe but had no luck. We are seeing PoolTimeoutException. SEVERE: [Thread_1] Error reading RowKeys com.netflix.astyanax.connectionpool.exceptions.PoolTimeoutException: PoolTimeoutException: [host=servername, latency=2003(2003), attempts=4]Timed out waiting for connection at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.waitForConnection(SimpleHostConnectionPool.java:231) at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.borrowConnection(SimpleHostConnectionPool.java:198) at com.netflix.astyanax.connectionpool.impl.RoundRobinExecuteWithFailover.borrowConnection(RoundRobinExecuteWithFailover.java:84) at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:117) at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:338) at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$2.execute(ThriftColumnFamilyQueryImpl.java:397) at com.netflix.astyanax.recipes.reader.AllRowsReader$1.call(AllRowsReader.java:447) at com.netflix.astyanax.recipes.reader.AllRowsReader$1.call(AllRowsReader.java:419) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) We did receive a portion of data which changes on every try. We used following method. boolean result = new AllRowsReader.Builder<String, String>(keyspace, CF_STANDARD1) .withColumnRange(null, null, false, 0) .withPartitioner(null) // this will use keyspace's partitioner .forEachRow(new Function<Row<String, String>, Boolean>() { @Override public Boolean apply(@Nullable Row<String, String> row) { // Process the row here ... return true; } }) .build() .call(); Tried setting concurrency level as mentioned in this post (https://github.com/Netflix/astyanax/issues/411) as well on both astyanax 1.56.49 and 2.0.0. Still nothing.
