[jira] [Created] (CASSANDRA-5986) Cassandra hangs while reading saved cache

2013-09-09 Thread cuser (JIRA)
cuser created CASSANDRA-5986:


 Summary: Cassandra hangs while reading saved cache
 Key: CASSANDRA-5986
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5986
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: cuser
 Fix For: 2.0


We've got a cluster ~6Tb size running on 5 nodes with vnodes(256) enabled.

After some cache heat-up one of the nodes was restarted and cassandra just 
hangs with no error. Last message:

cassandra.log:
...
 INFO 11:12:27,649 Opening 
/cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 bytes)
 INFO 11:12:27,649 Opening 
/cassandra/data/usertable/data/usertable-data-ic-1339 (522628274110 bytes)
 INFO 11:12:49,224 reading saved cache 
/cassandra/saved_caches/usertable-data-KeyCache-b.db

system.log:
...
 INFO [SSTableBatchOpen:6] 2013-09-09 11:12:27,649 SSTableReader.java (line 
213) Opening /cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 
bytes)
 INFO [SSTableBatchOpen:7] 2013-09-09 11:12:27,649 SSTableReader.java (line 
213) Opening /cassandra/data/usertable/data/usertable-data-ic-1339 
(522628274110 bytes)
 INFO [CompactionExecutor:1] 2013-09-09 11:12:49,224 AutoSavingCache.java (line 
142) reading saved cache /cassandra/saved_caches/usertable-data-KeyCache-b.db


usertable-data-KeyCache-b.db size is about 3.5G.

All other nodes are restarting successfuly.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-5986) Cassandra hangs while reading saved cache

2013-09-09 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761684#comment-13761684
 ] 

Jonathan Ellis commented on CASSANDRA-5986:
---

It's probably running out of memory, slowly.  3.5G is a huge cache for a 
default 8GB heap.  Definitely recommend cutting the cache size down.

 Cassandra hangs while reading saved cache
 -

 Key: CASSANDRA-5986
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5986
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: cuser
 Fix For: 2.0


 We've got a cluster ~6Tb size running on 5 nodes with vnodes(256) enabled.
 After some cache heat-up one of the nodes was restarted and cassandra just 
 hangs with no error. Last message:
 cassandra.log:
 ...
  INFO 11:12:27,649 Opening 
 /cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 bytes)
  INFO 11:12:27,649 Opening 
 /cassandra/data/usertable/data/usertable-data-ic-1339 (522628274110 bytes)
  INFO 11:12:49,224 reading saved cache 
 /cassandra/saved_caches/usertable-data-KeyCache-b.db
 system.log:
 ...
  INFO [SSTableBatchOpen:6] 2013-09-09 11:12:27,649 SSTableReader.java (line 
 213) Opening /cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 
 bytes)
  INFO [SSTableBatchOpen:7] 2013-09-09 11:12:27,649 SSTableReader.java (line 
 213) Opening /cassandra/data/usertable/data/usertable-data-ic-1339 
 (522628274110 bytes)
  INFO [CompactionExecutor:1] 2013-09-09 11:12:49,224 AutoSavingCache.java 
 (line 142) reading saved cache 
 /cassandra/saved_caches/usertable-data-KeyCache-b.db
 usertable-data-KeyCache-b.db size is about 3.5G.
 All other nodes are restarting successfuly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[3/6] git commit: typo

2013-09-09 Thread jbellis
typo


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30f5e561
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30f5e561
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30f5e561

Branch: refs/heads/trunk
Commit: 30f5e561312ef7359d9a8ad0c3803e0ae150a4b7
Parents: b5f4d03
Author: Jonathan Ellis jbel...@apache.org
Authored: Mon Sep 9 08:57:02 2013 +0100
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Sep 9 08:57:02 2013 +0100

--
 src/java/org/apache/cassandra/cql3/QueryProcessor.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/30f5e561/src/java/org/apache/cassandra/cql3/QueryProcessor.java
--
diff --git a/src/java/org/apache/cassandra/cql3/QueryProcessor.java 
b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
index f3cea48..dbc9577 100644
--- a/src/java/org/apache/cassandra/cql3/QueryProcessor.java
+++ b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
@@ -233,7 +233,7 @@ public class QueryProcessor
 if (statement instanceof CFStatement)
 ((CFStatement)statement).prepareKeyspace(clientState);
 
-Tracing.trace(Peparing statement);
+Tracing.trace(Preparing statement);
 return statement.prepare();
 }
 



[1/6] git commit: typo

2013-09-09 Thread jbellis
Updated Branches:
  refs/heads/cassandra-1.2 b5f4d03d0 - 30f5e5613
  refs/heads/cassandra-2.0 b248b1ae3 - ce9d3030b
  refs/heads/trunk 5fd463184 - 3b8690db5


typo


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30f5e561
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30f5e561
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30f5e561

Branch: refs/heads/cassandra-1.2
Commit: 30f5e561312ef7359d9a8ad0c3803e0ae150a4b7
Parents: b5f4d03
Author: Jonathan Ellis jbel...@apache.org
Authored: Mon Sep 9 08:57:02 2013 +0100
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Sep 9 08:57:02 2013 +0100

--
 src/java/org/apache/cassandra/cql3/QueryProcessor.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/30f5e561/src/java/org/apache/cassandra/cql3/QueryProcessor.java
--
diff --git a/src/java/org/apache/cassandra/cql3/QueryProcessor.java 
b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
index f3cea48..dbc9577 100644
--- a/src/java/org/apache/cassandra/cql3/QueryProcessor.java
+++ b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
@@ -233,7 +233,7 @@ public class QueryProcessor
 if (statement instanceof CFStatement)
 ((CFStatement)statement).prepareKeyspace(clientState);
 
-Tracing.trace(Peparing statement);
+Tracing.trace(Preparing statement);
 return statement.prepare();
 }
 



[4/6] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

2013-09-09 Thread jbellis
Merge branch 'cassandra-1.2' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ce9d3030
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ce9d3030
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ce9d3030

Branch: refs/heads/trunk
Commit: ce9d3030b876283f8b1e5d376f6238ec524ee6b8
Parents: b248b1a 30f5e56
Author: Jonathan Ellis jbel...@apache.org
Authored: Mon Sep 9 08:57:09 2013 +0100
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Sep 9 08:57:09 2013 +0100

--
 src/java/org/apache/cassandra/cql3/QueryProcessor.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce9d3030/src/java/org/apache/cassandra/cql3/QueryProcessor.java
--



[5/6] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

2013-09-09 Thread jbellis
Merge branch 'cassandra-1.2' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ce9d3030
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ce9d3030
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ce9d3030

Branch: refs/heads/cassandra-2.0
Commit: ce9d3030b876283f8b1e5d376f6238ec524ee6b8
Parents: b248b1a 30f5e56
Author: Jonathan Ellis jbel...@apache.org
Authored: Mon Sep 9 08:57:09 2013 +0100
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Sep 9 08:57:09 2013 +0100

--
 src/java/org/apache/cassandra/cql3/QueryProcessor.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce9d3030/src/java/org/apache/cassandra/cql3/QueryProcessor.java
--



[6/6] git commit: Merge branch 'cassandra-2.0' into trunk

2013-09-09 Thread jbellis
Merge branch 'cassandra-2.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3b8690db
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3b8690db
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3b8690db

Branch: refs/heads/trunk
Commit: 3b8690db5e78e3c8bec6f07a2c9a151212412a70
Parents: 5fd4631 ce9d303
Author: Jonathan Ellis jbel...@apache.org
Authored: Mon Sep 9 08:57:52 2013 +0100
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Sep 9 08:57:52 2013 +0100

--
 src/java/org/apache/cassandra/cql3/QueryProcessor.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--




[2/6] git commit: typo

2013-09-09 Thread jbellis
typo


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30f5e561
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30f5e561
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30f5e561

Branch: refs/heads/cassandra-2.0
Commit: 30f5e561312ef7359d9a8ad0c3803e0ae150a4b7
Parents: b5f4d03
Author: Jonathan Ellis jbel...@apache.org
Authored: Mon Sep 9 08:57:02 2013 +0100
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Sep 9 08:57:02 2013 +0100

--
 src/java/org/apache/cassandra/cql3/QueryProcessor.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/30f5e561/src/java/org/apache/cassandra/cql3/QueryProcessor.java
--
diff --git a/src/java/org/apache/cassandra/cql3/QueryProcessor.java 
b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
index f3cea48..dbc9577 100644
--- a/src/java/org/apache/cassandra/cql3/QueryProcessor.java
+++ b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
@@ -233,7 +233,7 @@ public class QueryProcessor
 if (statement instanceof CFStatement)
 ((CFStatement)statement).prepareKeyspace(clientState);
 
-Tracing.trace(Peparing statement);
+Tracing.trace(Preparing statement);
 return statement.prepare();
 }
 



[jira] [Updated] (CASSANDRA-5979) HintedHandOffManager task triggering java.util.NoSuchElementException

2013-09-09 Thread Jonathan Ellis (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5979:
--

Attachment: 5979.txt

Patch to special case the empty columns request.

 HintedHandOffManager task triggering java.util.NoSuchElementException
 -

 Key: CASSANDRA-5979
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5979
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ravi Prasad
 Attachments: 5979.txt


 Seeing this exception every 10 minutes, whenever scheduled HintedHandoff task 
 runs:
 ERROR [OptionalTasks:1] 2013-09-05 20:48:00,334 CassandraDaemon.java (line 
 192) Exception in thread Thread[OptionalTasks:1,5,main]
 java.util.NoSuchElementException
   at 
 com.google.common.collect.EmptyImmutableSortedSet.last(EmptyImmutableSortedSet.java:100)
   at 
 org.apache.cassandra.db.columniterator.SSTableNamesIterator.readSimpleColumns(SSTableNamesIterator.java:186)
   at 
 org.apache.cassandra.db.columniterator.SSTableNamesIterator.read(SSTableNamesIterator.java:156)
   at 
 org.apache.cassandra.db.columniterator.SSTableNamesIterator.init(SSTableNamesIterator.java:83)
   at 
 org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:86)
   at 
 org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:75)
   at 
 org.apache.cassandra.io.sstable.SSTableScanner$FilteredKeyScanningIterator$1.create(SSTableScanner.java:248)
   at 
 org.apache.cassandra.db.columniterator.LazyColumnIterator.getSubIterator(LazyColumnIterator.java:75)
   at 
 org.apache.cassandra.db.columniterator.LazyColumnIterator.getColumnFamily(LazyColumnIterator.java:87)
   at 
 org.apache.cassandra.db.RowIteratorFactory$2.reduce(RowIteratorFactory.java:95)
   at 
 org.apache.cassandra.db.RowIteratorFactory$2.reduce(RowIteratorFactory.java:79)
   at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:111)
   at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:97)
   at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
   at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
   at 
 org.apache.cassandra.db.ColumnFamilyStore$6.computeNext(ColumnFamilyStore.java:1433)
   at 
 org.apache.cassandra.db.ColumnFamilyStore$6.computeNext(ColumnFamilyStore.java:1429)
   at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
   at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
   at 
 org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1500)
   at 
 org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1477)
   at 
 org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1472)
   at 
 org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:464)
   at 
 org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:90)
   at 
 org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:133)
   at 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at 
 java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:722)
 this seems to be a regression from 
 https://issues.apache.org/jira/browse/CASSANDRA-5966. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-5986) Cassandra hangs while reading saved cache

2013-09-09 Thread cuser (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761690#comment-13761690
 ] 

cuser commented on CASSANDRA-5986:
--

Each node has a 90G of RAM and 24 CPU cores(with HT enabled, ofcourse)
We have tried to increase heap size up to 16G and then 20+ G with no luck.
Reducing key cache size affecting random read perfomance heavily.

Is there any other solution?



 Cassandra hangs while reading saved cache
 -

 Key: CASSANDRA-5986
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5986
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: cuser
 Fix For: 2.0


 We've got a cluster ~6Tb size running on 5 nodes with vnodes(256) enabled.
 After some cache heat-up one of the nodes was restarted and cassandra just 
 hangs with no error. Last message:
 cassandra.log:
 ...
  INFO 11:12:27,649 Opening 
 /cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 bytes)
  INFO 11:12:27,649 Opening 
 /cassandra/data/usertable/data/usertable-data-ic-1339 (522628274110 bytes)
  INFO 11:12:49,224 reading saved cache 
 /cassandra/saved_caches/usertable-data-KeyCache-b.db
 system.log:
 ...
  INFO [SSTableBatchOpen:6] 2013-09-09 11:12:27,649 SSTableReader.java (line 
 213) Opening /cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 
 bytes)
  INFO [SSTableBatchOpen:7] 2013-09-09 11:12:27,649 SSTableReader.java (line 
 213) Opening /cassandra/data/usertable/data/usertable-data-ic-1339 
 (522628274110 bytes)
  INFO [CompactionExecutor:1] 2013-09-09 11:12:49,224 AutoSavingCache.java 
 (line 142) reading saved cache 
 /cassandra/saved_caches/usertable-data-KeyCache-b.db
 usertable-data-KeyCache-b.db size is about 3.5G.
 All other nodes are restarting successfuly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-5986) Cassandra hangs while reading saved cache

2013-09-09 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761697#comment-13761697
 ] 

Jonathan Ellis commented on CASSANDRA-5986:
---

What does the log say?  Does top say it's busy GCing?  Try jtop here to see 
which actual threads are busy: https://github.com/jasta/jprocps

 Cassandra hangs while reading saved cache
 -

 Key: CASSANDRA-5986
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5986
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: cuser
 Fix For: 2.0


 We've got a cluster ~6Tb size running on 5 nodes with vnodes(256) enabled.
 After some cache heat-up one of the nodes was restarted and cassandra just 
 hangs with no error. Last message:
 cassandra.log:
 ...
  INFO 11:12:27,649 Opening 
 /cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 bytes)
  INFO 11:12:27,649 Opening 
 /cassandra/data/usertable/data/usertable-data-ic-1339 (522628274110 bytes)
  INFO 11:12:49,224 reading saved cache 
 /cassandra/saved_caches/usertable-data-KeyCache-b.db
 system.log:
 ...
  INFO [SSTableBatchOpen:6] 2013-09-09 11:12:27,649 SSTableReader.java (line 
 213) Opening /cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 
 bytes)
  INFO [SSTableBatchOpen:7] 2013-09-09 11:12:27,649 SSTableReader.java (line 
 213) Opening /cassandra/data/usertable/data/usertable-data-ic-1339 
 (522628274110 bytes)
  INFO [CompactionExecutor:1] 2013-09-09 11:12:49,224 AutoSavingCache.java 
 (line 142) reading saved cache 
 /cassandra/saved_caches/usertable-data-KeyCache-b.db
 usertable-data-KeyCache-b.db size is about 3.5G.
 All other nodes are restarting successfuly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-5986) Cassandra hangs while reading saved cache

2013-09-09 Thread cuser (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761730#comment-13761730
 ] 

cuser commented on CASSANDRA-5986:
--

Nothing new in logs:
INFO 11:12:49,224 reading saved cache 
/cassandra/saved_caches/usertable-data-KeyCache-b.db
INFO [CompactionExecutor:1] 2013-09-09 11:12:49,224 AutoSavingCache.java (line 
142) reading saved cache /cassandra/saved_caches/usertable-data-KeyCache-b.db

jtop shows ??? instead of process(thread?) names.

  PID   TID USER  %CPU  %MEM THREAD 


27465 27491 492   60.2  28.3 ???


27465 27466 492   16.6  28.3 ???

Here is some jstat output, maybe it will be useful

# jstat -gc 27465 1000
 S0CS1CS0US1U  EC   EUOC OU   PC PU 
   YGC YGCTFGCFGCT GCT   
450560.0 450560.0 450560.0  0.0   3604480.0 3604480.0 21708800.0 21708772.2 
30984.0 18488.1 28   41.718   7191.788  233.506
450560.0 450560.0 450560.0  0.0   3604480.0 3604480.0 21708800.0 21708772.2 
30984.0 18488.1 28   41.718   7191.788  233.506
450560.0 450560.0 450560.0  0.0   3604480.0 3604480.0 21708800.0 21708772.2 
30984.0 18488.1 28   41.718   7191.788  233.506


 Cassandra hangs while reading saved cache
 -

 Key: CASSANDRA-5986
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5986
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: cuser
 Fix For: 2.0


 We've got a cluster ~6Tb size running on 5 nodes with vnodes(256) enabled.
 After some cache heat-up one of the nodes was restarted and cassandra just 
 hangs with no error. Last message:
 cassandra.log:
 ...
  INFO 11:12:27,649 Opening 
 /cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 bytes)
  INFO 11:12:27,649 Opening 
 /cassandra/data/usertable/data/usertable-data-ic-1339 (522628274110 bytes)
  INFO 11:12:49,224 reading saved cache 
 /cassandra/saved_caches/usertable-data-KeyCache-b.db
 system.log:
 ...
  INFO [SSTableBatchOpen:6] 2013-09-09 11:12:27,649 SSTableReader.java (line 
 213) Opening /cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 
 bytes)
  INFO [SSTableBatchOpen:7] 2013-09-09 11:12:27,649 SSTableReader.java (line 
 213) Opening /cassandra/data/usertable/data/usertable-data-ic-1339 
 (522628274110 bytes)
  INFO [CompactionExecutor:1] 2013-09-09 11:12:49,224 AutoSavingCache.java 
 (line 142) reading saved cache 
 /cassandra/saved_caches/usertable-data-KeyCache-b.db
 usertable-data-KeyCache-b.db size is about 3.5G.
 All other nodes are restarting successfuly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-5939) Cache Providers calculate very different row sizes

2013-09-09 Thread Chris Burroughs (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761791#comment-13761791
 ] 

Chris Burroughs commented on CASSANDRA-5939:


I'm unlikely to have the time to dig into this myself. 

 Cache Providers calculate very different row sizes
 --

 Key: CASSANDRA-5939
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5939
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 1.2.8
Reporter: Chris Burroughs
Assignee: Vijay

 Took the same production node and bounced it 4 times comparing version and 
 cache provider.  ConcurrentLinkedHashCacheProvider and 
 SerializingCacheProvider produce very different results resulting in an order 
 of magnitude difference in rows cached.  In all cases the row cache size was 
 2048 MB.  Hit rate is provided for color, but entries  size are the 
 important part.
 1.2.8 ConcurrentLinkedHashCacheProvider:
  * entries: 23,217
  * hit rate: 43%
  * size: 2,147,398,344
 1.2.8 about 20 minutes of SerializingCacheProvider:
  * entries: 221,709
  * hit rate: 68%
  * size: 18,417254
 1.2.5 ConcurrentLinkedHashCacheProvider:
  * entries: 25,967
  * hit rate: ~ 50%
  * size:  2,147,421,704
 1.2.5 about 20 minutes of SerializingCacheProvider:
  * entries: 228,457
  * hit rate: ~ 70%
  * size: 19,070,315
 A related(?) problem is that the ConcurrentLinkedHashCacheProvider sizes seem 
 to be highly variable.  Digging up the values for 5 different nodes in the 
 cluster using ConcurrentLinkedHashCacheProvider shows a wide variance in 
 number of entries:
  * 12k
  * 444k
  * 10k
  * 25k
  * 25k

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Assigned] (CASSANDRA-5985) Paxos replay of in progress update is incorrect

2013-09-09 Thread Sylvain Lebresne (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne reassigned CASSANDRA-5985:
---

Assignee: Sylvain Lebresne  (was: Jonathan Ellis)

 Paxos replay of in progress update is incorrect
 ---

 Key: CASSANDRA-5985
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5985
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
 Fix For: 2.0.1


 When we replay {{inProgress}}, we need to refresh it with the newly prepared 
 ballot, or it will be (correctly) rejected.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-5985) Paxos replay of in progress update is incorrect

2013-09-09 Thread Sylvain Lebresne (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761827#comment-13761827
 ] 

Sylvain Lebresne commented on CASSANDRA-5985:
-

One nit: I'm not sure sleeping in the case where the propose work (and we 
commit) is really useful, the commit itself probably play a good enough sleep 
if we're contending (which, for what its worth, is confirmed by my unscientific 
test: moving the sleep only if the propose fails perform slightly better 
without making threads retry more often).

But overall, +1 on the fix (a quick test (that I'll commit to dtests) does 
confirm the current CAS timeout as soon a 2 inserts contend a bit too much).



 Paxos replay of in progress update is incorrect
 ---

 Key: CASSANDRA-5985
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5985
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
 Fix For: 2.0.1


 When we replay {{inProgress}}, we need to refresh it with the newly prepared 
 ballot, or it will be (correctly) rejected.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Created] (CASSANDRA-5987) IllegalArgumentException when bulk loading

2013-09-09 Thread Henrik Baastrup (JIRA)
Henrik Baastrup created CASSANDRA-5987:
--

 Summary: IllegalArgumentException when bulk loading
 Key: CASSANDRA-5987
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.0.0 on Linux
Reporter: Henrik Baastrup


I have created SSTables with the 
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I 
try to load the SSTables with sstableloader -d localhost nsmo_ds/nsmo_ds I 
get:
 WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
Streaming to the following hosts failed:
[/127.0.0.1]
java.util.concurrent.ExecutionException: 
org.apache.cassandra.streaming.StreamException: Stream failed
 WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming 
error occurred
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(Unknown Source)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
at sun.nio.ch.IOUtil.write(Unknown Source)
at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
at 
org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
at 
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
at 
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
at java.lang.Thread.run(Unknown Source)
 WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed

in the server log I have:
ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java 
(line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error 
occurred
java.io.IOException: Too many retries for Header (cfId: 
976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208, 
transfer size: 54794676, compressed?: false)
at 
org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
at 
org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
at 
org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
at 
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
at 
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Unknown Source)
at 
org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
at 
org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
at 
org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130)
at 
org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101)
at 
org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254)
at 
org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134)
at 
org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88)
at 
org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55)
... 4 more
 INFO [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,862 
StreamResultFuture.java (line 180) [Stream 
#7d0193a0-1945-11e3-ac84-f9df5751357f] Session with /172.27.14.242 is complete
 WARN [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,864 
StreamResultFuture.java (line 209) [Stream 
#7d0193a0-1945-11e3-ac84-f9df5751357f] Stream fail



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-5987) IllegalArgumentException when bulk loading

2013-09-09 Thread Henrik Baastrup (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761865#comment-13761865
 ] 

Henrik Baastrup commented on CASSANDRA-5987:


I have debugged Cassandra a bit and found the error occurs in the 
org.apache.cassandra.db.marshal.AbstractCompositeType, which also can be seen 
by the stack trace from the server log.

The problem is in the ByteBuffer passes to the getWithShortLength method: There 
is no length contained in the buffer, only a field name for the actual table. 
The local length variable will be constructed by using the two first characters 
of the name. This will later provoke the IllegalArgumentException when the 
position for the buffer is moved using the wrong calculated length.

My question is is it correct that the passed ByteBuffer does not contains a 
length, or does the SSTableSimpleUnsortedWriter class mix up someting when it 
creates it's SSTables?

 IllegalArgumentException when bulk loading
 --

 Key: CASSANDRA-5987
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.0.0 on Linux
Reporter: Henrik Baastrup

 I have created SSTables with the 
 org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I 
 try to load the SSTables with sstableloader -d localhost nsmo_ds/nsmo_ds I 
 get:
  WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 Streaming to the following hosts failed:
 [/127.0.0.1]
 java.util.concurrent.ExecutionException: 
 org.apache.cassandra.streaming.StreamException: Stream failed
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming 
 error occurred
 java.io.IOException: Broken pipe
   at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
   at sun.nio.ch.SocketDispatcher.write(Unknown Source)
   at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
   at sun.nio.ch.IOUtil.write(Unknown Source)
   at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
   at java.lang.Thread.run(Unknown Source)
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 in the server log I have:
 ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java 
 (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error 
 occurred
 java.io.IOException: Too many retries for Header (cfId: 
 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208, 
 transfer size: 54794676, compressed?: false)
   at 
 org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
   at java.lang.Thread.run(Unknown Source)
 Caused by: java.lang.IllegalArgumentException
   at java.nio.Buffer.limit(Unknown Source)
   at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
   at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
   at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130)
   at 
 org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254)
   at 
 org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134)
   at 
 org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55)
   ... 4 more
  INFO [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,862 
 StreamResultFuture.java (line 180) [Stream 
 #7d0193a0-1945-11e3-ac84-f9df5751357f] Session with /172.27.14.242 is complete
  WARN [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,864 
 StreamResultFuture.java (line 209) [Stream 
 #7d0193a0-1945-11e3-ac84-f9df5751357f] 

[jira] [Updated] (CASSANDRA-5987) IllegalArgumentException when bulk loading

2013-09-09 Thread Henrik Baastrup (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Henrik Baastrup updated CASSANDRA-5987:
---

Description: 
I have created SSTables with the 
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I 
try to load the SSTables with sstableloader -d localhost nsmo_ds/nsmo_ds I 
get:

 WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
Streaming to the following hosts failed:
{quote}
[/127.0.0.1]
java.util.concurrent.ExecutionException: 
org.apache.cassandra.streaming.StreamException: Stream failed
 WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming 
error occurred
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(Unknown Source)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
at sun.nio.ch.IOUtil.write(Unknown Source)
at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
at 
org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
at 
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
at 
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
at java.lang.Thread.run(Unknown Source)
 WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
{quote}

in the server log I have:
{quote}
ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java 
(line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error 
occurred
java.io.IOException: Too many retries for Header (cfId: 
976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208, 
transfer size: 54794676, compressed?: false)
at 
org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
at 
org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
at 
org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
at 
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
at 
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Unknown Source)
at 
org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
at 
org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
at 
org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130)
at 
org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101)
at 
org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254)
at 
org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134)
at 
org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88)
at 
org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55)
... 4 more
 INFO [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,862 
StreamResultFuture.java (line 180) [Stream 
#7d0193a0-1945-11e3-ac84-f9df5751357f] Session with /172.27.14.242 is complete
 WARN [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,864 
StreamResultFuture.java (line 209) [Stream 
#7d0193a0-1945-11e3-ac84-f9df5751357f] Stream fail
{quote}


  was:
I have created SSTables with the 
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I 
try to load the SSTables with sstableloader -d localhost nsmo_ds/nsmo_ds I 
get:
 WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
Streaming to the following hosts failed:
[/127.0.0.1]
java.util.concurrent.ExecutionException: 
org.apache.cassandra.streaming.StreamException: Stream failed
 WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming 
error occurred
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(Unknown Source)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
at sun.nio.ch.IOUtil.write(Unknown Source)
at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
at 
org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
at 

[jira] [Comment Edited] (CASSANDRA-5987) IllegalArgumentException when bulk loading

2013-09-09 Thread Henrik Baastrup (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761865#comment-13761865
 ] 

Henrik Baastrup edited comment on CASSANDRA-5987 at 9/9/13 2:13 PM:


I have debugged Cassandra a bit and found the error occurs in the 
*org.apache.cassandra.db.marshal.AbstractCompositeType*, which also can be seen 
by the stack trace from the server log.

The problem is in the ByteBuffer passes to the *getWithShortLength* method: 
There is no length contained in the buffer, only a field name for the actual 
table. The local length variable will be constructed by using the two first 
characters of the name. This will later provoke the *IllegalArgumentException* 
when the position for the buffer is moved using the wrong calculated length.

My question is is it correct that the passed ByteBuffer does not contains a 
length, or does the SSTableSimpleUnsortedWriter class mix up someting when it 
creates it's SSTables?

  was (Author: hbaastrup):
I have debugged Cassandra a bit and found the error occurs in the 
org.apache.cassandra.db.marshal.AbstractCompositeType, which also can be seen 
by the stack trace from the server log.

The problem is in the ByteBuffer passes to the getWithShortLength method: There 
is no length contained in the buffer, only a field name for the actual table. 
The local length variable will be constructed by using the two first characters 
of the name. This will later provoke the IllegalArgumentException when the 
position for the buffer is moved using the wrong calculated length.

My question is is it correct that the passed ByteBuffer does not contains a 
length, or does the SSTableSimpleUnsortedWriter class mix up someting when it 
creates it's SSTables?
  
 IllegalArgumentException when bulk loading
 --

 Key: CASSANDRA-5987
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.0.0 on Linux
Reporter: Henrik Baastrup

 I have created SSTables with the 
 org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I 
 try to load the SSTables with sstableloader -d localhost nsmo_ds/nsmo_ds I 
 get:
  WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 Streaming to the following hosts failed:
 {quote}
 [/127.0.0.1]
 java.util.concurrent.ExecutionException: 
 org.apache.cassandra.streaming.StreamException: Stream failed
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming 
 error occurred
 java.io.IOException: Broken pipe
   at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
   at sun.nio.ch.SocketDispatcher.write(Unknown Source)
   at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
   at sun.nio.ch.IOUtil.write(Unknown Source)
   at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
   at java.lang.Thread.run(Unknown Source)
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 {quote}
 in the server log I have:
 {quote}
 ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java 
 (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error 
 occurred
 java.io.IOException: Too many retries for Header (cfId: 
 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208, 
 transfer size: 54794676, compressed?: false)
   at 
 org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
   at java.lang.Thread.run(Unknown Source)
 Caused by: java.lang.IllegalArgumentException
   at java.nio.Buffer.limit(Unknown Source)
   at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
   at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
   at 
 

[jira] [Updated] (CASSANDRA-5987) IllegalArgumentException when bulk loading

2013-09-09 Thread Henrik Baastrup (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Henrik Baastrup updated CASSANDRA-5987:
---

Attachment: CassandraLoader.tgz

Attached my program there create the SSTables. Set-up the classpath to contain 
all necessary Cassandra jars and lunch the program like this:
java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf

Cassandra must contains a key space name nsmo_ds with the following table:
REATE TABLE nsmo_dns (
  endtime bigint,
  lastframetime bigint,
  begintime bigint,
  moduleid int,
  callsequencenumber bigint,
  partialcdanum int,
  server_ip_ip0 bigint,
  server_ip_ip1 bigint,
  server_ip_ip2 bigint,
  server_ip_ip3 bigint,
  client_ip_ip0 bigint,
  client_ip_ip1 bigint,
  client_ip_ip2 bigint,
  client_ip_ip3 bigint,
  ip_src_ip0 bigint,
  ip_src_ip1 bigint,
  ip_src_ip2 bigint,
  ip_src_ip3 bigint,
  dnsenume164 varchar,
  callduration bigint,
  e164number varchar,
  responsecode int,
  internalcauses int,
  queryname varchar,
  querytype int,
  queryclass int,
  PRIMARY KEY (callsequencenumber,moduleid)
) ;


 IllegalArgumentException when bulk loading
 --

 Key: CASSANDRA-5987
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.0.0 on Linux
Reporter: Henrik Baastrup
 Attachments: CassandraLoader.tgz


 I have created SSTables with the 
 org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I 
 try to load the SSTables with sstableloader -d localhost nsmo_ds/nsmo_ds I 
 get:
  WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 Streaming to the following hosts failed:
 {quote}
 [/127.0.0.1]
 java.util.concurrent.ExecutionException: 
 org.apache.cassandra.streaming.StreamException: Stream failed
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming 
 error occurred
 java.io.IOException: Broken pipe
   at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
   at sun.nio.ch.SocketDispatcher.write(Unknown Source)
   at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
   at sun.nio.ch.IOUtil.write(Unknown Source)
   at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
   at java.lang.Thread.run(Unknown Source)
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 {quote}
 in the server log I have:
 {quote}
 ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java 
 (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error 
 occurred
 java.io.IOException: Too many retries for Header (cfId: 
 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208, 
 transfer size: 54794676, compressed?: false)
   at 
 org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
   at java.lang.Thread.run(Unknown Source)
 Caused by: java.lang.IllegalArgumentException
   at java.nio.Buffer.limit(Unknown Source)
   at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
   at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
   at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130)
   at 
 org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254)
   at 
 org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134)
   at 
 org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55)
   ... 4 more
  INFO [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,862 
 StreamResultFuture.java (line 180) [Stream 
 

[jira] [Comment Edited] (CASSANDRA-5987) IllegalArgumentException when bulk loading

2013-09-09 Thread Henrik Baastrup (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761882#comment-13761882
 ] 

Henrik Baastrup edited comment on CASSANDRA-5987 at 9/9/13 2:30 PM:


Attached my program there create the SSTables. Set-up the classpath to contain 
all necessary Cassandra jars and lunch the program like this:
java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf

Cassandra must contains a key space name *nsmo_ds* with the following table:
{quote}
CREATE TABLE nsmo_dns (
  endtime bigint,
  lastframetime bigint,
  begintime bigint,
  moduleid int,
  callsequencenumber bigint,
  partialcdanum int,
  server_ip_ip0 bigint,
  server_ip_ip1 bigint,
  server_ip_ip2 bigint,
  server_ip_ip3 bigint,
  client_ip_ip0 bigint,
  client_ip_ip1 bigint,
  client_ip_ip2 bigint,
  client_ip_ip3 bigint,
  ip_src_ip0 bigint,
  ip_src_ip1 bigint,
  ip_src_ip2 bigint,
  ip_src_ip3 bigint,
  dnsenume164 varchar,
  callduration bigint,
  e164number varchar,
  responsecode int,
  internalcauses int,
  queryname varchar,
  querytype int,
  queryclass int,
  PRIMARY KEY (callsequencenumber,moduleid)
) ;
{quote}


  was (Author: hbaastrup):
Attached my program there create the SSTables. Set-up the classpath to 
contain all necessary Cassandra jars and lunch the program like this:
java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf

Cassandra must contains a key space name nsmo_ds with the following table:
REATE TABLE nsmo_dns (
  endtime bigint,
  lastframetime bigint,
  begintime bigint,
  moduleid int,
  callsequencenumber bigint,
  partialcdanum int,
  server_ip_ip0 bigint,
  server_ip_ip1 bigint,
  server_ip_ip2 bigint,
  server_ip_ip3 bigint,
  client_ip_ip0 bigint,
  client_ip_ip1 bigint,
  client_ip_ip2 bigint,
  client_ip_ip3 bigint,
  ip_src_ip0 bigint,
  ip_src_ip1 bigint,
  ip_src_ip2 bigint,
  ip_src_ip3 bigint,
  dnsenume164 varchar,
  callduration bigint,
  e164number varchar,
  responsecode int,
  internalcauses int,
  queryname varchar,
  querytype int,
  queryclass int,
  PRIMARY KEY (callsequencenumber,moduleid)
) ;

  
 IllegalArgumentException when bulk loading
 --

 Key: CASSANDRA-5987
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.0.0 on Linux
Reporter: Henrik Baastrup
 Attachments: CassandraLoader.tgz


 I have created SSTables with the 
 org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I 
 try to load the SSTables with sstableloader -d localhost nsmo_ds/nsmo_ds I 
 get:
  WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 Streaming to the following hosts failed:
 {quote}
 [/127.0.0.1]
 java.util.concurrent.ExecutionException: 
 org.apache.cassandra.streaming.StreamException: Stream failed
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming 
 error occurred
 java.io.IOException: Broken pipe
   at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
   at sun.nio.ch.SocketDispatcher.write(Unknown Source)
   at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
   at sun.nio.ch.IOUtil.write(Unknown Source)
   at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
   at java.lang.Thread.run(Unknown Source)
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 {quote}
 in the server log I have:
 {quote}
 ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java 
 (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error 
 occurred
 java.io.IOException: Too many retries for Header (cfId: 
 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208, 
 transfer size: 54794676, compressed?: false)
   at 
 org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)

[jira] [Comment Edited] (CASSANDRA-5987) IllegalArgumentException when bulk loading

2013-09-09 Thread Henrik Baastrup (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761882#comment-13761882
 ] 

Henrik Baastrup edited comment on CASSANDRA-5987 at 9/9/13 2:31 PM:


Attached my program there create the SSTables. Set-up the classpath to contain 
all necessary Cassandra jars and lunch the program like this:

*java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf*

Cassandra must contains a key space name *nsmo_ds* with the following table:
{quote}
CREATE TABLE nsmo_dns (
  endtime bigint,
  lastframetime bigint,
  begintime bigint,
  moduleid int,
  callsequencenumber bigint,
  partialcdanum int,
  server_ip_ip0 bigint,
  server_ip_ip1 bigint,
  server_ip_ip2 bigint,
  server_ip_ip3 bigint,
  client_ip_ip0 bigint,
  client_ip_ip1 bigint,
  client_ip_ip2 bigint,
  client_ip_ip3 bigint,
  ip_src_ip0 bigint,
  ip_src_ip1 bigint,
  ip_src_ip2 bigint,
  ip_src_ip3 bigint,
  dnsenume164 varchar,
  callduration bigint,
  e164number varchar,
  responsecode int,
  internalcauses int,
  queryname varchar,
  querytype int,
  queryclass int,
  PRIMARY KEY (callsequencenumber,moduleid)
) ;
{quote}


  was (Author: hbaastrup):
Attached my program there create the SSTables. Set-up the classpath to 
contain all necessary Cassandra jars and lunch the program like this:
java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf

Cassandra must contains a key space name *nsmo_ds* with the following table:
{quote}
CREATE TABLE nsmo_dns (
  endtime bigint,
  lastframetime bigint,
  begintime bigint,
  moduleid int,
  callsequencenumber bigint,
  partialcdanum int,
  server_ip_ip0 bigint,
  server_ip_ip1 bigint,
  server_ip_ip2 bigint,
  server_ip_ip3 bigint,
  client_ip_ip0 bigint,
  client_ip_ip1 bigint,
  client_ip_ip2 bigint,
  client_ip_ip3 bigint,
  ip_src_ip0 bigint,
  ip_src_ip1 bigint,
  ip_src_ip2 bigint,
  ip_src_ip3 bigint,
  dnsenume164 varchar,
  callduration bigint,
  e164number varchar,
  responsecode int,
  internalcauses int,
  queryname varchar,
  querytype int,
  queryclass int,
  PRIMARY KEY (callsequencenumber,moduleid)
) ;
{quote}

  
 IllegalArgumentException when bulk loading
 --

 Key: CASSANDRA-5987
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.0.0 on Linux
Reporter: Henrik Baastrup
 Attachments: CassandraLoader.tgz


 I have created SSTables with the 
 org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I 
 try to load the SSTables with sstableloader -d localhost nsmo_ds/nsmo_ds I 
 get:
  WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 Streaming to the following hosts failed:
 {quote}
 [/127.0.0.1]
 java.util.concurrent.ExecutionException: 
 org.apache.cassandra.streaming.StreamException: Stream failed
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming 
 error occurred
 java.io.IOException: Broken pipe
   at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
   at sun.nio.ch.SocketDispatcher.write(Unknown Source)
   at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
   at sun.nio.ch.IOUtil.write(Unknown Source)
   at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
   at java.lang.Thread.run(Unknown Source)
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 {quote}
 in the server log I have:
 {quote}
 ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java 
 (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error 
 occurred
 java.io.IOException: Too many retries for Header (cfId: 
 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208, 
 transfer size: 54794676, compressed?: false)
   at 
 org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
   at 
 

[jira] [Commented] (CASSANDRA-5987) IllegalArgumentException when bulk loading

2013-09-09 Thread Yuki Morishita (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761964#comment-13761964
 ] 

Yuki Morishita commented on CASSANDRA-5987:
---

I think you need to modify the way to create SSTableSimpleUnsortedWriter.
This probably helps. 
https://github.com/pmcfadin/cql3-bulk-loader-example/blob/master/src/main/java/com/datastax/bulkloader/CQL3SimpleDataImportExample.java#L93

 IllegalArgumentException when bulk loading
 --

 Key: CASSANDRA-5987
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.0.0 on Linux
Reporter: Henrik Baastrup
 Attachments: CassandraLoader.tgz


 I have created SSTables with the 
 org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I 
 try to load the SSTables with sstableloader -d localhost nsmo_ds/nsmo_ds I 
 get:
  WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 Streaming to the following hosts failed:
 {quote}
 [/127.0.0.1]
 java.util.concurrent.ExecutionException: 
 org.apache.cassandra.streaming.StreamException: Stream failed
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming 
 error occurred
 java.io.IOException: Broken pipe
   at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
   at sun.nio.ch.SocketDispatcher.write(Unknown Source)
   at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
   at sun.nio.ch.IOUtil.write(Unknown Source)
   at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
   at java.lang.Thread.run(Unknown Source)
  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream 
 failed
 {quote}
 in the server log I have:
 {quote}
 ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java 
 (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error 
 occurred
 java.io.IOException: Too many retries for Header (cfId: 
 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208, 
 transfer size: 54794676, compressed?: false)
   at 
 org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
   at 
 org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
   at 
 org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
   at java.lang.Thread.run(Unknown Source)
 Caused by: java.lang.IllegalArgumentException
   at java.nio.Buffer.limit(Unknown Source)
   at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
   at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
   at 
 org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130)
   at 
 org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101)
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254)
   at 
 org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134)
   at 
 org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88)
   at 
 org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55)
   ... 4 more
  INFO [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,862 
 StreamResultFuture.java (line 180) [Stream 
 #7d0193a0-1945-11e3-ac84-f9df5751357f] Session with /172.27.14.242 is complete
  WARN [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,864 
 StreamResultFuture.java (line 209) [Stream 
 #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream fail
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Created] (CASSANDRA-5989) java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2013-09-09 Thread Karl Mueller (JIRA)
Karl Mueller created CASSANDRA-5989:
---

 Summary: java.lang.OutOfMemoryError: Requested array size exceeds 
VM limit
 Key: CASSANDRA-5989
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5989
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra 1.2.8
Oracle Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
RHEL6

Reporter: Karl Mueller


This occurred in one of our nodes today. I don't have any helpful information 
on what is going on beforehand yet - logs don't have anything I could see 
that's tied for sure to it.

A few things happened in the logs beforehand. A little bit of standard GC, a 
bunch of status-logger entries 10 minutes before the crash, and a few nodes 
going up and down on the gossip.


ERROR [Thrift:7495] 2013-09-03 11:01:12,486 CassandraDaemon.java (line 192) 
Exception in thread Thread[Thrift:7495,5,main]
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at java.util.Arrays.copyOf(Arrays.java:2271)
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
at 
java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
at 
org.apache.thrift.transport.TFramedTransport.write(TFramedTransport.java:146)
at 
org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163)
at 
org.apache.cassandra.thrift.TBinaryProtocol.writeBinary(TBinaryProtocol.java:69)
at org.apache.cassandra.thrift.Column.write(Column.java:579)
at org.apache.cassandra.thrift.CqlRow.write(CqlRow.java:439)
at org.apache.cassandra.thrift.CqlResult.write(CqlResult.java:602)
at 
org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.write(Cassandra.java:37895)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:34)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at 
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:199)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Created] (CASSANDRA-5990) Hinted Handoff: java.lang.ArithmeticException: / by zero

2013-09-09 Thread Karl Mueller (JIRA)
Karl Mueller created CASSANDRA-5990:
---

 Summary: Hinted Handoff: java.lang.ArithmeticException: / by zero
 Key: CASSANDRA-5990
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5990
 Project: Cassandra
  Issue Type: Bug
 Environment: cassandra 1.2.8
Oracle Java 1.7.0_25-b15
RHEL6
Reporter: Karl Mueller
Priority: Minor


This node was down for a few hours. When bringing it back up, I saw this error 
in the logs. I'm not sure if it's receiving or sending hinted hand-offs.

 INFO [HintedHandoff:1] 2013-09-09 14:41:04,020 HintedHandOffManager.java (line 
292) Started hinted handoff for host: 42bba02f-3088-4be1-8cb2-748a6f15e15d with 
IP: /10.93.12.14
ERROR [HintedHandoff:1] 2013-09-09 14:41:04,024 CassandraDaemon.java (line 192) 
Exception in thread Thread[HintedHandoff:1,1,main]
java.lang.ArithmeticException: / by zero
at 
org.apache.cassandra.db.HintedHandOffManager.calculatePageSize(HintedHandOffManager.java:441)
at 
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:299)
at 
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:278)
at 
org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:90)
at 
org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:497)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[1/5] git commit: cleanup paxos tracing

2013-09-09 Thread jbellis
Updated Branches:
  refs/heads/cassandra-2.0 ce9d3030b - 04cd81cdb
  refs/heads/trunk 3b8690db5 - 241f7a455


cleanup paxos tracing


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/41798b6f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/41798b6f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/41798b6f

Branch: refs/heads/cassandra-2.0
Commit: 41798b6f693fb14b0a1bf2fc212714eadc973719
Parents: ce9d303
Author: Jonathan Ellis jbel...@apache.org
Authored: Sun Sep 8 23:40:59 2013 +0100
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Sep 9 23:09:22 2013 +0100

--
 .../apache/cassandra/service/paxos/PaxosState.java  | 16 +---
 1 file changed, 9 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/41798b6f/src/java/org/apache/cassandra/service/paxos/PaxosState.java
--
diff --git a/src/java/org/apache/cassandra/service/paxos/PaxosState.java 
b/src/java/org/apache/cassandra/service/paxos/PaxosState.java
index ca69c16..aa27628 100644
--- a/src/java/org/apache/cassandra/service/paxos/PaxosState.java
+++ b/src/java/org/apache/cassandra/service/paxos/PaxosState.java
@@ -72,14 +72,14 @@ public class PaxosState
 PaxosState state = SystemKeyspace.loadPaxosState(toPrepare.key, 
toPrepare.update.metadata());
 if (toPrepare.isAfter(state.inProgressCommit))
 {
-Tracing.trace(promising ballot {}, toPrepare.ballot);
+Tracing.trace(Promising ballot {}, toPrepare.ballot);
 SystemKeyspace.savePaxosPromise(toPrepare);
 // return the pre-promise ballot so coordinator can pick the 
most recent in-progress value to resume
 return new PrepareResponse(true, state.inProgressCommit, 
state.mostRecentCommit);
 }
 else
 {
-Tracing.trace(promise rejected; {} is not sufficiently newer 
than {}, toPrepare, state.inProgressCommit);
+Tracing.trace(Promise rejected; {} is not sufficiently newer 
than {}, toPrepare, state.inProgressCommit);
 return new PrepareResponse(false, state.inProgressCommit, 
state.mostRecentCommit);
 }
 }
@@ -92,13 +92,15 @@ public class PaxosState
 PaxosState state = SystemKeyspace.loadPaxosState(proposal.key, 
proposal.update.metadata());
 if (proposal.hasBallot(state.inProgressCommit.ballot) || 
proposal.isAfter(state.inProgressCommit))
 {
-Tracing.trace(accepting proposal {}, proposal);
+Tracing.trace(Accepting proposal {}, proposal);
 SystemKeyspace.savePaxosProposal(proposal);
 return true;
 }
-
-logger.debug(accept requested for {} but inProgress is now {}, 
proposal, state.inProgressCommit);
-return false;
+else
+{
+Tracing.trace(Rejecting proposal for {} because inProgress is 
now {}, proposal, state.inProgressCommit);
+return false;
+}
 }
 }
 
@@ -109,7 +111,7 @@ public class PaxosState
 // Committing it is however always safe due to column timestamps, so 
always do it. However,
 // if our current in-progress ballot is strictly greater than the 
proposal one, we shouldn't
 // erase the in-progress update.
-Tracing.trace(committing proposal {}, proposal);
+Tracing.trace(Committing proposal {}, proposal);
 RowMutation rm = proposal.makeMutation();
 Keyspace.open(rm.getKeyspaceName()).apply(rm, true);
 



[5/5] git commit: Merge branch 'cassandra-2.0' into trunk

2013-09-09 Thread jbellis
Merge branch 'cassandra-2.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/241f7a45
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/241f7a45
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/241f7a45

Branch: refs/heads/trunk
Commit: 241f7a4559f4846d0ea31a3c968dfdd75cea8195
Parents: 3b8690d 04cd81c
Author: Jonathan Ellis jbel...@apache.org
Authored: Mon Sep 9 23:11:21 2013 +0100
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Sep 9 23:11:21 2013 +0100

--
 .../org/apache/cassandra/service/StorageProxy.java  | 13 +++--
 .../apache/cassandra/service/paxos/PaxosState.java  | 16 +---
 2 files changed, 20 insertions(+), 9 deletions(-)
--




[3/5] git commit: fix re-propose of earlier, incomplete paxos round to use new ballot patch by jbellis; reviewed by slebresne for CASSANDRA-5985

2013-09-09 Thread jbellis
fix re-propose of earlier, incomplete paxos round to use new ballot
patch by jbellis; reviewed by slebresne for CASSANDRA-5985


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/04cd81cd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/04cd81cd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/04cd81cd

Branch: refs/heads/trunk
Commit: 04cd81cdbb16e4f64b950325b581c6ba573b14a8
Parents: 41798b6
Author: Jonathan Ellis jbel...@apache.org
Authored: Sun Sep 8 23:41:24 2013 +0100
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Sep 9 23:10:57 2013 +0100

--
 .../org/apache/cassandra/service/StorageProxy.java | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/04cd81cd/src/java/org/apache/cassandra/service/StorageProxy.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java 
b/src/java/org/apache/cassandra/service/StorageProxy.java
index 9af310d..50dfd07 100644
--- a/src/java/org/apache/cassandra/service/StorageProxy.java
+++ b/src/java/org/apache/cassandra/service/StorageProxy.java
@@ -382,8 +382,17 @@ public class StorageProxy implements StorageProxyMBean
 if (!inProgress.update.isEmpty()  inProgress.isAfter(mostRecent))
 {
 Tracing.trace(Finishing incomplete paxos round {}, 
inProgress);
-if (proposePaxos(inProgress, liveEndpoints, 
requiredParticipants))
-commitPaxos(inProgress, ConsistencyLevel.QUORUM);
+Commit refreshedInProgress = 
Commit.newProposal(inProgress.key, ballot, inProgress.update);
+if (proposePaxos(refreshedInProgress, liveEndpoints, 
requiredParticipants))
+{
+commitPaxos(refreshedInProgress, ConsistencyLevel.QUORUM);
+}
+else
+{
+Tracing.trace(Some replicas have already promised a 
higher ballot than ours; aborting);
+// sleep a random amount to give the other proposer a 
chance to finish
+
Uninterruptibles.sleepUninterruptibly(FBUtilities.threadLocalRandom().nextInt(100),
 TimeUnit.MILLISECONDS);
+}
 continue;
 }
 



[4/5] git commit: fix re-propose of earlier, incomplete paxos round to use new ballot patch by jbellis; reviewed by slebresne for CASSANDRA-5985

2013-09-09 Thread jbellis
fix re-propose of earlier, incomplete paxos round to use new ballot
patch by jbellis; reviewed by slebresne for CASSANDRA-5985


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/04cd81cd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/04cd81cd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/04cd81cd

Branch: refs/heads/cassandra-2.0
Commit: 04cd81cdbb16e4f64b950325b581c6ba573b14a8
Parents: 41798b6
Author: Jonathan Ellis jbel...@apache.org
Authored: Sun Sep 8 23:41:24 2013 +0100
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Sep 9 23:10:57 2013 +0100

--
 .../org/apache/cassandra/service/StorageProxy.java | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/04cd81cd/src/java/org/apache/cassandra/service/StorageProxy.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java 
b/src/java/org/apache/cassandra/service/StorageProxy.java
index 9af310d..50dfd07 100644
--- a/src/java/org/apache/cassandra/service/StorageProxy.java
+++ b/src/java/org/apache/cassandra/service/StorageProxy.java
@@ -382,8 +382,17 @@ public class StorageProxy implements StorageProxyMBean
 if (!inProgress.update.isEmpty()  inProgress.isAfter(mostRecent))
 {
 Tracing.trace(Finishing incomplete paxos round {}, 
inProgress);
-if (proposePaxos(inProgress, liveEndpoints, 
requiredParticipants))
-commitPaxos(inProgress, ConsistencyLevel.QUORUM);
+Commit refreshedInProgress = 
Commit.newProposal(inProgress.key, ballot, inProgress.update);
+if (proposePaxos(refreshedInProgress, liveEndpoints, 
requiredParticipants))
+{
+commitPaxos(refreshedInProgress, ConsistencyLevel.QUORUM);
+}
+else
+{
+Tracing.trace(Some replicas have already promised a 
higher ballot than ours; aborting);
+// sleep a random amount to give the other proposer a 
chance to finish
+
Uninterruptibles.sleepUninterruptibly(FBUtilities.threadLocalRandom().nextInt(100),
 TimeUnit.MILLISECONDS);
+}
 continue;
 }
 



[2/5] git commit: cleanup paxos tracing

2013-09-09 Thread jbellis
cleanup paxos tracing


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/41798b6f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/41798b6f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/41798b6f

Branch: refs/heads/trunk
Commit: 41798b6f693fb14b0a1bf2fc212714eadc973719
Parents: ce9d303
Author: Jonathan Ellis jbel...@apache.org
Authored: Sun Sep 8 23:40:59 2013 +0100
Committer: Jonathan Ellis jbel...@apache.org
Committed: Mon Sep 9 23:09:22 2013 +0100

--
 .../apache/cassandra/service/paxos/PaxosState.java  | 16 +---
 1 file changed, 9 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/41798b6f/src/java/org/apache/cassandra/service/paxos/PaxosState.java
--
diff --git a/src/java/org/apache/cassandra/service/paxos/PaxosState.java 
b/src/java/org/apache/cassandra/service/paxos/PaxosState.java
index ca69c16..aa27628 100644
--- a/src/java/org/apache/cassandra/service/paxos/PaxosState.java
+++ b/src/java/org/apache/cassandra/service/paxos/PaxosState.java
@@ -72,14 +72,14 @@ public class PaxosState
 PaxosState state = SystemKeyspace.loadPaxosState(toPrepare.key, 
toPrepare.update.metadata());
 if (toPrepare.isAfter(state.inProgressCommit))
 {
-Tracing.trace(promising ballot {}, toPrepare.ballot);
+Tracing.trace(Promising ballot {}, toPrepare.ballot);
 SystemKeyspace.savePaxosPromise(toPrepare);
 // return the pre-promise ballot so coordinator can pick the 
most recent in-progress value to resume
 return new PrepareResponse(true, state.inProgressCommit, 
state.mostRecentCommit);
 }
 else
 {
-Tracing.trace(promise rejected; {} is not sufficiently newer 
than {}, toPrepare, state.inProgressCommit);
+Tracing.trace(Promise rejected; {} is not sufficiently newer 
than {}, toPrepare, state.inProgressCommit);
 return new PrepareResponse(false, state.inProgressCommit, 
state.mostRecentCommit);
 }
 }
@@ -92,13 +92,15 @@ public class PaxosState
 PaxosState state = SystemKeyspace.loadPaxosState(proposal.key, 
proposal.update.metadata());
 if (proposal.hasBallot(state.inProgressCommit.ballot) || 
proposal.isAfter(state.inProgressCommit))
 {
-Tracing.trace(accepting proposal {}, proposal);
+Tracing.trace(Accepting proposal {}, proposal);
 SystemKeyspace.savePaxosProposal(proposal);
 return true;
 }
-
-logger.debug(accept requested for {} but inProgress is now {}, 
proposal, state.inProgressCommit);
-return false;
+else
+{
+Tracing.trace(Rejecting proposal for {} because inProgress is 
now {}, proposal, state.inProgressCommit);
+return false;
+}
 }
 }
 
@@ -109,7 +111,7 @@ public class PaxosState
 // Committing it is however always safe due to column timestamps, so 
always do it. However,
 // if our current in-progress ballot is strictly greater than the 
proposal one, we shouldn't
 // erase the in-progress update.
-Tracing.trace(committing proposal {}, proposal);
+Tracing.trace(Committing proposal {}, proposal);
 RowMutation rm = proposal.makeMutation();
 Keyspace.open(rm.getKeyspaceName()).apply(rm, true);
 



[jira] [Created] (CASSANDRA-5988) Make hint TTL customizable

2013-09-09 Thread Oleg Kibirev (JIRA)
Oleg Kibirev created CASSANDRA-5988:
---

 Summary: Make hint TTL customizable
 Key: CASSANDRA-5988
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5988
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Oleg Kibirev


Currently time to live for stored hints is hardcoded to be gc_grace_seconds. 
This causes problems for applications using backdated deletes as a form of 
optimistic locking. Hints for updates made to the same data on which delete was 
attempted can persist for days, making it impossible to determine if delete 
succeeded by doing read(ALL) after a reasonable delay. We need a way to 
explicitly configure hint TTL, either through schema parameter or through a 
yaml file.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Resolved] (CASSANDRA-5939) Cache Providers calculate very different row sizes

2013-09-09 Thread Jonathan Ellis (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-5939.
---

Resolution: Won't Fix

 Cache Providers calculate very different row sizes
 --

 Key: CASSANDRA-5939
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5939
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 1.2.8
Reporter: Chris Burroughs
Assignee: Vijay

 Took the same production node and bounced it 4 times comparing version and 
 cache provider.  ConcurrentLinkedHashCacheProvider and 
 SerializingCacheProvider produce very different results resulting in an order 
 of magnitude difference in rows cached.  In all cases the row cache size was 
 2048 MB.  Hit rate is provided for color, but entries  size are the 
 important part.
 1.2.8 ConcurrentLinkedHashCacheProvider:
  * entries: 23,217
  * hit rate: 43%
  * size: 2,147,398,344
 1.2.8 about 20 minutes of SerializingCacheProvider:
  * entries: 221,709
  * hit rate: 68%
  * size: 18,417254
 1.2.5 ConcurrentLinkedHashCacheProvider:
  * entries: 25,967
  * hit rate: ~ 50%
  * size:  2,147,421,704
 1.2.5 about 20 minutes of SerializingCacheProvider:
  * entries: 228,457
  * hit rate: ~ 70%
  * size: 19,070,315
 A related(?) problem is that the ConcurrentLinkedHashCacheProvider sizes seem 
 to be highly variable.  Digging up the values for 5 different nodes in the 
 cluster using ConcurrentLinkedHashCacheProvider shows a wide variance in 
 number of entries:
  * 12k
  * 444k
  * 10k
  * 25k
  * 25k

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Created] (CASSANDRA-5991) COPY FROM failing on maps with Unmatched named substitution

2013-09-09 Thread Christopher J. Bottaro (JIRA)
Christopher J. Bottaro created CASSANDRA-5991:
-

 Summary: COPY FROM failing on maps with Unmatched named 
substitution
 Key: CASSANDRA-5991
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5991
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra 1.2.9 installed via Homebrew on OS X.
Reporter: Christopher J. Bottaro


I created a dump using COPY TO, but then the dump won't load with COPY FROM.

$ echo COPY applications TO 'dev_development.csv'; | cqlsh -k dev_development 
localhost
776 rows exported in 7.935 seconds.

$ echo COPY applications FROM 'dev_development.csv'; | cqlsh -k 
beta_development localhost
stdin:2:Unmatched named substitution: '27' not given for 'INSERT INTO 
beta_development.applications (version, id, answers) VALUES (\'2013-01-22 
13:52:11-0600\', 219, {1: \'{result:Izaiah}\', 2: \'{result:Maegan}\', 
3: \'{result:Bergstrom}\', 4: \'{result:1-617-729-8621 x05429}\', 5: 
\'{result:Mailing/Shipping Address}\', 6: \'{result:257 O\'Reilly 
Tunnel}\', 7: \'{result:null}\', 8: \'{result:Sauerberg}\', 9: 
\'{result:null}\', 10: \'{result:Washington}\', 11: 
\'{result:99043-3492}\', 12: \'{result:Peru}\', 13: 
\'{result:null}\', 14: \'{result:Male}\', 15: \'{result:6/5/1980}\', 
16: \'{result:American Indian or Alaska Native (not Hispanic or Latino)}\', 
17: \'{result:Refugee or Protected Person}\', 18: 
\'{result:5/10/2011}\', 19: \'{result:[Indiana University-South 
Bend,IVY Tech-South Bend]}\', 20: 
\'{result:[Band,Basketball,Hockey]}\', 21: 
\'{result:Robertsville}\', 22: \'{result:null}\', 23: 
\'{result:OH}\', 24: \'{result:83615-4544}\', 25: \'{result:Planning 
to file}\', 26: \'{result:[]}\', 27: \'{result:null}\', 28: 
\'{result:null}\', 29: \'{result:Nemo odio et repellendus dolores quis non 
ut. Ut sint labore nostrum. Quo odit at explicabo veniam necessitatibus error 
aliquid.br /Magni saepe nam et deleniti. Laudantium aut rerum accusantium 
voluptatem voluptas. Incidunt facere rerum veritatis aliquid deleniti ut. Qui 
non ea dicta officiis deserunt.br /Aut voluptas sapiente non omnis optio 
illum laboriosam. Voluptatem veritatis quibusdam aperiam. Vero ipsum facere 
accusamus aut cumque vel.br /Dolorem tenetur ut occaecati quia eos assumenda. 
Officiis doloribus similique id harum corporis quas. Quis occaecati sint aut 
fugiat ut nisi voluptas culpa. Ea illo maxime ut nobis beatae qui. Velit est 
quasi earum autem totam ab repudiandae fuga.br /Maiores sint et sed. Non 
fugit esse molestiae. Eum praesentium quae sapiente repellendus. Aut ipsam 
alias dolor ducimus eum debitis eaque id.br /Et provident ad ut ullam 
mollitia laboriosam doloremque ex. Quo quia id qui omnis quos numquam. Cum 
ratione ab est. Quia aspernatur et cum iusto voluptas saepe voluptatem. Veniam 
et doloribus quam.br /Exercitationem quis reiciendis explicabo assumenda. 
Amet sit consequatur enim aut et veniam quia. Inventore et iusto autem ea 
praesentium voluptates. Nostrum vel minima quasi recusandae. Vel enim veniam 
omnis.br /Consequuntur modi laborum est aut repellendus occaecati. Vero hic 
mollitia consequuntur. Quas quia ut et. Sed sunt unde dolore. Explicabo amet 
error sunt.br /Sunt cupiditate provident aut. Aliquam suscipit quaerat culpa 
consectetur ab. Et optio nesciunt impedit doloremque quis facere quae 
dolorum.}\', 32: \'{result:678}\', 33: \'{result:84}\', 34: 
\'{result:1.52}\', 35: \'{result:Undergraduate Studies}\', 36: 
\'{result:5296}\', 37: \'{result:6612}\', 38: \'{result:1311}\', 39: 
\'{result:27}\'})'
stdin:2:Aborting import at record #28 (line 29). Previously-inserted values 
still present.
28 rows imported in 0.325 seconds.

Please let me know what other information I can provide.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


git commit: don't declare throwing exceptions that aren't thrown

2013-09-09 Thread dbrosius
Updated Branches:
  refs/heads/cassandra-2.0 04cd81cdb - 52678a284


don't declare throwing exceptions that aren't thrown


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/52678a28
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/52678a28
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/52678a28

Branch: refs/heads/cassandra-2.0
Commit: 52678a284382d4da911c007fe8bc84b47057925f
Parents: 04cd81c
Author: Dave Brosius dbros...@apache.org
Authored: Mon Sep 9 18:43:21 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Mon Sep 9 18:43:21 2013 -0400

--
 src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/52678a28/src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java 
b/src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java
index a22b7ca..d0aafa4 100644
--- a/src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java
+++ b/src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java
@@ -17,7 +17,6 @@
  */
 package org.apache.cassandra.db.compaction;
 
-import java.io.IOException;
 import java.util.*;
 
 import org.apache.cassandra.db.*;
@@ -34,7 +33,7 @@ public class SSTableSplitter {
 this.task = new SplittingCompactionTask(cfs, sstable, sstableSizeInMB);
 }
 
-public void split() throws IOException
+public void split()
 {
 task.execute(new StatsCollector());
 }



[jira] [Commented] (CASSANDRA-5988) Make hint TTL customizable

2013-09-09 Thread sankalp kohli (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762414#comment-13762414
 ] 

sankalp kohli commented on CASSANDRA-5988:
--

We also want to make sure that hint TTL is less than gc grace period. 

 Make hint TTL customizable
 --

 Key: CASSANDRA-5988
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5988
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Oleg Kibirev

 Currently time to live for stored hints is hardcoded to be gc_grace_seconds. 
 This causes problems for applications using backdated deletes as a form of 
 optimistic locking. Hints for updates made to the same data on which delete 
 was attempted can persist for days, making it impossible to determine if 
 delete succeeded by doing read(ALL) after a reasonable delay. We need a way 
 to explicitly configure hint TTL, either through schema parameter or through 
 a yaml file.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[1/2] git commit: don't declare throwing exceptions that aren't thrown

2013-09-09 Thread dbrosius
Updated Branches:
  refs/heads/trunk 241f7a455 - b21a0dab5


don't declare throwing exceptions that aren't thrown


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/52678a28
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/52678a28
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/52678a28

Branch: refs/heads/trunk
Commit: 52678a284382d4da911c007fe8bc84b47057925f
Parents: 04cd81c
Author: Dave Brosius dbros...@apache.org
Authored: Mon Sep 9 18:43:21 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Mon Sep 9 18:43:21 2013 -0400

--
 src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/52678a28/src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java 
b/src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java
index a22b7ca..d0aafa4 100644
--- a/src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java
+++ b/src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java
@@ -17,7 +17,6 @@
  */
 package org.apache.cassandra.db.compaction;
 
-import java.io.IOException;
 import java.util.*;
 
 import org.apache.cassandra.db.*;
@@ -34,7 +33,7 @@ public class SSTableSplitter {
 this.task = new SplittingCompactionTask(cfs, sstable, sstableSizeInMB);
 }
 
-public void split() throws IOException
+public void split()
 {
 task.execute(new StatsCollector());
 }



[2/2] git commit: Merge branch 'cassandra-2.0' into trunk

2013-09-09 Thread dbrosius
Merge branch 'cassandra-2.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b21a0dab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b21a0dab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b21a0dab

Branch: refs/heads/trunk
Commit: b21a0dab5944c5399a0301938a99c8f190ff6a8d
Parents: 241f7a4 52678a2
Author: Dave Brosius dbros...@apache.org
Authored: Mon Sep 9 18:44:15 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Mon Sep 9 18:44:15 2013 -0400

--
 src/java/org/apache/cassandra/db/compaction/SSTableSplitter.java | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
--