[jira] [Commented] (CASSANDRA-5855) Scrub does not understand compound primary key created in CQL 3 beta

2013-08-08 Thread Christopher Smith (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733211#comment-13733211
 ] 

Christopher Smith commented on CASSANDRA-5855:
--

Hehe.

I figured the idea would be to make it the _same_ as the same column family 
created with the new tool. So for example, if the problem was the missing 
markers, while a bug might exist with the 1.1.9 scrub tool with 1.1.9 column 
family, it'd be a different story after upgrade. Since the bug shows up you 
upgrade the column family, but not if you always created it in 1.2, that would 
imply the markers still weren't there after the upgrade, which you'd think 
would be noticed by someone just checking to make sure upgrade actually 
upgraded, even if nobody realized this tickled an actual bug in scrub.

 Scrub does not understand compound primary key created in CQL 3 beta
 

 Key: CASSANDRA-5855
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5855
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: J.B. Langston
Assignee: Tyler Hobbs

 We have a customer who was using the beta version of CQL 3 in DSE 3.0 which 
 includes Cassandra 1.1.9 plus patches backported from later versions.  
 They've now upgraded to DSE 3.1, which includes Cassandra 1.2.6 plus patches.
 When restarting for the first time after running upgradesstables, they 
 noticed the following error in the log:
 {noformat}
 Thread[SSTableBatchOpen:2,5,main]
 java.lang.AssertionError
 at 
 org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
 at 
 org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
 at 
 org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {noformat}
 This error was also reported on CASSANDRA-5703.  The comments suggested it 
 was caused by an empty row key, so I had them run scrub on it.  When they 
 did, scrub reported the following warning almost 4 million times:
 {noformat}
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 52) Row at 530332255 is unreadable; skipping to next
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 57) Non-fatal error reading row (stacktrace follows)
 java.lang.RuntimeException: Error validating row 
 DecoratedKey(139154446688383793922009760478335751546, 
 735fc9da503b11e2844b123140ff209f)
  at 
 org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:243)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.init(PrecompactedRow.java:98)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166)
  at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:529)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.doScrub(CompactionManager.java:518)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:73)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$3.perform(CompactionManager.java:283)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:253)
  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)
 Caused by:

[jira] [Updated] (CASSANDRA-5789) Data not fully replicated with 2 nodes and replication factor 2

2013-08-08 Thread James Lee (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Lee updated CASSANDRA-5789:
-

Attachment: CassTestData.py

Attaching missing module.

 Data not fully replicated with 2 nodes and replication factor 2
 ---

 Key: CASSANDRA-5789
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5789
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.2, 1.2.6
 Environment: Official Datastax Cassandra 1.2.6, running on Linux RHEL 
 6.2.  I've seen the same behavior with Cassandra 1.2.2.
 Sun Java 1.7.0_10-b18 64-bit
 Java heap settings: -Xms8192M -Xmx8192M -Xmn2048M
Reporter: James Lee
Assignee: Brandon Williams
 Attachments: CassBugRepro.py, CassTestData.py


 I'm seeing a problem with a 2-node Cassandra test deployment, where it seems 
 that data isn't being replicated among the nodes as I would expect.
 The setup and test is as follows:
 - Two Cassandra nodes in the cluster (they each have themselves and the other 
 node as seeds in cassandra.yaml).
 - Create 40 keyspaces, each with simple replication strategy and 
 replication factor 2.
 - Populate 125,000 rows into each keyspace, using a pycassa client with a 
 connection pool pointed at both nodes.  These are populated with writes using 
 consistency level of 1.
 - Wait until nodetool on each node reports that there are no hinted handoffs 
 outstanding (see output below).
 - Do random reads of the rows in the keyspaces, again using a pycassa client 
 with a connection pool pointed at both nodes.  These are read using 
 consistency level 1.
 I'm finding that the vast majority of reads are successful, but a small 
 proportion (~0.1%) are returned as Not Found.  If I manually try to look up 
 those keys using cassandra-cli, I see that they are returned when querying 
 one of the nodes, but not when querying the other.  So it seems like some of 
 the rows have simply not been replicated, even though the write for these 
 rows was reported to the client as successful.
 If I reduce the rate at which the test tool initially writes data into the 
 database then I don't see any failed reads, so this seems like a load-related 
 issue.  My understanding is that if all writes were successful and there are 
 no pending hinted handoffs, then the data should be fully-replicated and 
 reads should return it (even with read and write consistency of 1).
 Here's the output from notetool on the two nodes:
 comet-mvs01:/dsc-cassandra-1.2.6# ./bin/nodetool tpstats
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 0 0  2 0
  0
 RequestResponseStage  0 0 878494 0
  0
 MutationStage 0 02869107 0
  0
 ReadRepairStage   0 0  0 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0   2208 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0994 0
  0
 MemtablePostFlusher   0 0   4399 0
  0
 FlushWriter   0 0   2264 0
556
 MiscStage 0 0  0 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0153 0
  0
 HintedHandoff 0 0  2 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ 0
 MUTATION 87655
 _TRACE   0
 REQUEST_RESPONSE 0
 comet-mvs02:/dsc-cassandra-1.2.6# ./bin/nodetool tpstats
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 0 0868 0
  0
 RequestResponseStage  0 03919665 0
  0
 MutationStage 0 08177325 0
  0
 ReadRepairStage   0 0113 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage

[jira] [Commented] (CASSANDRA-5789) Data not fully replicated with 2 nodes and replication factor 2

2013-08-08 Thread James Lee (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733247#comment-13733247
 ] 

James Lee commented on CASSANDRA-5789:
--

Ah, sorry about forgetting to attach the CassTestData module - I've added it 
now.

As regards the errors, I think the test does specifically catch the right error 
- see the except pycassa.NotFoundException: branch at line 238.  Certainly 
every time I've seen that error, if I manually query the Cassandra database 
using cassandra-cli then I do indeed see that row present on one node and not 
the other.

In the case of the stack you've posted above, it looks like the pycassa client 
is failing to connect to the cassandra nodes at all - I hadn't added any 
exception handling for that case as I hadn't hit it at all in my testing.  
Presumably when running the script you don't get as far as seeing 
NotFoundException for DN printed out?

 Data not fully replicated with 2 nodes and replication factor 2
 ---

 Key: CASSANDRA-5789
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5789
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.2, 1.2.6
 Environment: Official Datastax Cassandra 1.2.6, running on Linux RHEL 
 6.2.  I've seen the same behavior with Cassandra 1.2.2.
 Sun Java 1.7.0_10-b18 64-bit
 Java heap settings: -Xms8192M -Xmx8192M -Xmn2048M
Reporter: James Lee
Assignee: Brandon Williams
 Attachments: CassBugRepro.py, CassTestData.py


 I'm seeing a problem with a 2-node Cassandra test deployment, where it seems 
 that data isn't being replicated among the nodes as I would expect.
 The setup and test is as follows:
 - Two Cassandra nodes in the cluster (they each have themselves and the other 
 node as seeds in cassandra.yaml).
 - Create 40 keyspaces, each with simple replication strategy and 
 replication factor 2.
 - Populate 125,000 rows into each keyspace, using a pycassa client with a 
 connection pool pointed at both nodes.  These are populated with writes using 
 consistency level of 1.
 - Wait until nodetool on each node reports that there are no hinted handoffs 
 outstanding (see output below).
 - Do random reads of the rows in the keyspaces, again using a pycassa client 
 with a connection pool pointed at both nodes.  These are read using 
 consistency level 1.
 I'm finding that the vast majority of reads are successful, but a small 
 proportion (~0.1%) are returned as Not Found.  If I manually try to look up 
 those keys using cassandra-cli, I see that they are returned when querying 
 one of the nodes, but not when querying the other.  So it seems like some of 
 the rows have simply not been replicated, even though the write for these 
 rows was reported to the client as successful.
 If I reduce the rate at which the test tool initially writes data into the 
 database then I don't see any failed reads, so this seems like a load-related 
 issue.  My understanding is that if all writes were successful and there are 
 no pending hinted handoffs, then the data should be fully-replicated and 
 reads should return it (even with read and write consistency of 1).
 Here's the output from notetool on the two nodes:
 comet-mvs01:/dsc-cassandra-1.2.6# ./bin/nodetool tpstats
 Pool NameActive   Pending  Completed   Blocked  All 
 time blocked
 ReadStage 0 0  2 0
  0
 RequestResponseStage  0 0 878494 0
  0
 MutationStage 0 02869107 0
  0
 ReadRepairStage   0 0  0 0
  0
 ReplicateOnWriteStage 0 0  0 0
  0
 GossipStage   0 0   2208 0
  0
 AntiEntropyStage  0 0  0 0
  0
 MigrationStage0 0994 0
  0
 MemtablePostFlusher   0 0   4399 0
  0
 FlushWriter   0 0   2264 0
556
 MiscStage 0 0  0 0
  0
 commitlog_archiver0 0  0 0
  0
 InternalResponseStage 0 0153 0
  0
 HintedHandoff 0 0  2 0
  0
 Message type   Dropped
 RANGE_SLICE  0
 READ_REPAIR  0
 BINARY   0
 READ

[jira] [Commented] (CASSANDRA-5855) Scrub does not understand compound primary key created in CQL 3 beta

2013-08-08 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733450#comment-13733450
 ] 

Jonathan Ellis commented on CASSANDRA-5855:
---

The problem is that it can't read the existing data file.  If it could, it 
would indeed write out a 1.2-format file.

 Scrub does not understand compound primary key created in CQL 3 beta
 

 Key: CASSANDRA-5855
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5855
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: J.B. Langston
Assignee: Tyler Hobbs

 We have a customer who was using the beta version of CQL 3 in DSE 3.0 which 
 includes Cassandra 1.1.9 plus patches backported from later versions.  
 They've now upgraded to DSE 3.1, which includes Cassandra 1.2.6 plus patches.
 When restarting for the first time after running upgradesstables, they 
 noticed the following error in the log:
 {noformat}
 Thread[SSTableBatchOpen:2,5,main]
 java.lang.AssertionError
 at 
 org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
 at 
 org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
 at 
 org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {noformat}
 This error was also reported on CASSANDRA-5703.  The comments suggested it 
 was caused by an empty row key, so I had them run scrub on it.  When they 
 did, scrub reported the following warning almost 4 million times:
 {noformat}
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 52) Row at 530332255 is unreadable; skipping to next
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 57) Non-fatal error reading row (stacktrace follows)
 java.lang.RuntimeException: Error validating row 
 DecoratedKey(139154446688383793922009760478335751546, 
 735fc9da503b11e2844b123140ff209f)
  at 
 org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:243)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.init(PrecompactedRow.java:98)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166)
  at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:529)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.doScrub(CompactionManager.java:518)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:73)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$3.perform(CompactionManager.java:283)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:253)
  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)
 Caused by: org.apache.cassandra.db.marshal.MarshalException: String didn't 
 validate.
  at org.apache.cassandra.db.marshal.UTF8Type.validate(UTF8Type.java:66)
  at org.apache.cassandra.db.Column.validateFields(Column.java:292)
  at 
 org.apache.cassandra.db.ColumnFamily.validateColumnFields(ColumnFamily.java:382)
  at 
 org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:239)
  ... 15 more
 {noformat}
 The customer did some testing and they've determined that the issue only

[jira] [Commented] (CASSANDRA-4131) Integrate Hive support to be in core cassandra

2013-08-08 Thread Cyril Scetbon (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733490#comment-13733490
 ] 

Cyril Scetbon commented on CASSANDRA-4131:
--

Did you have more time to test it ? Is there any documentation that describes 
how to use it ?

 Integrate Hive support to be in core cassandra
 --

 Key: CASSANDRA-4131
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4131
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jeremy Hanna
Assignee: Edward Capriolo
  Labels: hadoop, hive

 The standalone hive support (at https://github.com/riptano/hive) would be 
 great to have in-tree so that people don't have to go out to github to 
 download it and wonder if it's a left-for-dead external shim.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-4131) Integrate Hive support to be in core cassandra

2013-08-08 Thread Cyril Scetbon (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733490#comment-13733490
 ] 

Cyril Scetbon edited comment on CASSANDRA-4131 at 8/8/13 2:05 PM:
--

Did you have more time to test it ? I'll give it a try asap

  was (Author: cscetbon):
Did you have more time to test it ? Is there any documentation that 
describes how to use it ?
  
 Integrate Hive support to be in core cassandra
 --

 Key: CASSANDRA-4131
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4131
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jeremy Hanna
Assignee: Edward Capriolo
  Labels: hadoop, hive

 The standalone hive support (at https://github.com/riptano/hive) would be 
 great to have in-tree so that people don't have to go out to github to 
 download it and wonder if it's a left-for-dead external shim.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-4131) Integrate Hive support to be in core cassandra

2013-08-08 Thread Cyril Scetbon (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733490#comment-13733490
 ] 

Cyril Scetbon edited comment on CASSANDRA-4131 at 8/8/13 2:05 PM:
--

Did you have more time to test it ? I'll give it a try asaic

  was (Author: cscetbon):
Did you have more time to test it ? I'll give it a try asap
  
 Integrate Hive support to be in core cassandra
 --

 Key: CASSANDRA-4131
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4131
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jeremy Hanna
Assignee: Edward Capriolo
  Labels: hadoop, hive

 The standalone hive support (at https://github.com/riptano/hive) would be 
 great to have in-tree so that people don't have to go out to github to 
 download it and wonder if it's a left-for-dead external shim.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5857) NumberFormatException during decommission

2013-08-08 Thread Ryan McGuire (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733546#comment-13733546
 ] 

Ryan McGuire commented on CASSANDRA-5857:
-

OK, so this is upgrade related? How do I reproduce this? I'm wondering what 
scenarios we need to add to the upgrade_through_versions dtest to avoid this 
error again.

 NumberFormatException during decommission
 -

 Key: CASSANDRA-5857
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5857
 Project: Cassandra
  Issue Type: Bug
Reporter: Brandon Williams
Assignee: Brandon Williams

 We half-fixed this in CASSANDRA-5696, but unfortunately StorageService is 
 still looking at the token to get the expiretime in some cases.
 {noformat}
 java.lang.NumberFormatException: For input string: 
 113427455640312821154458202477256070484
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Long.parseLong(Long.java:444)
 at java.lang.Long.parseLong(Long.java:483)
 at 
 org.apache.cassandra.service.StorageService.extractExpireTime(StorageService.java:1660)
 at 
 org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1515)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:1234)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:953)
 at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:944)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:902)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:722)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5861) Test LEFT state serialization in upgrade_through_versions dtest.

2013-08-08 Thread Ryan McGuire (JIRA) 
Ryan McGuire created CASSANDRA-5861:
---

 Summary: Test LEFT state serialization in upgrade_through_versions 
dtest.
 Key: CASSANDRA-5861
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5861
 Project: Cassandra
  Issue Type: Test
Reporter: Ryan McGuire
Assignee: Ryan McGuire


Make sure things like CASSANDRA-5857 don't crop up again.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-5857) NumberFormatException during decommission

2013-08-08 Thread Ryan McGuire (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733546#comment-13733546
 ] 

Ryan McGuire edited comment on CASSANDRA-5857 at 8/8/13 2:37 PM:
-

-OK, so this is upgrade related? How do I reproduce this? I'm wondering what 
scenarios we need to add to the upgrade_through_versions dtest to avoid this 
error again.-

Nevermind, [~jjordan] set me straight that this is purely decomission.

  was (Author: enigmacurry):
OK, so this is upgrade related? How do I reproduce this? I'm wondering what 
scenarios we need to add to the upgrade_through_versions dtest to avoid this 
error again.
  
 NumberFormatException during decommission
 -

 Key: CASSANDRA-5857
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5857
 Project: Cassandra
  Issue Type: Bug
Reporter: Brandon Williams
Assignee: Brandon Williams

 We half-fixed this in CASSANDRA-5696, but unfortunately StorageService is 
 still looking at the token to get the expiretime in some cases.
 {noformat}
 java.lang.NumberFormatException: For input string: 
 113427455640312821154458202477256070484
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Long.parseLong(Long.java:444)
 at java.lang.Long.parseLong(Long.java:483)
 at 
 org.apache.cassandra.service.StorageService.extractExpireTime(StorageService.java:1660)
 at 
 org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1515)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:1234)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:953)
 at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:944)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:902)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:722)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5861) Test decomission LEFT state serialization

2013-08-08 Thread Ryan McGuire (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire updated CASSANDRA-5861:


Summary: Test decomission LEFT state serialization  (was: Test LEFT state 
serialization in upgrade_through_versions dtest.)

 Test decomission LEFT state serialization
 -

 Key: CASSANDRA-5861
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5861
 Project: Cassandra
  Issue Type: Test
Reporter: Ryan McGuire
Assignee: Ryan McGuire

 Make sure things like CASSANDRA-5857 don't crop up again.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread Jonathan Ellis (JIRA) 
Jonathan Ellis created CASSANDRA-5862:
-

 Summary: Switch to adler checksum for sstables
 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1


Adler is significantly faster than CRC32: 
http://java-performance.info/java-crc32-and-adler32/

(Adler is weaker for short inputs, so we should leave the commitlog alone, as 
it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data

2013-08-08 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733587#comment-13733587
 ] 

Jonathan Ellis commented on CASSANDRA-5661:
---

[~xedin], have you had a chance to get back to this?

 Discard pooled readers for cold data
 

 Key: CASSANDRA-5661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 2.0.1

 Attachments: CASSANDRA-5661-global-multiway-cache.patch, 
 CASSANDRA-5661.patch, DominatorTree.png, Histogram.png


 Reader pooling was introduced in CASSANDRA-4942 but pooled 
 RandomAccessReaders are never cleaned up until the SSTableReader is closed.  
 So memory use is the worst case simultaneous RAR we had open for this file, 
 forever.
 We should introduce a global limit on how much memory to use for RAR, and 
 evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5519) Reduce index summary memory use for cold sstables

2013-08-08 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733593#comment-13733593
 ] 

Jonathan Ellis commented on CASSANDRA-5519:
---

Sounds reasonable.

 Reduce index summary memory use for cold sstables
 -

 Key: CASSANDRA-5519
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5519
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Priority: Minor
 Fix For: 2.0.1




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-5843) SStableReader should enforce through code that no one creates multiple objects of it for same sstable.

2013-08-08 Thread Jonathan Ellis (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-5843.
---

Resolution: Won't Fix

As noted, committing this is impractical.

 SStableReader should enforce through code that no one creates multiple 
 objects of it for same sstable. 
 ---

 Key: CASSANDRA-5843
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5843
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.1.12
Reporter: sankalp kohli
Priority: Minor
 Attachments: 5843.txt


 CUrrently this is not enforced in the code. 
 In the View, we hold these readers in the set but the SStableReader equals 
 method is not implemented which makes the set meaningless. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4573) HSHA doesn't handle large messages gracefully

2013-08-08 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733613#comment-13733613
 ] 

Jonathan Ellis commented on CASSANDRA-4573:
---

[~xedin], can you add the license file for thrift-server to lib/licences?

 HSHA doesn't handle large messages gracefully
 -

 Key: CASSANDRA-4573
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4573
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Pavel Yaskevich
 Fix For: 2.0

 Attachments: CASSANDRA-4573.patch, 
 disruptor-thrift-server-0.2.2-SNAPSHOT.jar, repro.py


 HSHA doesn't seem to enforce any kind of max message length, and when 
 messages are too large, it doesn't fail gracefully.
 With debug logs enabled, you'll see this:
 {{DEBUG 13:13:31,805 Unexpected state 16}}
 Which seems to mean that there's a SelectionKey that's valid, but isn't ready 
 for reading, writing, or accepting.
 Client-side, you'll get this thrift error (while trying to read a frame as 
 part of {{recv_batch_mutate}}):
 {{TTransportException: TSocket read 0 bytes}}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5855) Scrub does not understand compound primary key created in CQL 3 beta

2013-08-08 Thread Tyler Hobbs (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733643#comment-13733643
 ] 

Tyler Hobbs commented on CASSANDRA-5855:


bq. Tyler, that makes sense... does that mean if you kept the composite key 
fields all the same type, it would work properly?

If all of the column values were UTF8Type, it would work properly, but that's 
definitely not a feasible workaround.  I'm looking into a proper fix, which 
would allow scrub to lookup the correct column validators.

 Scrub does not understand compound primary key created in CQL 3 beta
 

 Key: CASSANDRA-5855
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5855
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: J.B. Langston
Assignee: Tyler Hobbs

 We have a customer who was using the beta version of CQL 3 in DSE 3.0 which 
 includes Cassandra 1.1.9 plus patches backported from later versions.  
 They've now upgraded to DSE 3.1, which includes Cassandra 1.2.6 plus patches.
 When restarting for the first time after running upgradesstables, they 
 noticed the following error in the log:
 {noformat}
 Thread[SSTableBatchOpen:2,5,main]
 java.lang.AssertionError
 at 
 org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
 at 
 org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
 at 
 org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {noformat}
 This error was also reported on CASSANDRA-5703.  The comments suggested it 
 was caused by an empty row key, so I had them run scrub on it.  When they 
 did, scrub reported the following warning almost 4 million times:
 {noformat}
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 52) Row at 530332255 is unreadable; skipping to next
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 57) Non-fatal error reading row (stacktrace follows)
 java.lang.RuntimeException: Error validating row 
 DecoratedKey(139154446688383793922009760478335751546, 
 735fc9da503b11e2844b123140ff209f)
  at 
 org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:243)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.init(PrecompactedRow.java:98)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166)
  at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:529)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.doScrub(CompactionManager.java:518)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:73)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$3.perform(CompactionManager.java:283)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:253)
  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)
 Caused by: org.apache.cassandra.db.marshal.MarshalException: String didn't 
 validate.
  at org.apache.cassandra.db.marshal.UTF8Type.validate(UTF8Type.java:66)
  at org.apache.cassandra.db.Column.validateFields(Column.java:292)
  at 
 org.apache.cassandra.db.ColumnFamily.validateColumnFields(ColumnFamily.java:382)
  at

[14/16] git commit: comment

2013-08-08 Thread jbellis 
comment


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c5261312
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c5261312
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c5261312

Branch: refs/heads/cassandra-2.0
Commit: c5261312459791ba6356b73cd61abf778879ed0e
Parents: f2d0433
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:18:41 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:18:41 2013 -0500

--
 src/java/org/apache/cassandra/utils/BiMultiValMap.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c5261312/src/java/org/apache/cassandra/utils/BiMultiValMap.java
--
diff --git a/src/java/org/apache/cassandra/utils/BiMultiValMap.java 
b/src/java/org/apache/cassandra/utils/BiMultiValMap.java
index 3712d25..d827d3f 100644
--- a/src/java/org/apache/cassandra/utils/BiMultiValMap.java
+++ b/src/java/org/apache/cassandra/utils/BiMultiValMap.java
@@ -29,7 +29,7 @@ import com.google.common.collect.Multimaps;
 /**
  *
  * A variant of BiMap which does not enforce uniqueness of values. This means 
the inverse
- * is a Multimap.
+ * is a Multimap.  (But the forward view is not a multimap; keys may only 
each have one value.)
  *
  * @param K
  * @param V

[02/16] git commit: replace volatile regionCount w/ AtomicInteger

2013-08-08 Thread jbellis 
replace volatile regionCount w/ AtomicInteger


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f022354c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f022354c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f022354c

Branch: refs/heads/trunk
Commit: f022354cb44a15d034b3a28dec5d38cc97084faf
Parents: 23becce
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 10:57:48 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 10:57:48 2013 -0500

--
 src/java/org/apache/cassandra/utils/SlabAllocator.java | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f022354c/src/java/org/apache/cassandra/utils/SlabAllocator.java
--
diff --git a/src/java/org/apache/cassandra/utils/SlabAllocator.java 
b/src/java/org/apache/cassandra/utils/SlabAllocator.java
index 6ff66f8..edd6f86 100644
--- a/src/java/org/apache/cassandra/utils/SlabAllocator.java
+++ b/src/java/org/apache/cassandra/utils/SlabAllocator.java
@@ -48,7 +48,7 @@ public class SlabAllocator extends Allocator
 private final static int MAX_CLONED_SIZE = 128 * 1024; // bigger than this 
don't go in the region
 
 private final AtomicReferenceRegion currentRegion = new 
AtomicReferenceRegion();
-private volatile int regionCount = 0;
+private final AtomicInteger regionCount = new AtomicInteger(0);
 private AtomicLong unslabbed = new AtomicLong(0);
 
 public ByteBuffer allocate(int size)
@@ -99,7 +99,7 @@ public class SlabAllocator extends Allocator
 {
 // we won race - now we need to actually do the expensive 
allocation step
 region.init();
-regionCount++;
+regionCount.incrementAndGet();
 logger.trace({} regions now allocated in {}, regionCount, 
this);
 return region;
 }
@@ -113,7 +113,7 @@ public class SlabAllocator extends Allocator
  */
 public long getMinimumSize()
 {
-return unslabbed.get() + (regionCount - 1) * (long)REGION_SIZE;
+return unslabbed.get() + (regionCount.get() - 1) * (long)REGION_SIZE;
 }
 
 /**
@@ -121,7 +121,7 @@ public class SlabAllocator extends Allocator
  */
 public long getMaximumSize()
 {
-return unslabbed.get() + regionCount * (long)REGION_SIZE;
+return unslabbed.get() + regionCount.get() * (long)REGION_SIZE;
 }
 
 /**

[01/16] git commit: replace volatile regionCount w/ AtomicInteger

2013-08-08 Thread jbellis 
Updated Branches:
  refs/heads/cassandra-1.2 7d45d2bde - 23d7bd857
  refs/heads/cassandra-2.0 23becceb7 - c52613124
  refs/heads/cassandra-2.0.0 88fea9fe7 - f30629b83
  refs/heads/trunk 66182082c - c7782f2de


replace volatile regionCount w/ AtomicInteger


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f022354c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f022354c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f022354c

Branch: refs/heads/cassandra-2.0
Commit: f022354cb44a15d034b3a28dec5d38cc97084faf
Parents: 23becce
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 10:57:48 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 10:57:48 2013 -0500

--
 src/java/org/apache/cassandra/utils/SlabAllocator.java | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f022354c/src/java/org/apache/cassandra/utils/SlabAllocator.java
--
diff --git a/src/java/org/apache/cassandra/utils/SlabAllocator.java 
b/src/java/org/apache/cassandra/utils/SlabAllocator.java
index 6ff66f8..edd6f86 100644
--- a/src/java/org/apache/cassandra/utils/SlabAllocator.java
+++ b/src/java/org/apache/cassandra/utils/SlabAllocator.java
@@ -48,7 +48,7 @@ public class SlabAllocator extends Allocator
 private final static int MAX_CLONED_SIZE = 128 * 1024; // bigger than this 
don't go in the region
 
 private final AtomicReferenceRegion currentRegion = new 
AtomicReferenceRegion();
-private volatile int regionCount = 0;
+private final AtomicInteger regionCount = new AtomicInteger(0);
 private AtomicLong unslabbed = new AtomicLong(0);
 
 public ByteBuffer allocate(int size)
@@ -99,7 +99,7 @@ public class SlabAllocator extends Allocator
 {
 // we won race - now we need to actually do the expensive 
allocation step
 region.init();
-regionCount++;
+regionCount.incrementAndGet();
 logger.trace({} regions now allocated in {}, regionCount, 
this);
 return region;
 }
@@ -113,7 +113,7 @@ public class SlabAllocator extends Allocator
  */
 public long getMinimumSize()
 {
-return unslabbed.get() + (regionCount - 1) * (long)REGION_SIZE;
+return unslabbed.get() + (regionCount.get() - 1) * (long)REGION_SIZE;
 }
 
 /**
@@ -121,7 +121,7 @@ public class SlabAllocator extends Allocator
  */
 public long getMaximumSize()
 {
-return unslabbed.get() + regionCount * (long)REGION_SIZE;
+return unslabbed.get() + regionCount.get() * (long)REGION_SIZE;
 }
 
 /**

[06/16] git commit: fix potential overflow in extimatedKeys

2013-08-08 Thread jbellis 
fix potential overflow in extimatedKeys


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/23d7bd85
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/23d7bd85
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/23d7bd85

Branch: refs/heads/cassandra-1.2
Commit: 23d7bd85758762dbc2b717af56c4f590702a8b30
Parents: 7d45d2b
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:15:27 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:15:27 2013 -0500

--
 src/java/org/apache/cassandra/io/sstable/SSTableReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/23d7bd85/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index e9a03c8..412e4c1 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -597,7 +597,7 @@ public class SSTableReader extends SSTable
  */
 public long estimatedKeys()
 {
-return indexSummary.size() * DatabaseDescriptor.getIndexInterval();
+return ((long) indexSummary.size()) * 
DatabaseDescriptor.getIndexInterval();
 }
 
 /**

[04/16] git commit: r/m unnecessary null check

2013-08-08 Thread jbellis 
r/m unnecessary null check


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e533d5af
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e533d5af
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e533d5af

Branch: refs/heads/trunk
Commit: e533d5afbc0cc1c321060dc40224422d23578b4f
Parents: f022354
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:11:38 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:11:38 2013 -0500

--
 .../db/compaction/LeveledCompactionStrategy.java   | 13 +
 1 file changed, 5 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e533d5af/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
index 292220d..b3e01ed 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
@@ -54,16 +54,13 @@ public class LeveledCompactionStrategy extends 
AbstractCompactionStrategy implem
 super(cfs, options);
 int configuredMaxSSTableSize = 160;
 SizeTieredCompactionStrategyOptions localOptions = new 
SizeTieredCompactionStrategyOptions(options);
-if (options != null)
+if (options.containsKey(SSTABLE_SIZE_OPTION))
 {
-if (options.containsKey(SSTABLE_SIZE_OPTION))
+configuredMaxSSTableSize = 
Integer.parseInt(options.get(SSTABLE_SIZE_OPTION));
+if (configuredMaxSSTableSize = 1000)
 {
-configuredMaxSSTableSize = 
Integer.parseInt(options.get(SSTABLE_SIZE_OPTION));
-if (configuredMaxSSTableSize = 1000)
-{
-// Yes, people have done this
-logger.warn(Max sstable size of {}MB is configured; 
having a unit of compaction this large is probably a bad idea, 
configuredMaxSSTableSize);
-}
+// Yes, people have done this
+logger.warn(Max sstable size of {}MB is configured; having a 
unit of compaction this large is probably a bad idea, 
configuredMaxSSTableSize);
 }
 }
 maxSSTableSizeInMB = configuredMaxSSTableSize;

[03/16] git commit: r/m unnecessary null check

2013-08-08 Thread jbellis 
r/m unnecessary null check


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e533d5af
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e533d5af
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e533d5af

Branch: refs/heads/cassandra-2.0
Commit: e533d5afbc0cc1c321060dc40224422d23578b4f
Parents: f022354
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:11:38 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:11:38 2013 -0500

--
 .../db/compaction/LeveledCompactionStrategy.java   | 13 +
 1 file changed, 5 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e533d5af/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
index 292220d..b3e01ed 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
@@ -54,16 +54,13 @@ public class LeveledCompactionStrategy extends 
AbstractCompactionStrategy implem
 super(cfs, options);
 int configuredMaxSSTableSize = 160;
 SizeTieredCompactionStrategyOptions localOptions = new 
SizeTieredCompactionStrategyOptions(options);
-if (options != null)
+if (options.containsKey(SSTABLE_SIZE_OPTION))
 {
-if (options.containsKey(SSTABLE_SIZE_OPTION))
+configuredMaxSSTableSize = 
Integer.parseInt(options.get(SSTABLE_SIZE_OPTION));
+if (configuredMaxSSTableSize = 1000)
 {
-configuredMaxSSTableSize = 
Integer.parseInt(options.get(SSTABLE_SIZE_OPTION));
-if (configuredMaxSSTableSize = 1000)
-{
-// Yes, people have done this
-logger.warn(Max sstable size of {}MB is configured; 
having a unit of compaction this large is probably a bad idea, 
configuredMaxSSTableSize);
-}
+// Yes, people have done this
+logger.warn(Max sstable size of {}MB is configured; having a 
unit of compaction this large is probably a bad idea, 
configuredMaxSSTableSize);
 }
 }
 maxSSTableSizeInMB = configuredMaxSSTableSize;

[09/16] git commit: merge from 1.2

2013-08-08 Thread jbellis 
merge from 1.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f30629b8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f30629b8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f30629b8

Branch: refs/heads/trunk
Commit: f30629b83fb14b6a93e38d440f79a16200cfcef2
Parents: 88fea9f 23d7bd8
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:15:54 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:15:54 2013 -0500

--
 src/java/org/apache/cassandra/io/sstable/SSTableReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f30629b8/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index e30e417,412e4c1..31a40cb
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@@ -594,7 -597,7 +594,7 @@@ public class SSTableReader extends SSTa
   */
  public long estimatedKeys()
  {
- return indexSummary.size() * indexSummary.getIndexInterval();
 -return ((long) indexSummary.size()) * 
DatabaseDescriptor.getIndexInterval();
++return ((long) indexSummary.size()) * indexSummary.getIndexInterval();
  }
  
  /**

[16/16] git commit: Merge branch 'cassandra-2.0' into trunk

2013-08-08 Thread jbellis 
Merge branch 'cassandra-2.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c7782f2d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c7782f2d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c7782f2d

Branch: refs/heads/trunk
Commit: c7782f2de044ce18db3441ea27e90d60df531e3f
Parents: 6618208 c526131
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:33:47 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:33:47 2013 -0500

--
 .../db/compaction/LeveledCompactionStrategy.java   | 13 +
 .../org/apache/cassandra/io/sstable/SSTableReader.java |  2 +-
 src/java/org/apache/cassandra/utils/BiMultiValMap.java |  2 +-
 src/java/org/apache/cassandra/utils/SlabAllocator.java |  8 
 4 files changed, 11 insertions(+), 14 deletions(-)
--

[05/16] git commit: fix potential overflow in extimatedKeys

2013-08-08 Thread jbellis 
fix potential overflow in extimatedKeys


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/23d7bd85
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/23d7bd85
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/23d7bd85

Branch: refs/heads/trunk
Commit: 23d7bd85758762dbc2b717af56c4f590702a8b30
Parents: 7d45d2b
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:15:27 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:15:27 2013 -0500

--
 src/java/org/apache/cassandra/io/sstable/SSTableReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/23d7bd85/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index e9a03c8..412e4c1 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -597,7 +597,7 @@ public class SSTableReader extends SSTable
  */
 public long estimatedKeys()
 {
-return indexSummary.size() * DatabaseDescriptor.getIndexInterval();
+return ((long) indexSummary.size()) * 
DatabaseDescriptor.getIndexInterval();
 }
 
 /**

[07/16] git commit: fix potential overflow in extimatedKeys

2013-08-08 Thread jbellis 
fix potential overflow in extimatedKeys


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/23d7bd85
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/23d7bd85
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/23d7bd85

Branch: refs/heads/cassandra-2.0
Commit: 23d7bd85758762dbc2b717af56c4f590702a8b30
Parents: 7d45d2b
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:15:27 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:15:27 2013 -0500

--
 src/java/org/apache/cassandra/io/sstable/SSTableReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/23d7bd85/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index e9a03c8..412e4c1 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -597,7 +597,7 @@ public class SSTableReader extends SSTable
  */
 public long estimatedKeys()
 {
-return indexSummary.size() * DatabaseDescriptor.getIndexInterval();
+return ((long) indexSummary.size()) * 
DatabaseDescriptor.getIndexInterval();
 }
 
 /**

[08/16] git commit: fix potential overflow in extimatedKeys

2013-08-08 Thread jbellis 
fix potential overflow in extimatedKeys


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/23d7bd85
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/23d7bd85
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/23d7bd85

Branch: refs/heads/cassandra-2.0.0
Commit: 23d7bd85758762dbc2b717af56c4f590702a8b30
Parents: 7d45d2b
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:15:27 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:15:27 2013 -0500

--
 src/java/org/apache/cassandra/io/sstable/SSTableReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/23d7bd85/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index e9a03c8..412e4c1 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -597,7 +597,7 @@ public class SSTableReader extends SSTable
  */
 public long estimatedKeys()
 {
-return indexSummary.size() * DatabaseDescriptor.getIndexInterval();
+return ((long) indexSummary.size()) * 
DatabaseDescriptor.getIndexInterval();
 }
 
 /**

[11/16] git commit: merge from 1.2

2013-08-08 Thread jbellis 
merge from 1.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f30629b8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f30629b8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f30629b8

Branch: refs/heads/cassandra-2.0.0
Commit: f30629b83fb14b6a93e38d440f79a16200cfcef2
Parents: 88fea9f 23d7bd8
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:15:54 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:15:54 2013 -0500

--
 src/java/org/apache/cassandra/io/sstable/SSTableReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f30629b8/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index e30e417,412e4c1..31a40cb
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@@ -594,7 -597,7 +594,7 @@@ public class SSTableReader extends SSTa
   */
  public long estimatedKeys()
  {
- return indexSummary.size() * indexSummary.getIndexInterval();
 -return ((long) indexSummary.size()) * 
DatabaseDescriptor.getIndexInterval();
++return ((long) indexSummary.size()) * indexSummary.getIndexInterval();
  }
  
  /**

[10/16] git commit: merge from 1.2

2013-08-08 Thread jbellis 
merge from 1.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f30629b8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f30629b8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f30629b8

Branch: refs/heads/cassandra-2.0
Commit: f30629b83fb14b6a93e38d440f79a16200cfcef2
Parents: 88fea9f 23d7bd8
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:15:54 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:15:54 2013 -0500

--
 src/java/org/apache/cassandra/io/sstable/SSTableReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f30629b8/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index e30e417,412e4c1..31a40cb
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@@ -594,7 -597,7 +594,7 @@@ public class SSTableReader extends SSTa
   */
  public long estimatedKeys()
  {
- return indexSummary.size() * indexSummary.getIndexInterval();
 -return ((long) indexSummary.size()) * 
DatabaseDescriptor.getIndexInterval();
++return ((long) indexSummary.size()) * indexSummary.getIndexInterval();
  }
  
  /**

[13/16] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0

2013-08-08 Thread jbellis 
Merge branch 'cassandra-2.0.0' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f2d04333
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f2d04333
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f2d04333

Branch: refs/heads/trunk
Commit: f2d04333eb5af0ae96ff2c42b09a818f899d1e7a
Parents: e533d5a f30629b
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:16:01 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:16:01 2013 -0500

--
 src/java/org/apache/cassandra/io/sstable/SSTableReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--

[15/16] git commit: comment

2013-08-08 Thread jbellis 
comment


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c5261312
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c5261312
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c5261312

Branch: refs/heads/trunk
Commit: c5261312459791ba6356b73cd61abf778879ed0e
Parents: f2d0433
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:18:41 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:18:41 2013 -0500

--
 src/java/org/apache/cassandra/utils/BiMultiValMap.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c5261312/src/java/org/apache/cassandra/utils/BiMultiValMap.java
--
diff --git a/src/java/org/apache/cassandra/utils/BiMultiValMap.java 
b/src/java/org/apache/cassandra/utils/BiMultiValMap.java
index 3712d25..d827d3f 100644
--- a/src/java/org/apache/cassandra/utils/BiMultiValMap.java
+++ b/src/java/org/apache/cassandra/utils/BiMultiValMap.java
@@ -29,7 +29,7 @@ import com.google.common.collect.Multimaps;
 /**
  *
  * A variant of BiMap which does not enforce uniqueness of values. This means 
the inverse
- * is a Multimap.
+ * is a Multimap.  (But the forward view is not a multimap; keys may only 
each have one value.)
  *
  * @param K
  * @param V

[12/16] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0

2013-08-08 Thread jbellis 
Merge branch 'cassandra-2.0.0' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f2d04333
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f2d04333
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f2d04333

Branch: refs/heads/cassandra-2.0
Commit: f2d04333eb5af0ae96ff2c42b09a818f899d1e7a
Parents: e533d5a f30629b
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 11:16:01 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 11:16:01 2013 -0500

--
 src/java/org/apache/cassandra/io/sstable/SSTableReader.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--

[jira] [Created] (CASSANDRA-5863) Create a Decompressed Chunk Cache

2013-08-08 Thread T Jake Luciani (JIRA) 
T Jake Luciani created CASSANDRA-5863:
-

 Summary: Create a Decompressed Chunk Cache
 Key: CASSANDRA-5863
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5863
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: T Jake Luciani


Currently, for every read the CRAR we reads compressed chunks into a byte[], 
check sends it to ICompressor and gets back another byte[] and verifies a 
checksum.  

This process is where the majority of time is spent in a read request.  

Before compression we would have zero-copy of data and could respond directly 
from the page-cache.

It would be useful to have some kind of Chunk cache that could speed up this 
process for hot data. Initially this could be a off heap cache but it would be 
great to put these decompressed chunks onto a SSD so the hot data lives on a 
fast disk similar to https://github.com/facebook/flashcache.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5863) Create a Decompressed Chunk Cache

2013-08-08 Thread T Jake Luciani (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-5863:
--

Description: 
Currently, for every read, the CRAR reads each compressed chunk into a byte[], 
sends it to ICompressor, gets back another byte[] and verifies a checksum.  

This process is where the majority of time is spent in a read request.  

Before compression, we would have zero-copy of data and could respond directly 
from the page-cache.

It would be useful to have some kind of Chunk cache that could speed up this 
process for hot data. Initially this could be a off heap cache but it would be 
great to put these decompressed chunks onto a SSD so the hot data lives on a 
fast disk similar to https://github.com/facebook/flashcache.



  was:
Currently, for every read the CRAR we reads compressed chunks into a byte[], 
check sends it to ICompressor and gets back another byte[] and verifies a 
checksum.  

This process is where the majority of time is spent in a read request.  

Before compression we would have zero-copy of data and could respond directly 
from the page-cache.

It would be useful to have some kind of Chunk cache that could speed up this 
process for hot data. Initially this could be a off heap cache but it would be 
great to put these decompressed chunks onto a SSD so the hot data lives on a 
fast disk similar to https://github.com/facebook/flashcache.




 Create a Decompressed Chunk Cache
 -

 Key: CASSANDRA-5863
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5863
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: T Jake Luciani

 Currently, for every read, the CRAR reads each compressed chunk into a 
 byte[], sends it to ICompressor, gets back another byte[] and verifies a 
 checksum.  
 This process is where the majority of time is spent in a read request.  
 Before compression, we would have zero-copy of data and could respond 
 directly from the page-cache.
 It would be useful to have some kind of Chunk cache that could speed up this 
 process for hot data. Initially this could be a off heap cache but it would 
 be great to put these decompressed chunks onto a SSD so the hot data lives on 
 a fast disk similar to https://github.com/facebook/flashcache.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733697#comment-13733697
 ] 

T Jake Luciani commented on CASSANDRA-5862:
---

I have been working on a jni library to allow fast CRC checks using the newest 
SSE instruction set. this is what Hadoop and Impala are now using.  I'll do 
some micro benchmarks of adler32 to this approach and see if it's worth it.  
The benefit of a fast CRC is the code could be used on commit log and thereby 
speed up writes as well.



 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5857) NumberFormatException during decommission

2013-08-08 Thread Brandon Williams (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-5857:


Attachment: 5857.txt

Patch to have SS look at the correct piece for the expire time.

 NumberFormatException during decommission
 -

 Key: CASSANDRA-5857
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5857
 Project: Cassandra
  Issue Type: Bug
Reporter: Brandon Williams
Assignee: Brandon Williams
 Attachments: 5857.txt


 We half-fixed this in CASSANDRA-5696, but unfortunately StorageService is 
 still looking at the token to get the expiretime in some cases.
 {noformat}
 java.lang.NumberFormatException: For input string: 
 113427455640312821154458202477256070484
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Long.parseLong(Long.java:444)
 at java.lang.Long.parseLong(Long.java:483)
 at 
 org.apache.cassandra.service.StorageService.extractExpireTime(StorageService.java:1660)
 at 
 org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1515)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:1234)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:953)
 at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:944)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:902)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:722)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5857) NumberFormatException during decommission

2013-08-08 Thread Brandon Williams (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-5857:


 Reviewer: jasobrown
 Priority: Minor  (was: Major)
Fix Version/s: 1.2.9

 NumberFormatException during decommission
 -

 Key: CASSANDRA-5857
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5857
 Project: Cassandra
  Issue Type: Bug
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 1.2.9

 Attachments: 5857.txt


 We half-fixed this in CASSANDRA-5696, but unfortunately StorageService is 
 still looking at the token to get the expiretime in some cases.
 {noformat}
 java.lang.NumberFormatException: For input string: 
 113427455640312821154458202477256070484
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Long.parseLong(Long.java:444)
 at java.lang.Long.parseLong(Long.java:483)
 at 
 org.apache.cassandra.service.StorageService.extractExpireTime(StorageService.java:1660)
 at 
 org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1515)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:1234)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:953)
 at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:944)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:902)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:722)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread Radim Kolar (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733752#comment-13733752
 ] 

Radim Kolar commented on CASSANDRA-5862:


Adler is unreliable, i had bad experience with it, switched from crc32 to adler 
for a while. Use CRC32C.

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733753#comment-13733753
 ] 

T Jake Luciani commented on CASSANDRA-5862:
---

Looks like Adler32 wins!

{code}
benchmark:
 [java]  0% Scenario{vm=java, trial=0, benchmark=JavaCrc32} 58541.04 ns; 
σ=334.60 ns @ 3 trials
 [java] 33% Scenario{vm=java, trial=0, benchmark=SSECrc32} 15504.47 ns; 
σ=46.66 ns @ 3 trials
 [java] 67% Scenario{vm=java, trial=0, benchmark=JavaAdler32} 6610.37 ns; 
σ=232.76 ns @ 10 trials
 [java]
 [java]   benchmarkus linear runtime
 [java]   JavaCrc32 58.54 ==
 [java]SSECrc32 15.50 ===
 [java] JavaAdler32  6.61 ===
 [java]
 [java] vm: java
 [java] trial: 0
{code}

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Remove jvm revision checks.. for now

2013-08-08 Thread brandonwilliams 
Updated Branches:
  refs/heads/trunk c7782f2de - 93f838d18


Remove jvm revision checks.. for now


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/93f838d1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/93f838d1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/93f838d1

Branch: refs/heads/trunk
Commit: 93f838d18518b42427f6596041a7d7edecda4ff4
Parents: c7782f2
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 12:57:04 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 12:57:04 2013 -0500

--
 .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
 1 file changed, 2 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/93f838d1/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java 
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index f49d549..20c70e7 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -157,7 +157,7 @@ public class CassandraDaemon
 {
 logger.warn(Non-Oracle JVM detected.  Some features, such as 
immediate unmap of compacted SSTables, may not work as intended);
 }
-else
+ /*   else
 {
 String[] java_version = javaVersion.split(_);
 String java_major = java_version[0];
@@ -172,18 +172,8 @@ public class CassandraDaemon
 logger.info(Unable to parse java version {}, 
Arrays.toString(java_version));
 java_minor = 32;
 }
-if (java_major.equals(1.6.0))
-{
-// These need to be updated from time to time, but these are 
currently valid (12.18.2012)
-if (java_minor  29)
-// Seen to be a major contributing factor for heap and 
load issues
-logger.error(Your JVM is out of date. Please upgrade to 
the newest Oracle Java 6.);
-else if (java_minor  32)
-// Updates 32+ have been seen to work well enough in the 
wild
-logger.warn(Your JVM is out of date. Please upgrade to 
the newest Oracle Java 6.);
-}
 }
-
+ */
 logger.info(Heap size: {}/{}, Runtime.getRuntime().totalMemory(), 
Runtime.getRuntime().maxMemory());
 logger.info(Classpath: {}, System.getProperty(java.class.path));
 CLibrary.tryMlockall();

[jira] [Commented] (CASSANDRA-5857) NumberFormatException during decommission

2013-08-08 Thread Jason Brown (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733770#comment-13733770
 ] 

Jason Brown commented on CASSANDRA-5857:


lgtm

 NumberFormatException during decommission
 -

 Key: CASSANDRA-5857
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5857
 Project: Cassandra
  Issue Type: Bug
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 1.2.9

 Attachments: 5857.txt


 We half-fixed this in CASSANDRA-5696, but unfortunately StorageService is 
 still looking at the token to get the expiretime in some cases.
 {noformat}
 java.lang.NumberFormatException: For input string: 
 113427455640312821154458202477256070484
 at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Long.parseLong(Long.java:444)
 at java.lang.Long.parseLong(Long.java:483)
 at 
 org.apache.cassandra.service.StorageService.extractExpireTime(StorageService.java:1660)
 at 
 org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1515)
 at 
 org.apache.cassandra.service.StorageService.onChange(StorageService.java:1234)
 at 
 org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:953)
 at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:944)
 at 
 org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:902)
 at 
 org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:722)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[10/10] git commit: Merge branch 'cassandra-2.0' into trunk

2013-08-08 Thread brandonwilliams 
Merge branch 'cassandra-2.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ac624536
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ac624536
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ac624536

Branch: refs/heads/trunk
Commit: ac62453649e1b47ec5a8c8d0a13687c9d7d2c481
Parents: 93f838d 988aa31
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:02:01 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:02:01 2013 -0500

--
 src/java/org/apache/cassandra/service/StorageService.java | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)
--

[6/6] git commit: Merge branch 'cassandra-2.0' into trunk

2013-08-08 Thread brandonwilliams 
Merge branch 'cassandra-2.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/28b827a8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/28b827a8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/28b827a8

Branch: refs/heads/trunk
Commit: 28b827a855e838c81c28bfef1746df48774b5935
Parents: ac62453 5d9a33a
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:03:30 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:03:30 2013 -0500

--

--

[06/10] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0.0

2013-08-08 Thread brandonwilliams 
Merge branch 'cassandra-1.2' into cassandra-2.0.0

Conflicts:
src/java/org/apache/cassandra/service/StorageService.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8ee09f7a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8ee09f7a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8ee09f7a

Branch: refs/heads/cassandra-2.0.0
Commit: 8ee09f7ab37bfd1cd8c3a09c8e0844d1fa6dfca2
Parents: f30629b ccb32a8
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:01:40 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:01:40 2013 -0500

--
 src/java/org/apache/cassandra/service/StorageService.java | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ee09f7a/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 58a0634,588777b..a23db0c
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -1587,12 -1646,19 +1587,9 @@@ public class StorageService extends Not
  }
  }
  
 -protected long extractExpireTime(String[] pieces, int version)
 +protected long extractExpireTime(String[] pieces)
  {
- if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
- return Long.parseLong(pieces[1]);
- else
- return Long.parseLong(pieces[2]);
 -if (version  MessagingService.VERSION_12)
 -{
 -if (pieces.length = 3)
 -return Long.parseLong(pieces[2]);
 -else
 -return 0L;
 -}
 -else
 -{
 -return Long.parseLong(pieces[2]);
 -}
++return Long.parseLong(pieces[2]);
  }
  
  /**

[3/6] git commit: Remove jvm revision checks.. for now

2013-08-08 Thread brandonwilliams 
Remove jvm revision checks.. for now


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/50b40e91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/50b40e91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/50b40e91

Branch: refs/heads/trunk
Commit: 50b40e91f0605f5dea56994fe4de47dfd6e46b36
Parents: 8ee09f7
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:03:08 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:03:08 2013 -0500

--
 .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
 1 file changed, 2 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/50b40e91/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java 
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index f49d549..20c70e7 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -157,7 +157,7 @@ public class CassandraDaemon
 {
 logger.warn(Non-Oracle JVM detected.  Some features, such as 
immediate unmap of compacted SSTables, may not work as intended);
 }
-else
+ /*   else
 {
 String[] java_version = javaVersion.split(_);
 String java_major = java_version[0];
@@ -172,18 +172,8 @@ public class CassandraDaemon
 logger.info(Unable to parse java version {}, 
Arrays.toString(java_version));
 java_minor = 32;
 }
-if (java_major.equals(1.6.0))
-{
-// These need to be updated from time to time, but these are 
currently valid (12.18.2012)
-if (java_minor  29)
-// Seen to be a major contributing factor for heap and 
load issues
-logger.error(Your JVM is out of date. Please upgrade to 
the newest Oracle Java 6.);
-else if (java_minor  32)
-// Updates 32+ have been seen to work well enough in the 
wild
-logger.warn(Your JVM is out of date. Please upgrade to 
the newest Oracle Java 6.);
-}
 }
-
+ */
 logger.info(Heap size: {}/{}, Runtime.getRuntime().totalMemory(), 
Runtime.getRuntime().maxMemory());
 logger.info(Classpath: {}, System.getProperty(java.class.path));
 CLibrary.tryMlockall();

[03/10] git commit: Make SS look at the right piece for expire time. Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857

2013-08-08 Thread brandonwilliams 
Make SS look at the right piece for expire time.
Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ccb32a8c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ccb32a8c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ccb32a8c

Branch: refs/heads/cassandra-2.0
Commit: ccb32a8c8d6bd5d25ad6db172f7cd6ead00fdfaa
Parents: 23d7bd8
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 12:58:58 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 12:58:58 2013 -0500

--
 src/java/org/apache/cassandra/service/StorageService.java | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ccb32a8c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 54f8abd..588777b 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1654,12 +1654,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return Long.parseLong(pieces[2]);
 else
 return 0L;
-} else
+}
+else
 {
-if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
-return Long.parseLong(pieces[1]);
-else
-return Long.parseLong(pieces[2]);
+return Long.parseLong(pieces[2]);
 }
 }

[01/10] git commit: Make SS look at the right piece for expire time. Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857

2013-08-08 Thread brandonwilliams 
Updated Branches:
  refs/heads/cassandra-1.2 23d7bd857 - ccb32a8c8
  refs/heads/cassandra-2.0 c52613124 - 988aa3163
  refs/heads/cassandra-2.0.0 f30629b83 - 8ee09f7ab
  refs/heads/trunk 93f838d18 - ac6245364


Make SS look at the right piece for expire time.
Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ccb32a8c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ccb32a8c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ccb32a8c

Branch: refs/heads/cassandra-1.2
Commit: ccb32a8c8d6bd5d25ad6db172f7cd6ead00fdfaa
Parents: 23d7bd8
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 12:58:58 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 12:58:58 2013 -0500

--
 src/java/org/apache/cassandra/service/StorageService.java | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ccb32a8c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 54f8abd..588777b 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1654,12 +1654,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return Long.parseLong(pieces[2]);
 else
 return 0L;
-} else
+}
+else
 {
-if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
-return Long.parseLong(pieces[1]);
-else
-return Long.parseLong(pieces[2]);
+return Long.parseLong(pieces[2]);
 }
 }

[04/10] git commit: Make SS look at the right piece for expire time. Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857

2013-08-08 Thread brandonwilliams 
Make SS look at the right piece for expire time.
Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ccb32a8c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ccb32a8c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ccb32a8c

Branch: refs/heads/trunk
Commit: ccb32a8c8d6bd5d25ad6db172f7cd6ead00fdfaa
Parents: 23d7bd8
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 12:58:58 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 12:58:58 2013 -0500

--
 src/java/org/apache/cassandra/service/StorageService.java | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ccb32a8c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 54f8abd..588777b 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1654,12 +1654,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return Long.parseLong(pieces[2]);
 else
 return 0L;
-} else
+}
+else
 {
-if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
-return Long.parseLong(pieces[1]);
-else
-return Long.parseLong(pieces[2]);
+return Long.parseLong(pieces[2]);
 }
 }

[08/10] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0

2013-08-08 Thread brandonwilliams 
Merge branch 'cassandra-2.0.0' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/988aa316
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/988aa316
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/988aa316

Branch: refs/heads/trunk
Commit: 988aa31631f59f2949cde68e4ffd9ddbe24fabe3
Parents: c526131 8ee09f7
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:01:51 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:01:51 2013 -0500

--
 src/java/org/apache/cassandra/service/StorageService.java | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)
--

[09/10] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0

2013-08-08 Thread brandonwilliams 
Merge branch 'cassandra-2.0.0' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/988aa316
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/988aa316
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/988aa316

Branch: refs/heads/cassandra-2.0
Commit: 988aa31631f59f2949cde68e4ffd9ddbe24fabe3
Parents: c526131 8ee09f7
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:01:51 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:01:51 2013 -0500

--
 src/java/org/apache/cassandra/service/StorageService.java | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)
--

[05/10] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0.0

2013-08-08 Thread brandonwilliams 
Merge branch 'cassandra-1.2' into cassandra-2.0.0

Conflicts:
src/java/org/apache/cassandra/service/StorageService.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8ee09f7a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8ee09f7a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8ee09f7a

Branch: refs/heads/cassandra-2.0
Commit: 8ee09f7ab37bfd1cd8c3a09c8e0844d1fa6dfca2
Parents: f30629b ccb32a8
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:01:40 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:01:40 2013 -0500

--
 src/java/org/apache/cassandra/service/StorageService.java | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ee09f7a/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 58a0634,588777b..a23db0c
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -1587,12 -1646,19 +1587,9 @@@ public class StorageService extends Not
  }
  }
  
 -protected long extractExpireTime(String[] pieces, int version)
 +protected long extractExpireTime(String[] pieces)
  {
- if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
- return Long.parseLong(pieces[1]);
- else
- return Long.parseLong(pieces[2]);
 -if (version  MessagingService.VERSION_12)
 -{
 -if (pieces.length = 3)
 -return Long.parseLong(pieces[2]);
 -else
 -return 0L;
 -}
 -else
 -{
 -return Long.parseLong(pieces[2]);
 -}
++return Long.parseLong(pieces[2]);
  }
  
  /**

[5/6] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0

2013-08-08 Thread brandonwilliams 
Merge branch 'cassandra-2.0.0' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d9a33a4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d9a33a4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d9a33a4

Branch: refs/heads/cassandra-2.0
Commit: 5d9a33a46c24c5aff9fb3d3497edcd23d7a6c196
Parents: 988aa31 50b40e9
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:03:17 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:03:17 2013 -0500

--
 .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
 1 file changed, 2 insertions(+), 12 deletions(-)
--

[07/10] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0.0

2013-08-08 Thread brandonwilliams 
Merge branch 'cassandra-1.2' into cassandra-2.0.0

Conflicts:
src/java/org/apache/cassandra/service/StorageService.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8ee09f7a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8ee09f7a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8ee09f7a

Branch: refs/heads/trunk
Commit: 8ee09f7ab37bfd1cd8c3a09c8e0844d1fa6dfca2
Parents: f30629b ccb32a8
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:01:40 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:01:40 2013 -0500

--
 src/java/org/apache/cassandra/service/StorageService.java | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8ee09f7a/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 58a0634,588777b..a23db0c
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -1587,12 -1646,19 +1587,9 @@@ public class StorageService extends Not
  }
  }
  
 -protected long extractExpireTime(String[] pieces, int version)
 +protected long extractExpireTime(String[] pieces)
  {
- if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
- return Long.parseLong(pieces[1]);
- else
- return Long.parseLong(pieces[2]);
 -if (version  MessagingService.VERSION_12)
 -{
 -if (pieces.length = 3)
 -return Long.parseLong(pieces[2]);
 -else
 -return 0L;
 -}
 -else
 -{
 -return Long.parseLong(pieces[2]);
 -}
++return Long.parseLong(pieces[2]);
  }
  
  /**

[2/6] git commit: Remove jvm revision checks.. for now

2013-08-08 Thread brandonwilliams 
Remove jvm revision checks.. for now


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/50b40e91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/50b40e91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/50b40e91

Branch: refs/heads/cassandra-2.0.0
Commit: 50b40e91f0605f5dea56994fe4de47dfd6e46b36
Parents: 8ee09f7
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:03:08 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:03:08 2013 -0500

--
 .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
 1 file changed, 2 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/50b40e91/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java 
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index f49d549..20c70e7 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -157,7 +157,7 @@ public class CassandraDaemon
 {
 logger.warn(Non-Oracle JVM detected.  Some features, such as 
immediate unmap of compacted SSTables, may not work as intended);
 }
-else
+ /*   else
 {
 String[] java_version = javaVersion.split(_);
 String java_major = java_version[0];
@@ -172,18 +172,8 @@ public class CassandraDaemon
 logger.info(Unable to parse java version {}, 
Arrays.toString(java_version));
 java_minor = 32;
 }
-if (java_major.equals(1.6.0))
-{
-// These need to be updated from time to time, but these are 
currently valid (12.18.2012)
-if (java_minor  29)
-// Seen to be a major contributing factor for heap and 
load issues
-logger.error(Your JVM is out of date. Please upgrade to 
the newest Oracle Java 6.);
-else if (java_minor  32)
-// Updates 32+ have been seen to work well enough in the 
wild
-logger.warn(Your JVM is out of date. Please upgrade to 
the newest Oracle Java 6.);
-}
 }
-
+ */
 logger.info(Heap size: {}/{}, Runtime.getRuntime().totalMemory(), 
Runtime.getRuntime().maxMemory());
 logger.info(Classpath: {}, System.getProperty(java.class.path));
 CLibrary.tryMlockall();

[1/6] git commit: Remove jvm revision checks.. for now

2013-08-08 Thread brandonwilliams 
Updated Branches:
  refs/heads/cassandra-2.0 988aa3163 - 5d9a33a46
  refs/heads/cassandra-2.0.0 8ee09f7ab - 50b40e91f
  refs/heads/trunk ac6245364 - 28b827a85


Remove jvm revision checks.. for now


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/50b40e91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/50b40e91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/50b40e91

Branch: refs/heads/cassandra-2.0
Commit: 50b40e91f0605f5dea56994fe4de47dfd6e46b36
Parents: 8ee09f7
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:03:08 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:03:08 2013 -0500

--
 .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
 1 file changed, 2 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/50b40e91/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java 
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index f49d549..20c70e7 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -157,7 +157,7 @@ public class CassandraDaemon
 {
 logger.warn(Non-Oracle JVM detected.  Some features, such as 
immediate unmap of compacted SSTables, may not work as intended);
 }
-else
+ /*   else
 {
 String[] java_version = javaVersion.split(_);
 String java_major = java_version[0];
@@ -172,18 +172,8 @@ public class CassandraDaemon
 logger.info(Unable to parse java version {}, 
Arrays.toString(java_version));
 java_minor = 32;
 }
-if (java_major.equals(1.6.0))
-{
-// These need to be updated from time to time, but these are 
currently valid (12.18.2012)
-if (java_minor  29)
-// Seen to be a major contributing factor for heap and 
load issues
-logger.error(Your JVM is out of date. Please upgrade to 
the newest Oracle Java 6.);
-else if (java_minor  32)
-// Updates 32+ have been seen to work well enough in the 
wild
-logger.warn(Your JVM is out of date. Please upgrade to 
the newest Oracle Java 6.);
-}
 }
-
+ */
 logger.info(Heap size: {}/{}, Runtime.getRuntime().totalMemory(), 
Runtime.getRuntime().maxMemory());
 logger.info(Classpath: {}, System.getProperty(java.class.path));
 CLibrary.tryMlockall();

[02/10] git commit: Make SS look at the right piece for expire time. Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857

2013-08-08 Thread brandonwilliams 
Make SS look at the right piece for expire time.
Patch by brandonwilliams reviewed by jasobrown for CASSANDRA-5857


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ccb32a8c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ccb32a8c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ccb32a8c

Branch: refs/heads/cassandra-2.0.0
Commit: ccb32a8c8d6bd5d25ad6db172f7cd6ead00fdfaa
Parents: 23d7bd8
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 12:58:58 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 12:58:58 2013 -0500

--
 src/java/org/apache/cassandra/service/StorageService.java | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ccb32a8c/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 54f8abd..588777b 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1654,12 +1654,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return Long.parseLong(pieces[2]);
 else
 return 0L;
-} else
+}
+else
 {
-if (VersionedValue.STATUS_LEFT.equals(pieces[0]))
-return Long.parseLong(pieces[1]);
-else
-return Long.parseLong(pieces[2]);
+return Long.parseLong(pieces[2]);
 }
 }

[jira] [Commented] (CASSANDRA-5686) During node repair operation, Cassandra throws exception on one of our nodes: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS

2013-08-08 Thread Robert Coli (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733779#comment-13733779
 ] 

Robert Coli commented on CASSANDRA-5686:


@OP : what version of cassandra on what platform, with which JVM?

 During node repair operation, Cassandra throws exception on one of our nodes: 
 failed to uncompress the chunk: FAILED_TO_UNCOMPRESS 
 ---

 Key: CASSANDRA-5686
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5686
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.5
Reporter: Rao

 During node repair operation, Cassandra throws exception on one of our nodes: 
 failed to uncompress the chunk: FAILED_TO_UNCOMPRESS 
  INFO [AntiEntropyStage:1] 2013-06-20 03:08:03,876 AntiEntropyService.java 
 (line 213) [repair #4ae2a970-d991-11e2-bd05-094c1105e54d] Received merkle 
 tree for tags from /135.163.214.175
 ERROR [Thread-40] 2013-06-20 03:08:03,903 CassandraDaemon.java (line 175) 
 Exception in thread Thread[Thread-40,5,main]
 java.lang.RuntimeException: java.io.IOException: failed to uncompress the 
 chunk: FAILED_TO_UNCOMPRESS(5)
 at 
 org.apache.cassandra.service.AntiEntropyService$Validator$ValidatorSerializer.deserialize(AntiEntropyService.java:438)
 at 
 org.apache.cassandra.service.AntiEntropyService$Validator$ValidatorSerializer.deserialize(AntiEntropyService.java:421)
 at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94)
 at 
 org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:203)
 at 
 org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:135)
 at 
 org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:82)
 Caused by: java.io.IOException: failed to uncompress the chunk: 
 FAILED_TO_UNCOMPRESS(5)
 at 
 org.xerial.snappy.SnappyInputStream.hasNextChunk(SnappyInputStream.java:361)
 at 
 org.xerial.snappy.SnappyInputStream.read(SnappyInputStream.java:383)
 at java.io.DataInputStream.readInt(DataInputStream.java:387)
 at 
 org.apache.cassandra.utils.MerkleTree$Leaf$LeafSerializer.deserialize(MerkleTree.java:793)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:920)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896)
 at 
 org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:713)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896)
 at 
 org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896)
 at 
 org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896)
 at 
 org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896)
 at 
 org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896)
 at 
 org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896)
 at 
 org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:713)
 at 
 org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918)
 at

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733778#comment-13733778
 ] 

T Jake Luciani commented on CASSANDRA-5862:
---

[~hsn] do you have any details? what size was the payload you were checksumming?

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread Lior Golan (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733784#comment-13733784
 ] 

Lior Golan commented on CASSANDRA-5862:
---

Are you computing the checksum on the data before or after compression? If you 
are computing the checksum of the uncompressed data, then it might be useful to 
switch to checksum the compressed data (less data to checksum)

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[4/6] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0

2013-08-08 Thread brandonwilliams 
Merge branch 'cassandra-2.0.0' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d9a33a4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d9a33a4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d9a33a4

Branch: refs/heads/trunk
Commit: 5d9a33a46c24c5aff9fb3d3497edcd23d7a6c196
Parents: 988aa31 50b40e9
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Aug 8 13:03:17 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Aug 8 13:03:17 2013 -0500

--
 .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
 1 file changed, 2 insertions(+), 12 deletions(-)
--

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread Radim Kolar (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733792#comment-13733792
 ] 

Radim Kolar commented on CASSANDRA-5862:


60-80 bytes

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733806#comment-13733806
 ] 

T Jake Luciani commented on CASSANDRA-5862:
---

[~hsn] ah ok, well as [~jbellis] mentioned it's a poor choice for small values. 
 Compression chunks are a much larger value (default of 64kb)

[~liorgo2] good point, we do checksum the uncompressed data, but since adler32 
is bad with small values we should prob keep it as is.

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: add thrift-server and LMAX Disruptor licences

2013-08-08 Thread xedin 
Updated Branches:
  refs/heads/cassandra-2.0.0 50b40e91f - 7a8cbb3ea


add thrift-server and LMAX Disruptor licences


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7a8cbb3e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7a8cbb3e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7a8cbb3e

Branch: refs/heads/cassandra-2.0.0
Commit: 7a8cbb3ea7e0898778ed41a6d19c40d67c065667
Parents: 50b40e9
Author: Pavel Yaskevich xe...@apache.org
Authored: Thu Aug 8 11:30:02 2013 -0700
Committer: Pavel Yaskevich xe...@apache.org
Committed: Thu Aug 8 11:30:02 2013 -0700

--
 lib/licenses/disruptor-3.0.1.txt | 201 +
 lib/licenses/thrift-server-0.3.0.txt | 202 ++
 2 files changed, 403 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7a8cbb3e/lib/licenses/disruptor-3.0.1.txt
--
diff --git a/lib/licenses/disruptor-3.0.1.txt b/lib/licenses/disruptor-3.0.1.txt
new file mode 100644
index 000..50086f8
--- /dev/null
+++ b/lib/licenses/disruptor-3.0.1.txt
@@ -0,0 +1,201 @@
+ Apache License
+   Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+  License shall mean the terms and conditions for use, reproduction,
+  and distribution as defined by Sections 1 through 9 of this document.
+
+  Licensor shall mean the copyright owner or entity authorized by
+  the copyright owner that is granting the License.
+
+  Legal Entity shall mean the union of the acting entity and all
+  other entities that control, are controlled by, or are under common
+  control with that entity. For the purposes of this definition,
+  control means (i) the power, direct or indirect, to cause the
+  direction or management of such entity, whether by contract or
+  otherwise, or (ii) ownership of fifty percent (50%) or more of the
+  outstanding shares, or (iii) beneficial ownership of such entity.
+
+  You (or Your) shall mean an individual or Legal Entity
+  exercising permissions granted by this License.
+
+  Source form shall mean the preferred form for making modifications,
+  including but not limited to software source code, documentation
+  source, and configuration files.
+
+  Object form shall mean any form resulting from mechanical
+  transformation or translation of a Source form, including but
+  not limited to compiled object code, generated documentation,
+  and conversions to other media types.
+
+  Work shall mean the work of authorship, whether in Source or
+  Object form, made available under the License, as indicated by a
+  copyright notice that is included in or attached to the work
+  (an example is provided in the Appendix below).
+
+  Derivative Works shall mean any work, whether in Source or Object
+  form, that is based on (or derived from) the Work and for which the
+  editorial revisions, annotations, elaborations, or other modifications
+  represent, as a whole, an original work of authorship. For the purposes
+  of this License, Derivative Works shall not include works that remain
+  separable from, or merely link (or bind by name) to the interfaces of,
+  the Work and Derivative Works thereof.
+
+  Contribution shall mean any work of authorship, including
+  the original version of the Work and any modifications or additions
+  to that Work or Derivative Works thereof, that is intentionally
+  submitted to Licensor for inclusion in the Work by the copyright owner
+  or by an individual or Legal Entity authorized to submit on behalf of
+  the copyright owner. For the purposes of this definition, submitted
+  means any form of electronic, verbal, or written communication sent
+  to the Licensor or its representatives, including but not limited to
+  communication on electronic mailing lists, source code control systems,
+  and issue tracking systems that are managed by, or on behalf of, the
+  Licensor for the purpose of discussing and improving the Work, but
+  excluding communication that is conspicuously marked or otherwise
+  designated in writing by the copyright owner as Not a Contribution.
+
+  Contributor shall mean Licensor and any individual or Legal Entity
+  on behalf of whom a Contribution has been received by Licensor and
+  subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+  this License, each

[jira] [Commented] (CASSANDRA-4573) HSHA doesn't handle large messages gracefully

2013-08-08 Thread Pavel Yaskevich (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733813#comment-13733813
 ] 

Pavel Yaskevich commented on CASSANDRA-4573:


[~jbellis] Done, licences committed to cassandra-2.0.0 for thrift-server and 
LMAX disruptor.

 HSHA doesn't handle large messages gracefully
 -

 Key: CASSANDRA-4573
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4573
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Pavel Yaskevich
 Fix For: 2.0

 Attachments: CASSANDRA-4573.patch, 
 disruptor-thrift-server-0.2.2-SNAPSHOT.jar, repro.py


 HSHA doesn't seem to enforce any kind of max message length, and when 
 messages are too large, it doesn't fail gracefully.
 With debug logs enabled, you'll see this:
 {{DEBUG 13:13:31,805 Unexpected state 16}}
 Which seems to mean that there's a SelectionKey that's valid, but isn't ready 
 for reading, writing, or accepting.
 Client-side, you'll get this thrift error (while trying to read a frame as 
 part of {{recv_batch_mutate}}):
 {{TTransportException: TSocket read 0 bytes}}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733811#comment-13733811
 ] 

Jonathan Ellis commented on CASSANDRA-5862:
---

bq. we do checksum the uncompressed data, but since adler32 is bad with small 
values we should prob keep it as is

Even with 10x compression (2x as high as I've actually seen) that's still 
thousands of bytes to work with.  I think we'd be okay.

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread Lior Golan (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733814#comment-13733814
 ] 

Lior Golan commented on CASSANDRA-5862:
---

What's the typical size of a compressed chunk (after compression)? If it's 
above a few 100 bytes then Adler should be ok. In addition, compressed data has 
higher entropy which improves the quality of the adler32 checksum

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data

2013-08-08 Thread Pavel Yaskevich (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733815#comment-13733815
 ] 

Pavel Yaskevich commented on CASSANDRA-5661:


I was actually destructed with other things and kind of waiting until code is 
done so I can test the version of multipool with all of the changes.

 Discard pooled readers for cold data
 

 Key: CASSANDRA-5661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 2.0.1

 Attachments: CASSANDRA-5661-global-multiway-cache.patch, 
 CASSANDRA-5661.patch, DominatorTree.png, Histogram.png


 Reader pooling was introduced in CASSANDRA-4942 but pooled 
 RandomAccessReaders are never cleaned up until the SSTableReader is closed.  
 So memory use is the worst case simultaneous RAR we had open for this file, 
 forever.
 We should introduce a global limit on how much memory to use for RAR, and 
 evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Cassandra Wiki] Update of HowToPublishReleases by EricEvans

2013-08-08 Thread Apache Wiki 
Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The HowToPublishReleases page has been changed by EricEvans:
https://wiki.apache.org/cassandra/HowToPublishReleases?action=diffrev1=24rev2=25

  [2]: http://s.apache.org/link (NEWS.txt)
  [3]: http://s.apache.org/link (Release notes from JIRA)
  }}}
+ 
+ === If The Vote Passes ===
+  1. Send a message declaring the vote closed, and summarizing the results.
+  1. Mark the version as released in 
[[https://issues.apache.org/jira/plugins/servlet/project-config/CASSANDRA/versions|JIRA]]
-  1. If the vote is a success, login to people.apache.org and put the release 
artifacts into {{{/www/www.apache.org/dist/cassandra}}}, e.g.
+  1. Login to people.apache.org and put the release artifacts into 
{{{/www/www.apache.org/dist/cassandra/version}}}, e.g.
{{{
  cd /www/www.apache.org/dist/cassandra
  mkdir X.Y.Z
@@ -105, +109 @@

  find X.Y.Z -type f -exec chmod 664 {} \;
  }}}
   After doing this, login to 
[[http://repository.apache.org|repository.apache.org]] and release the staging 
repository.
- 
-  If the vote failed then login to 
[[http://repository.apache.org|repository.apache.org]] and drop the staging 
repository and start over again with ''Take N+1''
- 
   1. ''TODO: Some stuff to do with updating the project site that I don't know 
(Stephen)''
   1. Marke the version as released in JIRA on the 
[[https://issues.apache.org/jira/secure/project/ManageVersions.jspa?pid=12310865|Manage
 Versions]] screen. ''TODO: Can somebody who has access please verify the link 
- I've inferred it (Stephen)''
   1. Wait for the sync
@@ -139, +140 @@

   1. Add the release to the PMC's next board report
   1. Remove Old Versions from ASF Distribution Directory. Login to 
people.apache.org and remove any superceded releases from 
{{{/www/www.apache.org/dist/cassandra}}} (after confirming that they exist in 
{{{/www/archive.apache.org/dist/cassandra}}}).
  
+ === If the vote fails ===
+  1. Login to [[http://repository.apache.org|repository.apache.org]] and drop 
the staging repository and start over again with ''Take N+1''
+

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data

2013-08-08 Thread Ben Manes (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733830#comment-13733830
 ] 

Ben Manes commented on CASSANDRA-5661:
--

My benchmark had a bug and EBS may only be on par with LTQ performance wise. I 
need to investigate that again, though.

I shifted focus to fixing the performance bottleneck in Guava's cache. The way 
we tracked usage history (e.g. LRU) was focused on common usage, but is a 
bottleneck on synthetic benchmarks. I made the fixes to CLHM (v1.4) and offered 
them upstream (issue 1487). I'll experiment with using CLHM instead to see if 
that removes the hotspot.

 Discard pooled readers for cold data
 

 Key: CASSANDRA-5661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 2.0.1

 Attachments: CASSANDRA-5661-global-multiway-cache.patch, 
 CASSANDRA-5661.patch, DominatorTree.png, Histogram.png


 Reader pooling was introduced in CASSANDRA-4942 but pooled 
 RandomAccessReaders are never cleaned up until the SSTableReader is closed.  
 So memory use is the worst case simultaneous RAR we had open for this file, 
 forever.
 We should introduce a global limit on how much memory to use for RAR, and 
 evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Cassandra Wiki] Update of HowToPublishReleases by EricEvans

2013-08-08 Thread Apache Wiki 
Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The HowToPublishReleases page has been changed by EricEvans:
https://wiki.apache.org/cassandra/HowToPublishReleases?action=diffrev1=25rev2=26

   1. Send a message declaring the vote closed, and summarizing the results.
   1. Mark the version as released in 
[[https://issues.apache.org/jira/plugins/servlet/project-config/CASSANDRA/versions|JIRA]]
   1. Login to people.apache.org and put the release artifacts into 
{{{/www/www.apache.org/dist/cassandra/version}}}, e.g.
-   {{{
+ {{{
- cd /www/www.apache.org/dist/cassandra
- mkdir X.Y.Z
- cd X.Y.Z
+ STAGING_REPOID=orgapachecassandra-062
+ RELEASE=2.0.0-rc1
+ 
  for type in bin src; do \
-   for part in gz gz.md5 gz.sha1 gz.asc gz.asc.md5 gz.asc.sha1; do \
+   for part in gz gz.md5 gz.sha1 gz.asc; do \
- curl -O https://repository.apache.org/content/repositories/''staging 
repository 
id''/org/apache/cassandra/apache-cassandra/X.Y.Z/apache-cassandra-X.Y.Z-$type.tar.$part;
 \
+ curl -O 
https://repository.apache.org/content/repositories/$STAGING_REPOID/org/apache/cassandra/apache-cassandra/$RELEASE/apache-cassandra-$RELEASE-$type.tar.$part;
 \
done \
  done
- cd ..
+ 
- find X.Y.Z -type d -exec chmod a+rx,g+w {} \;
+ find . -exec chmod a+rx,g+w {} \;
- find X.Y.Z -type f -exec chmod 664 {} \;
+ find . -type f -exec chmod 664 {} \;
  }}}
-  After doing this, login to 
[[http://repository.apache.org|repository.apache.org]] and release the staging 
repository.
+  1. Login to [[http://repository.apache.org|repository.apache.org]] and 
release the staging repository.
   1. ''TODO: Some stuff to do with updating the project site that I don't know 
(Stephen)''
   1. Marke the version as released in JIRA on the 
[[https://issues.apache.org/jira/secure/project/ManageVersions.jspa?pid=12310865|Manage
 Versions]] screen. ''TODO: Can somebody who has access please verify the link 
- I've inferred it (Stephen)''
   1. Wait for the sync

[Cassandra Wiki] Update of HowToPublishReleases by EricEvans

2013-08-08 Thread Apache Wiki 
Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The HowToPublishReleases page has been changed by EricEvans:
https://wiki.apache.org/cassandra/HowToPublishReleases?action=diffrev1=26rev2=27

  find . -type f -exec chmod 664 {} \;
  }}}
   1. Login to [[http://repository.apache.org|repository.apache.org]] and 
release the staging repository.
-  1. ''TODO: Some stuff to do with updating the project site that I don't know 
(Stephen)''
-  1. Marke the version as released in JIRA on the 
[[https://issues.apache.org/jira/secure/project/ManageVersions.jspa?pid=12310865|Manage
 Versions]] screen. ''TODO: Can somebody who has access please verify the link 
- I've inferred it (Stephen)''
-  1. Wait for the sync
+  1. Remove the tentative tag; Create the permanent tag (preferably signed / 
annotated)
+  1. Wait for the sync to http://www.apache.org/dist/cassandra before 
continuing
+  1. Update the website.
   1. Publish the announce email to 
[[mailto:u...@cassandra.apache.org|u...@cassandra.apache.org]]
{{{
  Subject: [ANN] Apache Cassandra X.Y.Z released

Git Push Summary

2013-08-08 Thread eevans 
Updated Tags:  refs/tags/2.0.0-rc1-tentative [deleted] e8ae6720e

Git Push Summary

2013-08-08 Thread eevans 
Updated Tags:  refs/tags/cassandra-2.0.0-rc1 [created] 930e60a7a

[jira] [Updated] (CASSANDRA-5855) Scrub does not understand compound primary key created in CQL 3 beta

2013-08-08 Thread Tyler Hobbs (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-5855:
---

Attachment: 0001-Correctly-validate-sparse-composite-columns.patch

Patch 0001 should apply to 1.1 or 1.2.  It's not the most general technique, so 
I welcome any suggestions, but it does fix this particular issue.  If a sparse 
composite schema is being used, only the last composite component is used to 
try to look up the column validator.

I'll also note that this code path is currently only used by scrub, so it 
should be safe to test.

 Scrub does not understand compound primary key created in CQL 3 beta
 

 Key: CASSANDRA-5855
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5855
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: J.B. Langston
Assignee: Tyler Hobbs
 Attachments: 0001-Correctly-validate-sparse-composite-columns.patch


 We have a customer who was using the beta version of CQL 3 in DSE 3.0 which 
 includes Cassandra 1.1.9 plus patches backported from later versions.  
 They've now upgraded to DSE 3.1, which includes Cassandra 1.2.6 plus patches.
 When restarting for the first time after running upgradesstables, they 
 noticed the following error in the log:
 {noformat}
 Thread[SSTableBatchOpen:2,5,main]
 java.lang.AssertionError
 at 
 org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
 at 
 org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
 at 
 org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {noformat}
 This error was also reported on CASSANDRA-5703.  The comments suggested it 
 was caused by an empty row key, so I had them run scrub on it.  When they 
 did, scrub reported the following warning almost 4 million times:
 {noformat}
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 52) Row at 530332255 is unreadable; skipping to next
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 57) Non-fatal error reading row (stacktrace follows)
 java.lang.RuntimeException: Error validating row 
 DecoratedKey(139154446688383793922009760478335751546, 
 735fc9da503b11e2844b123140ff209f)
  at 
 org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:243)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.init(PrecompactedRow.java:98)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166)
  at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:529)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.doScrub(CompactionManager.java:518)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:73)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$3.perform(CompactionManager.java:283)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:253)
  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)
 Caused by: org.apache.cassandra.db.marshal.MarshalException: String didn't 
 validate.
  at org.apache.cassandra.db.marshal.UTF8Type.validate(UTF8Type.java:66)
  at

[jira] [Updated] (CASSANDRA-5859) Don't return internal StreamState objects from streaming mbeans

2013-08-08 Thread Yuki Morishita (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-5859:
--

Attachment: 5859.txt

Initial patch attached. (Also on github: 
https://github.com/yukim/cassandra/commits/5859)

Since StreamState is complex object, I used JMX's 
[CompositeData|http://docs.oracle.com/javase/7/docs/api/javax/management/openmbean/CompositeData.html]
 to convert StreamState to be transferred over JMX.
Patch also adds support for exposing JMX notification support for stream events 
to StreamManagerMBean, so one can just monitor start/progress/completion of 
streaming.

 Don't return internal StreamState objects from streaming mbeans
 ---

 Key: CASSANDRA-5859
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5859
 Project: Cassandra
  Issue Type: Bug
Reporter: Nick Bailey
Assignee: Yuki Morishita
 Fix For: 2.0

 Attachments: 5859.txt


 The stream manager mbean returns StreamState objects. We want to avoid 
 returning internal C* objects over jmx. We should switch to a map or 
 something similar that can represent streaming state.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1511991 - in /cassandra/site: publish/download/index.html src/settings.py

2013-08-08 Thread eevans 
Author: eevans
Date: Thu Aug  8 19:46:18 2013
New Revision: 1511991

URL: http://svn.apache.org/r1511991
Log:
update versioning or 2.0.0 rc1 release

Modified:
cassandra/site/publish/download/index.html
cassandra/site/src/settings.py

Modified: cassandra/site/publish/download/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1511991r1=1511990r2=1511991view=diff
==
--- cassandra/site/publish/download/index.html (original)
+++ cassandra/site/publish/download/index.html Thu Aug  8 19:46:18 2013
@@ -107,16 +107,16 @@
 
   h2 class=hdrDevelopment Cassandra Server Releases (not production 
ready)/h2
   p
-  The latest development release is 2.0.0-beta2 (released on
-  2013-07-25).
+  The latest development release is 2.0.0-rc1 (released on
+  2013-08-08).
   /p
 
   ul
 li
-a class=filename 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-bin.tar.gz;apache-cassandra-2.0.0-beta2-bin.tar.gz/a
-[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-bin.tar.gz.asc;PGP/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-bin.tar.gz.md5;MD5/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-bin.tar.gz.sha1;SHA1/a]
+a class=filename 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-bin.tar.gz;apache-cassandra-2.0.0-rc1-bin.tar.gz/a
+[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-bin.tar.gz.asc;PGP/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-bin.tar.gz.md5;MD5/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-bin.tar.gz.sha1;SHA1/a]
 /li
   /ul
   
@@ -195,10 +195,10 @@
   
   
 li
-a class=filename 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-src.tar.gz;apache-cassandra-2.0.0-beta2-src.tar.gz/a
-[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-src.tar.gz.asc;PGP/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-src.tar.gz.md5;MD5/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-beta2-src.tar.gz.sha1;SHA1/a]
+a class=filename 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-src.tar.gz;apache-cassandra-2.0.0-rc1-src.tar.gz/a
+[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-src.tar.gz.asc;PGP/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-src.tar.gz.md5;MD5/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.0.0/apache-cassandra-2.0.0-rc1-src.tar.gz.sha1;SHA1/a]
 /li
   
   /ul 

Modified: cassandra/site/src/settings.py
URL: 
http://svn.apache.org/viewvc/cassandra/site/src/settings.py?rev=1511991r1=1511990r2=1511991view=diff
==
--- cassandra/site/src/settings.py (original)
+++ cassandra/site/src/settings.py Thu Aug  8 19:46:18 2013
@@ -100,8 +100,8 @@ class CassandraDef(object):
 veryoldstable_exists = True
 stable_version = '1.2.8'
 stable_release_date = '2013-07-28'
-devel_version = '2.0.0-beta2'
-devel_release_date = '2013-07-25'
+devel_version = '2.0.0-rc1'
+devel_release_date = '2013-08-08'
 devel_exists = True
 _apache_base_url = 'http://www.apache.org'
 _svn_base_url = 'https://svn.apache.org/repos/asf'

[jira] [Updated] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread T Jake Luciani (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-5862:
--

Attachment: 5862.txt

Patch attached, I switched to adler32 and changed checksumming to be on the 
compressed value.  Going to run some benchmarks now.

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1

 Attachments: 5862.txt


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5859) Don't return internal StreamState objects from streaming mbeans

2013-08-08 Thread Jonathan Ellis (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5859:
--

Reviewer: thobbs

 Don't return internal StreamState objects from streaming mbeans
 ---

 Key: CASSANDRA-5859
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5859
 Project: Cassandra
  Issue Type: Bug
Reporter: Nick Bailey
Assignee: Yuki Morishita
  Labels: streaming
 Fix For: 2.0

 Attachments: 5859.txt


 The stream manager mbean returns StreamState objects. We want to avoid 
 returning internal C* objects over jmx. We should switch to a map or 
 something similar that can represent streaming state.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread Radim Kolar (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733923#comment-13733923
 ] 

Radim Kolar commented on CASSANDRA-5862:


Today all new projects should use CRC32C, it can be done in cpu hardware SSE4 
and polynomial is improved from crc32. 

In my test notes Adler32 has 5 times higher collision rate then CRC32, CRC32C 
is better then CRC32 but very little.

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1

 Attachments: 5862.txt


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5855) Scrub does not understand compound primary key created in CQL 3 beta

2013-08-08 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733928#comment-13733928
 ] 

Jonathan Ellis commented on CASSANDRA-5855:
---

It looks to me like we actually need to handle 3 cases:

# classic one-cell-per-column, no composites; the existing code
# CQL-style one-cell-per-column; the code you added
# COMPACT style, multiple columns in one cell; need to split them up and check 
each value

Nit: prefer assigning variables only once, e.g. we could write the 
validationName code [without addressing case 3 above] as

{code}
ByteBuffer validationName;
if (cfdef.isComposite  !cfdef.isCompact)
{
AbstractCompositeType comparator = (AbstractCompositeType) 
metadata.comparator;
ListAbstractCompositeType.CompositeComponent components = 
comparator.deconstruct(name);
validationName = components.get(components.size() - 1).value;
}
else
{
validationName = name;
}
{code}


 Scrub does not understand compound primary key created in CQL 3 beta
 

 Key: CASSANDRA-5855
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5855
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: J.B. Langston
Assignee: Tyler Hobbs
 Attachments: 0001-Correctly-validate-sparse-composite-columns.patch


 We have a customer who was using the beta version of CQL 3 in DSE 3.0 which 
 includes Cassandra 1.1.9 plus patches backported from later versions.  
 They've now upgraded to DSE 3.1, which includes Cassandra 1.2.6 plus patches.
 When restarting for the first time after running upgradesstables, they 
 noticed the following error in the log:
 {noformat}
 Thread[SSTableBatchOpen:2,5,main]
 java.lang.AssertionError
 at 
 org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
 at 
 org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
 at 
 org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {noformat}
 This error was also reported on CASSANDRA-5703.  The comments suggested it 
 was caused by an empty row key, so I had them run scrub on it.  When they 
 did, scrub reported the following warning almost 4 million times:
 {noformat}
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 52) Row at 530332255 is unreadable; skipping to next
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 57) Non-fatal error reading row (stacktrace follows)
 java.lang.RuntimeException: Error validating row 
 DecoratedKey(139154446688383793922009760478335751546, 
 735fc9da503b11e2844b123140ff209f)
  at 
 org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:243)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.init(PrecompactedRow.java:98)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166)
  at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:529)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.doScrub(CompactionManager.java:518)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:73)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$3.perform(CompactionManager.java:283)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:253)
  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  at

[jira] [Commented] (CASSANDRA-5855) Scrub does not understand compound primary key created in CQL 3 beta

2013-08-08 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733932#comment-13733932
 ] 

Jonathan Ellis commented on CASSANDRA-5855:
---

Actually, COMPACT case is handled by validateName so we're good.  I'll fix the 
nit and commit.

 Scrub does not understand compound primary key created in CQL 3 beta
 

 Key: CASSANDRA-5855
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5855
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: J.B. Langston
Assignee: Tyler Hobbs
 Attachments: 0001-Correctly-validate-sparse-composite-columns.patch


 We have a customer who was using the beta version of CQL 3 in DSE 3.0 which 
 includes Cassandra 1.1.9 plus patches backported from later versions.  
 They've now upgraded to DSE 3.1, which includes Cassandra 1.2.6 plus patches.
 When restarting for the first time after running upgradesstables, they 
 noticed the following error in the log:
 {noformat}
 Thread[SSTableBatchOpen:2,5,main]
 java.lang.AssertionError
 at 
 org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
 at 
 org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
 at 
 org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {noformat}
 This error was also reported on CASSANDRA-5703.  The comments suggested it 
 was caused by an empty row key, so I had them run scrub on it.  When they 
 did, scrub reported the following warning almost 4 million times:
 {noformat}
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 52) Row at 530332255 is unreadable; skipping to next
  WARN [CompactionExecutor:27] 2013-08-02 10:13:13,041 OutputHandler.java 
 (line 57) Non-fatal error reading row (stacktrace follows)
 java.lang.RuntimeException: Error validating row 
 DecoratedKey(139154446688383793922009760478335751546, 
 735fc9da503b11e2844b123140ff209f)
  at 
 org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:243)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114)
  at 
 org.apache.cassandra.db.compaction.PrecompactedRow.init(PrecompactedRow.java:98)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160)
  at 
 org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166)
  at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:529)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.doScrub(CompactionManager.java:518)
  at 
 org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:73)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$3.perform(CompactionManager.java:283)
  at 
 org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:253)
  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)
 Caused by: org.apache.cassandra.db.marshal.MarshalException: String didn't 
 validate.
  at org.apache.cassandra.db.marshal.UTF8Type.validate(UTF8Type.java:66)
  at org.apache.cassandra.db.Column.validateFields(Column.java:292)
  at 
 org.apache.cassandra.db.ColumnFamily.validateColumnFields(ColumnFamily.java:382)
  at 
 org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:239)
  ... 15 more
 {noformat}
 The customer did

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733952#comment-13733952
 ] 

Jonathan Ellis commented on CASSANDRA-5862:
---

Radim: hardware CRC32C is exactly what Jake tested above.

Jake: the dual checksum fields in CRAR are a bit clunky; the approach of 
instantiating the correct one in the constructor that you use in CIS is cleaner.

Nit: if we can come up with a single name to represent adler and 
post-compression then I'd prefer that to passing two booleans around that are 
always the same value.

Otherwise LGTM.

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1

 Attachments: 5862.txt


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733953#comment-13733953
 ] 

T Jake Luciani commented on CASSANDRA-5862:
---

with current crc32
{code}
verages from the middle 80% of values:
interval_op_rate  : 14556
interval_key_rate : 14556
latency median: 2.7
latency 95th percentile   : 4.7
latency 99.9th percentile : 37.0
Total operation time  : 00:01:15
END
{code}

with adler32
{code}
Averages from the middle 80% of values:
interval_op_rate  : 17985
interval_key_rate : 17985
latency median: 2.0
latency 95th percentile   : 3.9
latency 99.9th percentile : 73.7
Total operation time  : 00:01:02
END
{code}

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1

 Attachments: 5862.txt


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733955#comment-13733955
 ] 

T Jake Luciani edited comment on CASSANDRA-5862 at 8/8/13 8:45 PM:
---

bq. the dual checksum fields in CRAR are a bit clunky; the approach of 
instantiating the correct one in the constructor that you use in CIS is cleaner.

My concern was the instance would be re-used across multiple sstables therefore 
should be chosen per buffer. Is that not the case?

  was (Author: tjake):
.bq the dual checksum fields in CRAR are a bit clunky; the approach of 
instantiating the correct one in the constructor that you use in CIS is cleaner.

My concern was the instance would be re-used across multiple sstables therefore 
should be chosen per buffer. Is that not the case?
  
 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1

 Attachments: 5862.txt


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733955#comment-13733955
 ] 

T Jake Luciani commented on CASSANDRA-5862:
---

.bq the dual checksum fields in CRAR are a bit clunky; the approach of 
instantiating the correct one in the constructor that you use in CIS is cleaner.

My concern was the instance would be re-used across multiple sstables therefore 
should be chosen per buffer. Is that not the case?

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1

 Attachments: 5862.txt


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5862) Switch to adler checksum for sstables

2013-08-08 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13733960#comment-13733960
 ] 

T Jake Luciani commented on CASSANDRA-5862:
---

bq.  if we can come up with a single name to represent adler and 
post-compression then I'd prefer 

I was trying to future proof us switching back to crc or something but not 
changing the post-compression.  Think its safe to merge? 

 Switch to adler checksum for sstables
 -

 Key: CASSANDRA-5862
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5862
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Assignee: T Jake Luciani
 Fix For: 2.0.1

 Attachments: 5862.txt


 Adler is significantly faster than CRC32: 
 http://java-performance.info/java-crc32-and-adler32/
 (Adler is weaker for short inputs, so we should leave the commitlog alone, as 
 it checksums each mutation individually.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Correctly validate sparse composite cells patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855

2013-08-08 Thread jbellis 
Updated Branches:
  refs/heads/cassandra-1.1 238139cda - eb884a582


Correctly validate sparse composite cells
patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb884a58
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb884a58
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb884a58

Branch: refs/heads/cassandra-1.1
Commit: eb884a582b709d67fdd075aedf19facf6803bf93
Parents: 238139c
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:33:39 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:43:43 2013 -0500

--
 CHANGES.txt  |  1 +
 src/java/org/apache/cassandra/db/Column.java | 19 ++-
 2 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 75b349f..8790518 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
 1.1.next
  * Backport compaction exception handling from 1.2
+ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
 
 
 1.1.12

http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java 
b/src/java/org/apache/cassandra/db/Column.java
index 97aece0..ea83146 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -29,6 +29,7 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CFDefinition;
 import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.io.util.DataOutputBuffer;
 import org.apache.cassandra.utils.Allocator;
@@ -277,7 +278,23 @@ public class Column implements IColumn
 public void validateFields(CFMetaData metadata) throws MarshalException
 {
 validateName(metadata);
-AbstractType? valueValidator = metadata.getValueValidator(name());
+CFDefinition cfdef = metadata.getCfDef();
+
+// If this is a CQL table, we need to pull out the CQL column name to 
look up the correct column type.
+// (Note that COMPACT composites are handled by validateName, above.)
+ByteBuffer internalName;
+if (cfdef.isComposite  !cfdef.isCompact)
+{
+AbstractCompositeType comparator = (AbstractCompositeType) 
metadata.comparator;
+ListAbstractCompositeType.CompositeComponent components = 
comparator.deconstruct(name);
+internalName = components.get(components.size() - 1).value;
+}
+else
+{
+internalName = name;
+}
+
+AbstractType? valueValidator = 
metadata.getValueValidator(internalName);
 if (valueValidator != null)
 valueValidator.validate(value());
 }

[04/16] git commit: merge from 1.1

2013-08-08 Thread jbellis 
merge from 1.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1143df1a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1143df1a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1143df1a

Branch: refs/heads/cassandra-2.0
Commit: 1143df1aa439618bff6005ca1cee34cb066a
Parents: 23d7bd8 eb884a5
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:46:52 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:46:52 2013 -0500

--
 CHANGES.txt  |  2 ++
 src/java/org/apache/cassandra/db/Column.java | 19 ++-
 2 files changed, 20 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1143df1a/CHANGES.txt
--
diff --cc CHANGES.txt
index 859e7f9,8790518..30ce976
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,214 -1,19 +1,216 @@@
 -1.1.next
 - * Backport compaction exception handling from 1.2
 +1.2.9
 + * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831)
 + * fix bulk-loading compressed sstables (CASSANDRA-5820)
 + * (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter 
 +   (CASSANDRA-5824)
 + * update default LCS sstable size to 160MB (CASSANDRA-5727)
 + * Allow compacting 2Is via nodetool (CASSANDRA-5670)
 + * Hex-encode non-String keys in OPP (CASSANDRA-5793)
 + * nodetool history logging (CASSANDRA-5823)
 + * (Hadoop) fix support for Thrift tables in CqlPagingRecordReader 
 +   (CASSANDRA-5752)
 + * add all time blocked to StatusLogger output (CASSANDRA-5825)
 + * Future-proof inter-major-version schema migrations (CASSANDRA-5845)
 + * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
 +   (CASSANDRA-5718)
++Merged from 1.1:
+  * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
  
  
 -1.1.12
 +1.2.8
 + * Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814)
 + * cqlsh: add collections support to COPY (CASSANDRA-5698)
 + * retry important messages for any IOException (CASSANDRA-5804)
 + * Allow empty IN relations in SELECT/UPDATE/DELETE statements 
(CASSANDRA-5626)
 + * cqlsh: fix crashing on Windows due to libedit detection (CASSANDRA-5812)
 +
 +
 +1.2.7
 + * if no seeds can be a reached a node won't start in a ring by itself 
(CASSANDRA-5768)
 + * add cassandra.unsafesystem property (CASSANDRA-5704)
 + * (Hadoop) quote identifiers in CqlPagingRecordReader (CASSANDRA-5763)
 + * Add replace_node functionality for vnodes (CASSANDRA-5337)
 + * Add timeout events to query traces (CASSANDRA-5520)
 + * make starting native protocol server idempotent (CASSANDRA-5728)
 + * Fix loading key cache when a saved entry is no longer valid 
(CASSANDRA-5706)
 + * Fix serialization of the LEFT gossip value (CASSANDRA-5696)
 + * Pig: support for cql3 tables (CASSANDRA-5234)
 + * cqlsh: Don't show 'null' in place of empty values (CASSANDRA-5675)
 + * Race condition in detecting version on a mixed 1.1/1.2 cluster
 +   (CASSANDRA-5692)
 + * Fix skipping range tombstones with reverse queries (CASSANDRA-5712)
 + * Expire entries out of ThriftSessionManager (CASSANRDA-5719)
 + * Don't keep ancestor information in memory (CASSANDRA-5342)
 + * cqlsh: fix handling of semicolons inside BATCH queries (CASSANDRA-5697)
 + * Expose native protocol server status in nodetool info (CASSANDRA-5735)
 + * Fix pathetic performance of range tombstones (CASSANDRA-5677)
 + * Fix querying with an empty (impossible) range (CASSANDRA-5573)
 + * cqlsh: handle CUSTOM 2i in DESCRIBE output (CASSANDRA-5760)
 + * Fix minor bug in Range.intersects(Bound) (CASSANDRA-5771)
 + * cqlsh: handle disabled compression in DESCRIBE output (CASSANDRA-5766)
 + * Ensure all UP events are notified on the native protocol (CASSANDRA-5769)
 + * Fix formatting of sstable2json with multiple -k arguments (CASSANDRA-5781)
 + * Don't rely on row marker for queries in general to hide lost markers
 +   after TTL expires (CASSANDRA-5762)
 + * Sort nodetool help output (CASSANDRA-5776)
 + * Fix column expiring during 2 phases compaction (CASSANDRA-5799)
 + * now() is being rejected in INSERTs when inside collections (CASSANDRA-5795)
 +
 +
 +1.2.6
 + * Fix tracing when operation completes before all responses arrive 
 +   (CASSANDRA-5668)
 + * Fix cross-DC mutation forwarding (CASSANDRA-5632)
 + * Reduce SSTableLoader memory usage (CASSANDRA-)
 + * Scale hinted_handoff_throttle_in_kb to cluster size (CASSANDRA-5272)
 + * (Hadoop) Add CQL3 input/output formats (CASSANDRA-4421, 5622)
 + * (Hadoop) Fix InputKeyRange in CFIF (CASSANDRA-5536)
 + * Fix dealing with ridiculously large max sstable sizes in LCS 
(CASSANDRA-5589)
 + * Ignore pre-truncate hints (CASSANDRA-4655)
 + * Move

[05/16] git commit: merge from 1.1

2013-08-08 Thread jbellis 
merge from 1.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1143df1a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1143df1a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1143df1a

Branch: refs/heads/trunk
Commit: 1143df1aa439618bff6005ca1cee34cb066a
Parents: 23d7bd8 eb884a5
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:46:52 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:46:52 2013 -0500

--
 CHANGES.txt  |  2 ++
 src/java/org/apache/cassandra/db/Column.java | 19 ++-
 2 files changed, 20 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1143df1a/CHANGES.txt
--
diff --cc CHANGES.txt
index 859e7f9,8790518..30ce976
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,214 -1,19 +1,216 @@@
 -1.1.next
 - * Backport compaction exception handling from 1.2
 +1.2.9
 + * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831)
 + * fix bulk-loading compressed sstables (CASSANDRA-5820)
 + * (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter 
 +   (CASSANDRA-5824)
 + * update default LCS sstable size to 160MB (CASSANDRA-5727)
 + * Allow compacting 2Is via nodetool (CASSANDRA-5670)
 + * Hex-encode non-String keys in OPP (CASSANDRA-5793)
 + * nodetool history logging (CASSANDRA-5823)
 + * (Hadoop) fix support for Thrift tables in CqlPagingRecordReader 
 +   (CASSANDRA-5752)
 + * add all time blocked to StatusLogger output (CASSANDRA-5825)
 + * Future-proof inter-major-version schema migrations (CASSANDRA-5845)
 + * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
 +   (CASSANDRA-5718)
++Merged from 1.1:
+  * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
  
  
 -1.1.12
 +1.2.8
 + * Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814)
 + * cqlsh: add collections support to COPY (CASSANDRA-5698)
 + * retry important messages for any IOException (CASSANDRA-5804)
 + * Allow empty IN relations in SELECT/UPDATE/DELETE statements 
(CASSANDRA-5626)
 + * cqlsh: fix crashing on Windows due to libedit detection (CASSANDRA-5812)
 +
 +
 +1.2.7
 + * if no seeds can be a reached a node won't start in a ring by itself 
(CASSANDRA-5768)
 + * add cassandra.unsafesystem property (CASSANDRA-5704)
 + * (Hadoop) quote identifiers in CqlPagingRecordReader (CASSANDRA-5763)
 + * Add replace_node functionality for vnodes (CASSANDRA-5337)
 + * Add timeout events to query traces (CASSANDRA-5520)
 + * make starting native protocol server idempotent (CASSANDRA-5728)
 + * Fix loading key cache when a saved entry is no longer valid 
(CASSANDRA-5706)
 + * Fix serialization of the LEFT gossip value (CASSANDRA-5696)
 + * Pig: support for cql3 tables (CASSANDRA-5234)
 + * cqlsh: Don't show 'null' in place of empty values (CASSANDRA-5675)
 + * Race condition in detecting version on a mixed 1.1/1.2 cluster
 +   (CASSANDRA-5692)
 + * Fix skipping range tombstones with reverse queries (CASSANDRA-5712)
 + * Expire entries out of ThriftSessionManager (CASSANRDA-5719)
 + * Don't keep ancestor information in memory (CASSANDRA-5342)
 + * cqlsh: fix handling of semicolons inside BATCH queries (CASSANDRA-5697)
 + * Expose native protocol server status in nodetool info (CASSANDRA-5735)
 + * Fix pathetic performance of range tombstones (CASSANDRA-5677)
 + * Fix querying with an empty (impossible) range (CASSANDRA-5573)
 + * cqlsh: handle CUSTOM 2i in DESCRIBE output (CASSANDRA-5760)
 + * Fix minor bug in Range.intersects(Bound) (CASSANDRA-5771)
 + * cqlsh: handle disabled compression in DESCRIBE output (CASSANDRA-5766)
 + * Ensure all UP events are notified on the native protocol (CASSANDRA-5769)
 + * Fix formatting of sstable2json with multiple -k arguments (CASSANDRA-5781)
 + * Don't rely on row marker for queries in general to hide lost markers
 +   after TTL expires (CASSANDRA-5762)
 + * Sort nodetool help output (CASSANDRA-5776)
 + * Fix column expiring during 2 phases compaction (CASSANDRA-5799)
 + * now() is being rejected in INSERTs when inside collections (CASSANDRA-5795)
 +
 +
 +1.2.6
 + * Fix tracing when operation completes before all responses arrive 
 +   (CASSANDRA-5668)
 + * Fix cross-DC mutation forwarding (CASSANDRA-5632)
 + * Reduce SSTableLoader memory usage (CASSANDRA-)
 + * Scale hinted_handoff_throttle_in_kb to cluster size (CASSANDRA-5272)
 + * (Hadoop) Add CQL3 input/output formats (CASSANDRA-4421, 5622)
 + * (Hadoop) Fix InputKeyRange in CFIF (CASSANDRA-5536)
 + * Fix dealing with ridiculously large max sstable sizes in LCS 
(CASSANDRA-5589)
 + * Ignore pre-truncate hints (CASSANDRA-4655)
 + * Move

[07/16] git commit: merge from 1.2

2013-08-08 Thread jbellis 
merge from 1.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fdfdd5c1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fdfdd5c1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fdfdd5c1

Branch: refs/heads/trunk
Commit: fdfdd5c12ed9dcc85fa85a038b6149d06e26e1be
Parents: f30629b 1143df1
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:50:08 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:50:08 2013 -0500

--
 CHANGES.txt  |  4 
 src/java/org/apache/cassandra/db/Column.java | 18 +-
 2 files changed, 21 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/CHANGES.txt
--
diff --cc CHANGES.txt
index 86401d9,30ce976..756e45c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,5 +1,22 @@@
 -1.2.9
 - * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831)
 +2.0.0
 + * fix CAS contention timeout (CASSANDRA-5830)
 + * fix HsHa to respect max frame size (CASSANDRA-4573)
++Merged from 1.2:
++ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
 +
 +
 +2.0.0-rc1
 + * improve DecimalSerializer performance (CASSANDRA-5837)
 + * fix potential spurious wakeup in AsyncOneResponse (CASSANDRA-5690)
 + * fix schema-related trigger issues (CASSANDRA-5774)
 + * Better validation when accessing CQL3 table from thrift (CASSANDRA-5138)
 + * Fix assertion error during repair (CASSANDRA-5801)
 + * Fix range tombstone bug (CASSANDRA-5805)
 + * DC-local CAS (CASSANDRA-5797)
 + * Add a native_protocol_version column to the system.local table 
(CASSANRDA-5819)
 + * Use index_interval from cassandra.yaml when upgraded (CASSANDRA-5822)
 + * Fix buffer underflow on socket close (CASSANDRA-5792)
 +Merged from 1.2:
   * fix bulk-loading compressed sstables (CASSANDRA-5820)
   * (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter 
 (CASSANDRA-5824)
@@@ -26,6 -11,13 +28,8 @@@
 (CASSANDRA-5752)
   * add all time blocked to StatusLogger output (CASSANDRA-5825)
   * Future-proof inter-major-version schema migrations (CASSANDRA-5845)
+  * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
+(CASSANDRA-5718)
 -Merged from 1.1:
 - * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
 -
 -
 -1.2.8
   * Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814)
   * cqlsh: add collections support to COPY (CASSANDRA-5698)
   * retry important messages for any IOException (CASSANDRA-5804)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/src/java/org/apache/cassandra/db/Column.java
--
diff --cc src/java/org/apache/cassandra/db/Column.java
index 297b280,412808b..d9e0416
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@@ -24,19 -22,15 +24,20 @@@ import java.net.InetAddress
  import java.nio.ByteBuffer;
  import java.security.MessageDigest;
  import java.util.ArrayList;
 -import java.util.Collection;
 +import java.util.Iterator;
  import java.util.List;
  
 +import com.google.common.collect.AbstractIterator;
 +
  import org.apache.cassandra.config.CFMetaData;
+ import org.apache.cassandra.cql3.CFDefinition;
  import org.apache.cassandra.db.marshal.*;
 +import org.apache.cassandra.io.sstable.Descriptor;
  import org.apache.cassandra.io.util.DataOutputBuffer;
 +import org.apache.cassandra.serializers.MarshalException;
  import org.apache.cassandra.utils.Allocator;
  import org.apache.cassandra.utils.ByteBufferUtil;
 +import org.apache.cassandra.utils.FBUtilities;
  import org.apache.cassandra.utils.HeapAllocator;
  
  /**
@@@ -297,7 -288,23 +298,22 @@@ public class Column implements OnDiskAt
  public void validateFields(CFMetaData metadata) throws MarshalException
  {
  validateName(metadata);
- AbstractType? valueValidator = metadata.getValueValidator(name());
+ CFDefinition cfdef = metadata.getCfDef();
+ 
+ // If this is a CQL table, we need to pull out the CQL column name to 
look up the correct column type.
+ // (Note that COMPACT composites are handled by validateName, above.)
+ ByteBuffer internalName;
+ if (cfdef.isComposite  !cfdef.isCompact)
+ {
+ CompositeType comparator = (CompositeType) metadata.comparator;
 -ListAbstractCompositeType.CompositeComponent components = 
comparator.deconstruct(name);
 -internalName = components.get(components.size() - 1).value;
++internalName = comparator.extractLastComponent(name);
+ }
+ else
+

[16/16] git commit: Merge remote-tracking branch 'origin/trunk' into trunk

2013-08-08 Thread jbellis 
Merge remote-tracking branch 'origin/trunk' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a1212c59
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a1212c59
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a1212c59

Branch: refs/heads/trunk
Commit: a1212c597482ba91dc41036f23199273e9177040
Parents: 670979d 28b827a
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:52:15 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:52:15 2013 -0500

--
 .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
 .../org/apache/cassandra/service/StorageService.java  |  5 +
 2 files changed, 3 insertions(+), 16 deletions(-)
--

[08/16] git commit: merge from 1.2

2013-08-08 Thread jbellis 
merge from 1.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fdfdd5c1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fdfdd5c1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fdfdd5c1

Branch: refs/heads/cassandra-2.0.0
Commit: fdfdd5c12ed9dcc85fa85a038b6149d06e26e1be
Parents: f30629b 1143df1
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:50:08 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:50:08 2013 -0500

--
 CHANGES.txt  |  4 
 src/java/org/apache/cassandra/db/Column.java | 18 +-
 2 files changed, 21 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/CHANGES.txt
--
diff --cc CHANGES.txt
index 86401d9,30ce976..756e45c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,5 +1,22 @@@
 -1.2.9
 - * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831)
 +2.0.0
 + * fix CAS contention timeout (CASSANDRA-5830)
 + * fix HsHa to respect max frame size (CASSANDRA-4573)
++Merged from 1.2:
++ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
 +
 +
 +2.0.0-rc1
 + * improve DecimalSerializer performance (CASSANDRA-5837)
 + * fix potential spurious wakeup in AsyncOneResponse (CASSANDRA-5690)
 + * fix schema-related trigger issues (CASSANDRA-5774)
 + * Better validation when accessing CQL3 table from thrift (CASSANDRA-5138)
 + * Fix assertion error during repair (CASSANDRA-5801)
 + * Fix range tombstone bug (CASSANDRA-5805)
 + * DC-local CAS (CASSANDRA-5797)
 + * Add a native_protocol_version column to the system.local table 
(CASSANRDA-5819)
 + * Use index_interval from cassandra.yaml when upgraded (CASSANDRA-5822)
 + * Fix buffer underflow on socket close (CASSANDRA-5792)
 +Merged from 1.2:
   * fix bulk-loading compressed sstables (CASSANDRA-5820)
   * (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter 
 (CASSANDRA-5824)
@@@ -26,6 -11,13 +28,8 @@@
 (CASSANDRA-5752)
   * add all time blocked to StatusLogger output (CASSANDRA-5825)
   * Future-proof inter-major-version schema migrations (CASSANDRA-5845)
+  * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
+(CASSANDRA-5718)
 -Merged from 1.1:
 - * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
 -
 -
 -1.2.8
   * Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814)
   * cqlsh: add collections support to COPY (CASSANDRA-5698)
   * retry important messages for any IOException (CASSANDRA-5804)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/src/java/org/apache/cassandra/db/Column.java
--
diff --cc src/java/org/apache/cassandra/db/Column.java
index 297b280,412808b..d9e0416
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@@ -24,19 -22,15 +24,20 @@@ import java.net.InetAddress
  import java.nio.ByteBuffer;
  import java.security.MessageDigest;
  import java.util.ArrayList;
 -import java.util.Collection;
 +import java.util.Iterator;
  import java.util.List;
  
 +import com.google.common.collect.AbstractIterator;
 +
  import org.apache.cassandra.config.CFMetaData;
+ import org.apache.cassandra.cql3.CFDefinition;
  import org.apache.cassandra.db.marshal.*;
 +import org.apache.cassandra.io.sstable.Descriptor;
  import org.apache.cassandra.io.util.DataOutputBuffer;
 +import org.apache.cassandra.serializers.MarshalException;
  import org.apache.cassandra.utils.Allocator;
  import org.apache.cassandra.utils.ByteBufferUtil;
 +import org.apache.cassandra.utils.FBUtilities;
  import org.apache.cassandra.utils.HeapAllocator;
  
  /**
@@@ -297,7 -288,23 +298,22 @@@ public class Column implements OnDiskAt
  public void validateFields(CFMetaData metadata) throws MarshalException
  {
  validateName(metadata);
- AbstractType? valueValidator = metadata.getValueValidator(name());
+ CFDefinition cfdef = metadata.getCfDef();
+ 
+ // If this is a CQL table, we need to pull out the CQL column name to 
look up the correct column type.
+ // (Note that COMPACT composites are handled by validateName, above.)
+ ByteBuffer internalName;
+ if (cfdef.isComposite  !cfdef.isCompact)
+ {
+ CompositeType comparator = (CompositeType) metadata.comparator;
 -ListAbstractCompositeType.CompositeComponent components = 
comparator.deconstruct(name);
 -internalName = components.get(components.size() - 1).value;
++internalName = comparator.extractLastComponent(name);
+ }
+

[01/16] git commit: Correctly validate sparse composite cells patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855

2013-08-08 Thread jbellis 
Updated Branches:
  refs/heads/cassandra-1.2 ccb32a8c8 - 9e0efa3db
  refs/heads/cassandra-2.0 5d9a33a46 - e911b767e
  refs/heads/cassandra-2.0.0 7a8cbb3ea - 5411e5ff8
  refs/heads/trunk 28b827a85 - a1212c597


Correctly validate sparse composite cells
patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb884a58
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb884a58
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb884a58

Branch: refs/heads/cassandra-2.0
Commit: eb884a582b709d67fdd075aedf19facf6803bf93
Parents: 238139c
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:33:39 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:43:43 2013 -0500

--
 CHANGES.txt  |  1 +
 src/java/org/apache/cassandra/db/Column.java | 19 ++-
 2 files changed, 19 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 75b349f..8790518 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
 1.1.next
  * Backport compaction exception handling from 1.2
+ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
 
 
 1.1.12

http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb884a58/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java 
b/src/java/org/apache/cassandra/db/Column.java
index 97aece0..ea83146 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -29,6 +29,7 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CFDefinition;
 import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.io.util.DataOutputBuffer;
 import org.apache.cassandra.utils.Allocator;
@@ -277,7 +278,23 @@ public class Column implements IColumn
 public void validateFields(CFMetaData metadata) throws MarshalException
 {
 validateName(metadata);
-AbstractType? valueValidator = metadata.getValueValidator(name());
+CFDefinition cfdef = metadata.getCfDef();
+
+// If this is a CQL table, we need to pull out the CQL column name to 
look up the correct column type.
+// (Note that COMPACT composites are handled by validateName, above.)
+ByteBuffer internalName;
+if (cfdef.isComposite  !cfdef.isCompact)
+{
+AbstractCompositeType comparator = (AbstractCompositeType) 
metadata.comparator;
+ListAbstractCompositeType.CompositeComponent components = 
comparator.deconstruct(name);
+internalName = components.get(components.size() - 1).value;
+}
+else
+{
+internalName = name;
+}
+
+AbstractType? valueValidator = 
metadata.getValueValidator(internalName);
 if (valueValidator != null)
 valueValidator.validate(value());
 }

[06/16] git commit: merge from 1.1

2013-08-08 Thread jbellis 
merge from 1.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1143df1a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1143df1a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1143df1a

Branch: refs/heads/cassandra-2.0.0
Commit: 1143df1aa439618bff6005ca1cee34cb066a
Parents: 23d7bd8 eb884a5
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:46:52 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:46:52 2013 -0500

--
 CHANGES.txt  |  2 ++
 src/java/org/apache/cassandra/db/Column.java | 19 ++-
 2 files changed, 20 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1143df1a/CHANGES.txt
--
diff --cc CHANGES.txt
index 859e7f9,8790518..30ce976
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,214 -1,19 +1,216 @@@
 -1.1.next
 - * Backport compaction exception handling from 1.2
 +1.2.9
 + * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831)
 + * fix bulk-loading compressed sstables (CASSANDRA-5820)
 + * (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter 
 +   (CASSANDRA-5824)
 + * update default LCS sstable size to 160MB (CASSANDRA-5727)
 + * Allow compacting 2Is via nodetool (CASSANDRA-5670)
 + * Hex-encode non-String keys in OPP (CASSANDRA-5793)
 + * nodetool history logging (CASSANDRA-5823)
 + * (Hadoop) fix support for Thrift tables in CqlPagingRecordReader 
 +   (CASSANDRA-5752)
 + * add all time blocked to StatusLogger output (CASSANDRA-5825)
 + * Future-proof inter-major-version schema migrations (CASSANDRA-5845)
 + * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
 +   (CASSANDRA-5718)
++Merged from 1.1:
+  * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
  
  
 -1.1.12
 +1.2.8
 + * Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814)
 + * cqlsh: add collections support to COPY (CASSANDRA-5698)
 + * retry important messages for any IOException (CASSANDRA-5804)
 + * Allow empty IN relations in SELECT/UPDATE/DELETE statements 
(CASSANDRA-5626)
 + * cqlsh: fix crashing on Windows due to libedit detection (CASSANDRA-5812)
 +
 +
 +1.2.7
 + * if no seeds can be a reached a node won't start in a ring by itself 
(CASSANDRA-5768)
 + * add cassandra.unsafesystem property (CASSANDRA-5704)
 + * (Hadoop) quote identifiers in CqlPagingRecordReader (CASSANDRA-5763)
 + * Add replace_node functionality for vnodes (CASSANDRA-5337)
 + * Add timeout events to query traces (CASSANDRA-5520)
 + * make starting native protocol server idempotent (CASSANDRA-5728)
 + * Fix loading key cache when a saved entry is no longer valid 
(CASSANDRA-5706)
 + * Fix serialization of the LEFT gossip value (CASSANDRA-5696)
 + * Pig: support for cql3 tables (CASSANDRA-5234)
 + * cqlsh: Don't show 'null' in place of empty values (CASSANDRA-5675)
 + * Race condition in detecting version on a mixed 1.1/1.2 cluster
 +   (CASSANDRA-5692)
 + * Fix skipping range tombstones with reverse queries (CASSANDRA-5712)
 + * Expire entries out of ThriftSessionManager (CASSANRDA-5719)
 + * Don't keep ancestor information in memory (CASSANDRA-5342)
 + * cqlsh: fix handling of semicolons inside BATCH queries (CASSANDRA-5697)
 + * Expose native protocol server status in nodetool info (CASSANDRA-5735)
 + * Fix pathetic performance of range tombstones (CASSANDRA-5677)
 + * Fix querying with an empty (impossible) range (CASSANDRA-5573)
 + * cqlsh: handle CUSTOM 2i in DESCRIBE output (CASSANDRA-5760)
 + * Fix minor bug in Range.intersects(Bound) (CASSANDRA-5771)
 + * cqlsh: handle disabled compression in DESCRIBE output (CASSANDRA-5766)
 + * Ensure all UP events are notified on the native protocol (CASSANDRA-5769)
 + * Fix formatting of sstable2json with multiple -k arguments (CASSANDRA-5781)
 + * Don't rely on row marker for queries in general to hide lost markers
 +   after TTL expires (CASSANDRA-5762)
 + * Sort nodetool help output (CASSANDRA-5776)
 + * Fix column expiring during 2 phases compaction (CASSANDRA-5799)
 + * now() is being rejected in INSERTs when inside collections (CASSANDRA-5795)
 +
 +
 +1.2.6
 + * Fix tracing when operation completes before all responses arrive 
 +   (CASSANDRA-5668)
 + * Fix cross-DC mutation forwarding (CASSANDRA-5632)
 + * Reduce SSTableLoader memory usage (CASSANDRA-)
 + * Scale hinted_handoff_throttle_in_kb to cluster size (CASSANDRA-5272)
 + * (Hadoop) Add CQL3 input/output formats (CASSANDRA-4421, 5622)
 + * (Hadoop) Fix InputKeyRange in CFIF (CASSANDRA-5536)
 + * Fix dealing with ridiculously large max sstable sizes in LCS 
(CASSANDRA-5589)
 + * Ignore pre-truncate hints (CASSANDRA-4655)
 + * Move

[11/16] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0

2013-08-08 Thread jbellis 
Merge branch 'cassandra-2.0.0' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b5f445b4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b5f445b4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b5f445b4

Branch: refs/heads/cassandra-2.0
Commit: b5f445b448e6047e717b17985c117843e66a2d40
Parents: c526131 fdfdd5c
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:50:16 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:50:16 2013 -0500

--
 CHANGES.txt  |  4 
 src/java/org/apache/cassandra/db/Column.java | 18 +-
 2 files changed, 21 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5f445b4/CHANGES.txt
--

[10/16] git commit: Merge branch 'cassandra-2.0.0' into cassandra-2.0

2013-08-08 Thread jbellis 
Merge branch 'cassandra-2.0.0' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b5f445b4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b5f445b4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b5f445b4

Branch: refs/heads/trunk
Commit: b5f445b448e6047e717b17985c117843e66a2d40
Parents: c526131 fdfdd5c
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:50:16 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:50:16 2013 -0500

--
 CHANGES.txt  |  4 
 src/java/org/apache/cassandra/db/Column.java | 18 +-
 2 files changed, 21 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5f445b4/CHANGES.txt
--

[09/16] git commit: merge from 1.2

2013-08-08 Thread jbellis 
merge from 1.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fdfdd5c1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fdfdd5c1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fdfdd5c1

Branch: refs/heads/cassandra-2.0
Commit: fdfdd5c12ed9dcc85fa85a038b6149d06e26e1be
Parents: f30629b 1143df1
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:50:08 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:50:08 2013 -0500

--
 CHANGES.txt  |  4 
 src/java/org/apache/cassandra/db/Column.java | 18 +-
 2 files changed, 21 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/CHANGES.txt
--
diff --cc CHANGES.txt
index 86401d9,30ce976..756e45c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,20 -1,5 +1,22 @@@
 -1.2.9
 - * add --migrate option to sstableupgrade and sstablescrub (CASSANDRA-5831)
 +2.0.0
 + * fix CAS contention timeout (CASSANDRA-5830)
 + * fix HsHa to respect max frame size (CASSANDRA-4573)
++Merged from 1.2:
++ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
 +
 +
 +2.0.0-rc1
 + * improve DecimalSerializer performance (CASSANDRA-5837)
 + * fix potential spurious wakeup in AsyncOneResponse (CASSANDRA-5690)
 + * fix schema-related trigger issues (CASSANDRA-5774)
 + * Better validation when accessing CQL3 table from thrift (CASSANDRA-5138)
 + * Fix assertion error during repair (CASSANDRA-5801)
 + * Fix range tombstone bug (CASSANDRA-5805)
 + * DC-local CAS (CASSANDRA-5797)
 + * Add a native_protocol_version column to the system.local table 
(CASSANRDA-5819)
 + * Use index_interval from cassandra.yaml when upgraded (CASSANDRA-5822)
 + * Fix buffer underflow on socket close (CASSANDRA-5792)
 +Merged from 1.2:
   * fix bulk-loading compressed sstables (CASSANDRA-5820)
   * (Hadoop) fix quoting in CqlPagingRecordReader and CqlRecordWriter 
 (CASSANDRA-5824)
@@@ -26,6 -11,13 +28,8 @@@
 (CASSANDRA-5752)
   * add all time blocked to StatusLogger output (CASSANDRA-5825)
   * Future-proof inter-major-version schema migrations (CASSANDRA-5845)
+  * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
+(CASSANDRA-5718)
 -Merged from 1.1:
 - * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
 -
 -
 -1.2.8
   * Fix reading DeletionTime from 1.1-format sstables (CASSANDRA-5814)
   * cqlsh: add collections support to COPY (CASSANDRA-5698)
   * retry important messages for any IOException (CASSANDRA-5804)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/fdfdd5c1/src/java/org/apache/cassandra/db/Column.java
--
diff --cc src/java/org/apache/cassandra/db/Column.java
index 297b280,412808b..d9e0416
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@@ -24,19 -22,15 +24,20 @@@ import java.net.InetAddress
  import java.nio.ByteBuffer;
  import java.security.MessageDigest;
  import java.util.ArrayList;
 -import java.util.Collection;
 +import java.util.Iterator;
  import java.util.List;
  
 +import com.google.common.collect.AbstractIterator;
 +
  import org.apache.cassandra.config.CFMetaData;
+ import org.apache.cassandra.cql3.CFDefinition;
  import org.apache.cassandra.db.marshal.*;
 +import org.apache.cassandra.io.sstable.Descriptor;
  import org.apache.cassandra.io.util.DataOutputBuffer;
 +import org.apache.cassandra.serializers.MarshalException;
  import org.apache.cassandra.utils.Allocator;
  import org.apache.cassandra.utils.ByteBufferUtil;
 +import org.apache.cassandra.utils.FBUtilities;
  import org.apache.cassandra.utils.HeapAllocator;
  
  /**
@@@ -297,7 -288,23 +298,22 @@@ public class Column implements OnDiskAt
  public void validateFields(CFMetaData metadata) throws MarshalException
  {
  validateName(metadata);
- AbstractType? valueValidator = metadata.getValueValidator(name());
+ CFDefinition cfdef = metadata.getCfDef();
+ 
+ // If this is a CQL table, we need to pull out the CQL column name to 
look up the correct column type.
+ // (Note that COMPACT composites are handled by validateName, above.)
+ ByteBuffer internalName;
+ if (cfdef.isComposite  !cfdef.isCompact)
+ {
+ CompositeType comparator = (CompositeType) metadata.comparator;
 -ListAbstractCompositeType.CompositeComponent components = 
comparator.deconstruct(name);
 -internalName = components.get(components.size() - 1).value;
++internalName = comparator.extractLastComponent(name);
+ }
+

[13/16] git commit: Correctly validate sparse composite cells patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855

2013-08-08 Thread jbellis 
Correctly validate sparse composite cells
patch by Tyler Hobbs; reviewed by jbellis for CASSANDRA-5855


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9e0efa3d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9e0efa3d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9e0efa3d

Branch: refs/heads/cassandra-1.2
Commit: 9e0efa3dbbeddf382c8e0f45cfdf2ebb91bc9db0
Parents: ccb32a8
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:33:39 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:51:34 2013 -0500

--
 CHANGES.txt  |  2 ++
 src/java/org/apache/cassandra/db/Column.java | 19 ++-
 2 files changed, 20 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e0efa3d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 859e7f9..30ce976 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -13,6 +13,8 @@
  * Future-proof inter-major-version schema migrations (CASSANDRA-5845)
  * (Hadoop) add CqlPagingRecordReader support for ReversedType in Thrift table
(CASSANDRA-5718)
+Merged from 1.1:
+ * Correctly validate sparse composite cells in scrub (CASSANDRA-5855)
 
 
 1.2.8

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e0efa3d/src/java/org/apache/cassandra/db/Column.java
--
diff --git a/src/java/org/apache/cassandra/db/Column.java 
b/src/java/org/apache/cassandra/db/Column.java
index 616f3c0..7b614f5 100644
--- a/src/java/org/apache/cassandra/db/Column.java
+++ b/src/java/org/apache/cassandra/db/Column.java
@@ -26,6 +26,7 @@ import java.util.Collection;
 import java.util.List;
 
 import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CFDefinition;
 import org.apache.cassandra.db.marshal.*;
 import org.apache.cassandra.io.util.DataOutputBuffer;
 import org.apache.cassandra.utils.Allocator;
@@ -287,7 +288,23 @@ public class Column implements IColumn
 public void validateFields(CFMetaData metadata) throws MarshalException
 {
 validateName(metadata);
-AbstractType? valueValidator = metadata.getValueValidator(name());
+CFDefinition cfdef = metadata.getCfDef();
+
+// If this is a CQL table, we need to pull out the CQL column name to 
look up the correct column type.
+// (Note that COMPACT composites are handled by validateName, above.)
+ByteBuffer internalName;
+if (cfdef.isComposite  !cfdef.isCompact)
+{
+AbstractCompositeType comparator = (AbstractCompositeType) 
metadata.comparator;
+ListAbstractCompositeType.CompositeComponent components = 
comparator.deconstruct(name);
+internalName = components.get(components.size() - 1).value;
+}
+else
+{
+internalName = name;
+}
+
+AbstractType? valueValidator = 
metadata.getValueValidator(internalName);
 if (valueValidator != null)
 valueValidator.validate(value());
 }

[15/16] git commit: Merge remote-tracking branch 'origin/cassandra-2.0' into cassandra-2.0

2013-08-08 Thread jbellis 
Merge remote-tracking branch 'origin/cassandra-2.0' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e911b767
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e911b767
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e911b767

Branch: refs/heads/cassandra-2.0
Commit: e911b767e1ca90acbb4be65475a239b4a663308f
Parents: b5f445b 5d9a33a
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Aug 8 15:52:07 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Aug 8 15:52:07 2013 -0500

--
 .../org/apache/cassandra/service/CassandraDaemon.java | 14 ++
 .../org/apache/cassandra/service/StorageService.java  |  5 +
 2 files changed, 3 insertions(+), 16 deletions(-)
--

  1   2   >