[jira] [Comment Edited] (CASSANDRA-6018) Add option to encrypt commitlog

2015-08-23 Thread Jason Brown (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708426#comment-14708426
 ] 

Jason Brown edited comment on CASSANDRA-6018 at 8/23/15 4:26 PM:
-

bq. the way the header parameters are used by the descriptor

You are correct - the only thing I'm using the headerParameters field for is to 
stash the IV, so that we can get it at read-time (when replaying the commit 
log). I was hoping to keep EncyptionContext/TransparentDataEncryptionOptions a 
bit more general, and not stuff file-specific details in those classes, but 
perhaps somewhere in those classes is the best for the IV to go, after all. 
I'll think about that some more.

bq. Since you are compressing to ON_HEAP, I think it will help to use the same 
buffer type for the input as well

Will do. As a side note, I originally wrote this code using the byte array APIs 
of Cipher, so thus everything needed to be on-heap. After I discovered the 
ByteBuffer versions of the Cipher methods, I switched to those and honestly 
didn't think about the on/off heap question again. I can give a shot to trying 
everything off-heap vs. on-heap, but not sure there's a lot of gain in the 
off-heap model as, after we compress to off-heap, we would need to pull those 
bytes back into local process memory to encrypt. Any opinions, [~benedict]?

bq. {{doFinal}} is copy-safe

Huh, somehow I've always been using two buffers (separate in/out) with 
encryption operations with Cipher, and never saw that clause in the API :). 
I'll give that a shot and see what happens - thanks for pointing it out!

Will fix the other items, as well, in next update.


was (Author: jasobrown):
bq. the way the header parameters are used by the descriptor

You are correct - the only thing I'm using the headerParameters field for is to 
stash the IV, so that we can get it at read-time (when replaying the commit 
log). I was hoping to keep EncyptionContext/TransparentDataEncryptionOptions a 
bit more general, and not stuff file-specific details in those classes, but 
perhaps somewhere in those classes is the best for the IV to go, after all. 
I'll think about that some more.

bq. Since you are compressing to ON_HEAP, I think it will help to use the same 
buffer type for the input as well

Will do. As a side note, I originally wrote this code using the byte array APIs 
of Cipher, so thus everything needed to be on-heap. After I discovered the 
ByteBuffer versions of the Cipher methods, I switched to those and honestly 
didn't think about the on/off heap question again. I can give a shot to trying 
everything off-heap vs. on-heap, but not sure there's a lot of gain in the 
off-heap model as, after we compress to off-heap, we would need to pull those 
bytes back into local process memory to encrypt. Any opinions, [~benedict]?

bq. {{doFinal}} is copy-safe

Huh, somehow I've always been using two buffers (separate in/out) with 
encryption operations, and never saw that clause in the API :). I'll give that 
a shot and see what happens - thanks for pointing it out!

Will fix the other items, as well, in next update.

 Add option to encrypt commitlog 
 

 Key: CASSANDRA-6018
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6018
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jason Brown
Assignee: Jason Brown
  Labels: commit_log, encryption, security
 Fix For: 3.x


 We are going to start using cassandra for a billing system, and while I can 
 encrypt sstables at rest (via Datastax Enterprise), commit logs are more or 
 less plain text. Thus, an attacker would be able to easily read, for example, 
 credit card numbers in the clear text commit log (if the calling app does not 
 encrypt the data itself before sending it to cassandra).
 I want to allow the option of encrypting the commit logs, most likely 
 controlled by a property in the yaml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6018) Add option to encrypt commitlog

2015-08-23 Thread Jason Brown (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708426#comment-14708426
 ] 

Jason Brown commented on CASSANDRA-6018:


bq. the way the header parameters are used by the descriptor

You are correct - the only thing I'm using the headerParameters field for is to 
stash the IV, so that we can get it at read-time (when replaying the commit 
log). I was hoping to keep EncyptionContext/TransparentDataEncryptionOptions a 
bit more general, and not stuff file-specific details in those classes, but 
perhaps somewhere in those classes is the best for the IV to go, after all. 
I'll think about that some more.

bq. Since you are compressing to ON_HEAP, I think it will help to use the same 
buffer type for the input as well

Will do. As a side note, I originally wrote this code using the byte array APIs 
of Cipher, so thus everything needed to be on-heap. After I discovered the 
ByteBuffer versions of the Cipher methods, I switched to those and honestly 
didn't think about the on/off heap question again. I can give a shot to trying 
everything off-heap vs. on-heap, but not sure there's a lot of gain in the 
off-heap model as, after we compress to off-heap, we would need to pull those 
bytes back into local process memory to encrypt. Any opinions, [~benedict]?

bq. {{doFinal}} is copy-safe

Huh, somehow I've always been using two buffers (separate in/out) with 
encryption operations, and never saw that clause in the API :). I'll give that 
a shot and see what happens - thanks for pointing it out!

Will fix the other items, as well, in next update.

 Add option to encrypt commitlog 
 

 Key: CASSANDRA-6018
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6018
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jason Brown
Assignee: Jason Brown
  Labels: commit_log, encryption, security
 Fix For: 3.x


 We are going to start using cassandra for a billing system, and while I can 
 encrypt sstables at rest (via Datastax Enterprise), commit logs are more or 
 less plain text. Thus, an attacker would be able to easily read, for example, 
 credit card numbers in the clear text commit log (if the calling app does not 
 encrypt the data itself before sending it to cassandra).
 I want to allow the option of encrypting the commit logs, most likely 
 controlled by a property in the yaml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9738) Migrate key-cache to be fully off-heap

2015-08-23 Thread Robert Stupp (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708431#comment-14708431
 ] 

Robert Stupp commented on CASSANDRA-9738:
-

Branch is ready for another review round.
It's rebased against current 3.0 and trunk.

Changes are:
* updated to ohc 0.4.2 (to get rid of {{ByteBuffer.order()}} calls and the 
try-finally-clauses)
* no more duplicate serialization effort in {{serializedSize()}} + 
{{serialize()}} - uses a thread-local serialization buffer. Can be further 
improved with CASSANDRA-9929.
* added/moved string serialization methods to {{ByteBufferUtil}}, which has 
distinct code paths for 7-bit ASCII and strings with wider chars


 Migrate key-cache to be fully off-heap
 --

 Key: CASSANDRA-9738
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9738
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Robert Stupp
Assignee: Robert Stupp
 Fix For: 3.0 beta 2


 Key cache still uses a concurrent map on-heap. This could go to off-heap and 
 feels doable now after CASSANDRA-8099.
 Evaluation should be done in advance based on a POC to prove that pure 
 off-heap counter cache buys a performance and/or gc-pressure improvement.
 In theory, elimination of on-heap management of the map should buy us some 
 benefit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10061) Only use batchlog when paired view replica is remote

2015-08-23 Thread Aleksey Yeschenko (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708484#comment-14708484
 ] 

Aleksey Yeschenko commented on CASSANDRA-10061:
---

What happens if a node dies between the apply() calls? We'll still recover all 
right?

 Only use batchlog when paired view replica is remote
 

 Key: CASSANDRA-10061
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10061
 Project: Cassandra
  Issue Type: Improvement
Reporter: T Jake Luciani
Assignee: T Jake Luciani
 Fix For: 3.0 beta 2


 As described in the MV design doc the base and view replicas are paired one 
 to one.
 If the replica selected for the view is the local node itself there is no 
 need to create a local batchlog and we can simply apply the view mutations 
 locally.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9973) java.lang.IllegalStateException: Unable to compute when histogram overflowed

2015-08-23 Thread Mark Manley (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708448#comment-14708448
 ] 

Mark Manley commented on CASSANDRA-9973:


Do we have an ETA for the release date of 2.2.1?  My ring was again crippled 
this morning when several of my nodes spewed out hundreds of these errors a 
minute.  It corresponds to the time that they stopped answering requests 
reliably.  If there is a workaround for 2.2.0, I am all ears.

Thanks!

 java.lang.IllegalStateException: Unable to compute when histogram overflowed
 

 Key: CASSANDRA-9973
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9973
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Mark Manley
Assignee: T Jake Luciani
 Fix For: 2.2.1

 Attachments: 9973.txt


 I recently, and probably mistakenly, upgraded one of my production C* 
 clusters to 2.2.0.  I am seeing these errors in the logs, followed by an 
 intense period of garbage collection until the node, then the ring, becomes 
 crippled:
 {code}
 ERROR [OptionalTasks:1] 2015-08-04 03:24:56,057 CassandraDaemon.java:182 - 
 Exception in thread Thread[OptionalTasks:1,5,main]
 java.lang.IllegalStateException: Unable to compute when histogram overflowed
 at 
 org.apache.cassandra.utils.EstimatedHistogram.percentile(EstimatedHistogram.java:179)
  ~[apache-cassandra-2.2.0.jar:2.2.0]
 at 
 org.apache.cassandra.metrics.EstimatedHistogramReservoir$HistogramSnapshot.getValue(EstimatedHistogramReservoir.java:84)
  ~[apache-cassandra-2.2.0.jar:2.2.0]
 at 
 org.apache.cassandra.db.ColumnFamilyStore$3.run(ColumnFamilyStore.java:405) 
 ~[apache-cassandra-2.2.0.jar:2.2.0]
 at 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
  ~[apache-cassandra-2.2.0.jar:2.2.0]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
 [na:1.8.0_45]
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
 [na:1.8.0_45]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
  [na:1.8.0_45]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
  [na:1.8.0_45]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  [na:1.8.0_45]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  [na:1.8.0_45]
 at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
 {code}
 I am not sure if the GC instability is this or something else, but I though 
 this histogram overflow issue was fixed in 2.1.3?  Anyway, reporting now as a 
 possible regression.  Please let me know what I can provide in terms of 
 information to help with this.  Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10157) SchemaKeyspace.addAggregateToSchemaMutation keeps previous on CREATE OR REPLACE AGGREGATE

2015-08-23 Thread Robert Stupp (JIRA) 
Robert Stupp created CASSANDRA-10157:


 Summary: SchemaKeyspace.addAggregateToSchemaMutation keeps 
previous on CREATE OR REPLACE AGGREGATE
 Key: CASSANDRA-10157
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10157
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
 Fix For: 3.0 beta 2






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9673) Improve batchlog write path

2015-08-23 Thread Aleksey Yeschenko (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708398#comment-14708398
 ] 

Aleksey Yeschenko commented on CASSANDRA-9673:
--

Rebased against most recent trunk and made some minor changes, here: 
https://github.com/iamaleksey/cassandra/tree/9673-3.0. Didn't go very far. 
Removed {{BATCHLOG_MUTATION}} Stage that Carl and I think is no longer needed 
(but we'll bring it back if that's a mistake), and did some renames and minor 
cleanup of unused methods.

Good work. There is only one real issue so far, with that fixed, we could 
commit as is, but I have a bunch of nits that'd be nice to address: we are not 
properly handling old-format batchlog mutations sent live from 2.1/2.2 nodes, 
during live upgrades.

You need to modify {{MutationVerbHandler}} and add special treatment to 
mutations for the old batchlog table.

Nits:
1. I would prefer {{BatchStoreMessage}} and {{BatchRemoveMessage}} to be dumb 
and only be concerned about serialization/deserialization. This means no 
{{getMutation()}} in either. For {{BatchRemoveMessage}}, the actual logic 
should be in {{BatchRemoveVerbHandler}}.
2. I would like to separate {{Batch}} and {{BatchStoreMessage}} and put actual 
saving to batchlog logic into a new {{Batch}} class - to be used both for local 
(MV) writes and proper BL remote writes.
3. If moved to {{Batch}} (and if not, too), I would prefer the mutation 
collections in them to only be set in the constructor.
4. Somehow we are calling {{LegacyBatchMigrator.convertBatchEntries()}} on 
every replay, reverting Branimir's changes, instead of explicitly only doing it 
once, upon first replay. I'm presuming that this is done as a workaround for 
the live upgrade issue. I would prefer that we moved the migration to 
{{CassandraDaemon.setup()}}, where legacy schema and hints migration already 
get called.

 Improve batchlog write path
 ---

 Key: CASSANDRA-9673
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9673
 Project: Cassandra
  Issue Type: Improvement
Reporter: Aleksey Yeschenko
Assignee: Stefania
  Labels: performance
 Fix For: 3.0 beta 2

 Attachments: 9673_001.tar.gz, 9673_004.tar.gz, 
 gc_times_first_node_patched_004.png, gc_times_first_node_trunk_004.png


 Currently we allocate an on-heap {{ByteBuffer}} to serialize the batched 
 mutations into, before sending it to a distant node, generating unnecessary 
 garbage (potentially a lot of it).
 With materialized views using the batchlog, it would be nice to optimise the 
 write path:
 - introduce a new verb ({{Batch}})
 - introduce a new message ({{BatchMessage}}) that would encapsulate the 
 mutations, expiration, and creation time (similar to {{HintMessage}} in 
 CASSANDRA-6230)
 - have MS serialize it directly instead of relying on an intermediate buffer
 To avoid merely shifting the temp buffer to the receiving side(s) we should 
 change the structure of the batchlog table to use a list or a map of 
 individual mutations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/6] cassandra git commit: 10151

2015-08-23 Thread benedict 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 a6d654044 - 941a5dd8a
  refs/heads/cassandra-3.0 342e33ffb - 29d3d44d2
  refs/heads/trunk b1fb6b939 - 62f20789b


10151


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/941a5dd8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/941a5dd8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/941a5dd8

Branch: refs/heads/cassandra-2.2
Commit: 941a5dd8a1ca9b759e2091ca151bb339bc947c58
Parents: a6d6540
Author: Benedict Elliott Smith bened...@apache.org
Authored: Fri Aug 21 10:50:36 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:50:53 2015 +0100

--
 test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/941a5dd8/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java 
b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
index dbd5287..93b88b4 100644
--- a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
+++ b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
@@ -82,7 +82,7 @@ public class TrackerTest
 {
 ColumnFamilyStore cfs = MockSchema.newCFS();
 Tracker tracker = new Tracker(cfs, false);
-ListSSTableReader readers = ImmutableList.of(MockSchema.sstable(0, 
cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
+ListSSTableReader readers = ImmutableList.of(MockSchema.sstable(0, 
true, cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
 tracker.addInitialSSTables(copyOf(readers));
 
Assert.assertNull(tracker.tryModify(ImmutableList.of(MockSchema.sstable(0, 
cfs)), OperationType.COMPACTION));
 try (LifecycleTransaction txn = tracker.tryModify(readers.get(0), 
OperationType.COMPACTION);)
@@ -97,6 +97,7 @@ public class TrackerTest
 Assert.assertNotNull(txn);
 Assert.assertEquals(0, txn.originals().size());
 }
+readers.get(0).selfRef().release();
 }
 
 @Test

[3/6] cassandra git commit: 10151

2015-08-23 Thread benedict 
10151


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/941a5dd8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/941a5dd8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/941a5dd8

Branch: refs/heads/trunk
Commit: 941a5dd8a1ca9b759e2091ca151bb339bc947c58
Parents: a6d6540
Author: Benedict Elliott Smith bened...@apache.org
Authored: Fri Aug 21 10:50:36 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:50:53 2015 +0100

--
 test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/941a5dd8/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java 
b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
index dbd5287..93b88b4 100644
--- a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
+++ b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
@@ -82,7 +82,7 @@ public class TrackerTest
 {
 ColumnFamilyStore cfs = MockSchema.newCFS();
 Tracker tracker = new Tracker(cfs, false);
-ListSSTableReader readers = ImmutableList.of(MockSchema.sstable(0, 
cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
+ListSSTableReader readers = ImmutableList.of(MockSchema.sstable(0, 
true, cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
 tracker.addInitialSSTables(copyOf(readers));
 
Assert.assertNull(tracker.tryModify(ImmutableList.of(MockSchema.sstable(0, 
cfs)), OperationType.COMPACTION));
 try (LifecycleTransaction txn = tracker.tryModify(readers.get(0), 
OperationType.COMPACTION);)
@@ -97,6 +97,7 @@ public class TrackerTest
 Assert.assertNotNull(txn);
 Assert.assertEquals(0, txn.originals().size());
 }
+readers.get(0).selfRef().release();
 }
 
 @Test

[6/6] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

2015-08-23 Thread benedict 
Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/62f20789
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/62f20789
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/62f20789

Branch: refs/heads/trunk
Commit: 62f20789b14b516e29790365a857087644e494d9
Parents: b1fb6b9 29d3d44
Author: Benedict Elliott Smith bened...@apache.org
Authored: Sun Aug 23 11:51:17 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:51:17 2015 +0100

--
 test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--

[4/6] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2015-08-23 Thread benedict 
Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/29d3d44d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/29d3d44d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/29d3d44d

Branch: refs/heads/trunk
Commit: 29d3d44d2056e5ab84a2a5357cd782344683fc8e
Parents: 342e33f 941a5dd
Author: Benedict Elliott Smith bened...@apache.org
Authored: Sun Aug 23 11:51:10 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:51:10 2015 +0100

--
 test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/29d3d44d/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
--

[2/6] cassandra git commit: 10151

2015-08-23 Thread benedict 
10151


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/941a5dd8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/941a5dd8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/941a5dd8

Branch: refs/heads/cassandra-3.0
Commit: 941a5dd8a1ca9b759e2091ca151bb339bc947c58
Parents: a6d6540
Author: Benedict Elliott Smith bened...@apache.org
Authored: Fri Aug 21 10:50:36 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:50:53 2015 +0100

--
 test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/941a5dd8/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java 
b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
index dbd5287..93b88b4 100644
--- a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
+++ b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
@@ -82,7 +82,7 @@ public class TrackerTest
 {
 ColumnFamilyStore cfs = MockSchema.newCFS();
 Tracker tracker = new Tracker(cfs, false);
-ListSSTableReader readers = ImmutableList.of(MockSchema.sstable(0, 
cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
+ListSSTableReader readers = ImmutableList.of(MockSchema.sstable(0, 
true, cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
 tracker.addInitialSSTables(copyOf(readers));
 
Assert.assertNull(tracker.tryModify(ImmutableList.of(MockSchema.sstable(0, 
cfs)), OperationType.COMPACTION));
 try (LifecycleTransaction txn = tracker.tryModify(readers.get(0), 
OperationType.COMPACTION);)
@@ -97,6 +97,7 @@ public class TrackerTest
 Assert.assertNotNull(txn);
 Assert.assertEquals(0, txn.originals().size());
 }
+readers.get(0).selfRef().release();
 }
 
 @Test

[5/6] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2015-08-23 Thread benedict 
Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/29d3d44d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/29d3d44d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/29d3d44d

Branch: refs/heads/cassandra-3.0
Commit: 29d3d44d2056e5ab84a2a5357cd782344683fc8e
Parents: 342e33f 941a5dd
Author: Benedict Elliott Smith bened...@apache.org
Authored: Sun Aug 23 11:51:10 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:51:10 2015 +0100

--
 test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/29d3d44d/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
--

[03/10] cassandra git commit: Support BigDecimal in cassanra-stress

2015-08-23 Thread benedict 
Support BigDecimal in cassanra-stress

patch by Sebastian Estevez; reviewed by Benedict for CASSANDRA-10048


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bee617f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bee617f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bee617f

Branch: refs/heads/cassandra-3.0
Commit: 5bee617fea68e7b745c18d14b9b7076ab6fe219b
Parents: df9e798
Author: phact sdrumme...@gmail.com
Authored: Fri Aug 21 09:50:43 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:27:16 2015 +0100

--
 .../apache/cassandra/stress/StressProfile.java  |  3 +-
 .../stress/generate/values/BigDecimals.java | 39 
 2 files changed, 41 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java 
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 49c4682..192b535 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -502,6 +502,7 @@ public class StressProfile implements Serializable
 case BOOLEAN:
 return new Booleans(name, config);
 case DECIMAL:
+return new BigDecimals(name, config);
 case DOUBLE:
 return new Doubles(name, config);
 case FLOAT:
@@ -523,7 +524,7 @@ public class StressProfile implements Serializable
 case LIST:
 return new Lists(name, getGenerator(name, 
type.getTypeArguments().get(0), config), config);
 default:
-throw new UnsupportedOperationException();
+throw new UnsupportedOperationException(Because of this 
name: +name+ if you removed it from the yaml and are still seeing this, make 
sure to drop table);
 }
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
new file mode 100644
index 000..42758ed
--- /dev/null
+++ 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
@@ -0,0 +1,39 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package org.apache.cassandra.stress.generate.values;
+
+import org.apache.cassandra.db.marshal.DecimalType;
+
+import java.math.BigDecimal;
+
+public class BigDecimals extends GeneratorBigDecimal
+{
+public BigDecimals(String name, GeneratorConfig config)
+{
+super(DecimalType.instance, config, name, BigDecimal.class);
+}
+
+@Override
+public BigDecimal generate()
+{
+return BigDecimal.valueOf(identityDistribution.next());
+}
+}

[05/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

2015-08-23 Thread benedict 
Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6d65404
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6d65404
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6d65404

Branch: refs/heads/cassandra-3.0
Commit: a6d65404467117c5b69b982c3576afb7e583db55
Parents: dd19a7f 5bee617
Author: Benedict Elliott Smith bened...@apache.org
Authored: Sun Aug 23 11:30:43 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:30:43 2015 +0100

--
 .../apache/cassandra/stress/StressProfile.java  |  3 +-
 .../stress/generate/values/BigDecimals.java | 39 
 2 files changed, 41 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6d65404/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--

[06/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

2015-08-23 Thread benedict 
Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6d65404
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6d65404
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6d65404

Branch: refs/heads/cassandra-2.2
Commit: a6d65404467117c5b69b982c3576afb7e583db55
Parents: dd19a7f 5bee617
Author: Benedict Elliott Smith bened...@apache.org
Authored: Sun Aug 23 11:30:43 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:30:43 2015 +0100

--
 .../apache/cassandra/stress/StressProfile.java  |  3 +-
 .../stress/generate/values/BigDecimals.java | 39 
 2 files changed, 41 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6d65404/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--

[07/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

2015-08-23 Thread benedict 
Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6d65404
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6d65404
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6d65404

Branch: refs/heads/trunk
Commit: a6d65404467117c5b69b982c3576afb7e583db55
Parents: dd19a7f 5bee617
Author: Benedict Elliott Smith bened...@apache.org
Authored: Sun Aug 23 11:30:43 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:30:43 2015 +0100

--
 .../apache/cassandra/stress/StressProfile.java  |  3 +-
 .../stress/generate/values/BigDecimals.java | 39 
 2 files changed, 41 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6d65404/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--

[10/10] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

2015-08-23 Thread benedict 
Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b1fb6b93
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b1fb6b93
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b1fb6b93

Branch: refs/heads/trunk
Commit: b1fb6b939ac892432ace5e7f5c7597bd573b669c
Parents: 7aa49ac 342e33f
Author: Benedict Elliott Smith bened...@apache.org
Authored: Sun Aug 23 11:31:03 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:31:03 2015 +0100

--
 .../apache/cassandra/stress/StressProfile.java  |  3 +-
 .../stress/generate/values/BigDecimals.java | 39 
 2 files changed, 41 insertions(+), 1 deletion(-)
--

[09/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2015-08-23 Thread benedict 
Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/342e33ff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/342e33ff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/342e33ff

Branch: refs/heads/trunk
Commit: 342e33ffb14d10c29d040fa12dbaf48d6f16e410
Parents: 6e48573 a6d6540
Author: Benedict Elliott Smith bened...@apache.org
Authored: Sun Aug 23 11:30:57 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:30:57 2015 +0100

--
 .../apache/cassandra/stress/StressProfile.java  |  3 +-
 .../stress/generate/values/BigDecimals.java | 39 
 2 files changed, 41 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/342e33ff/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--

[04/10] cassandra git commit: Support BigDecimal in cassanra-stress

2015-08-23 Thread benedict 
Support BigDecimal in cassanra-stress

patch by Sebastian Estevez; reviewed by Benedict for CASSANDRA-10048


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bee617f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bee617f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bee617f

Branch: refs/heads/trunk
Commit: 5bee617fea68e7b745c18d14b9b7076ab6fe219b
Parents: df9e798
Author: phact sdrumme...@gmail.com
Authored: Fri Aug 21 09:50:43 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:27:16 2015 +0100

--
 .../apache/cassandra/stress/StressProfile.java  |  3 +-
 .../stress/generate/values/BigDecimals.java | 39 
 2 files changed, 41 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java 
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 49c4682..192b535 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -502,6 +502,7 @@ public class StressProfile implements Serializable
 case BOOLEAN:
 return new Booleans(name, config);
 case DECIMAL:
+return new BigDecimals(name, config);
 case DOUBLE:
 return new Doubles(name, config);
 case FLOAT:
@@ -523,7 +524,7 @@ public class StressProfile implements Serializable
 case LIST:
 return new Lists(name, getGenerator(name, 
type.getTypeArguments().get(0), config), config);
 default:
-throw new UnsupportedOperationException();
+throw new UnsupportedOperationException(Because of this 
name: +name+ if you removed it from the yaml and are still seeing this, make 
sure to drop table);
 }
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
new file mode 100644
index 000..42758ed
--- /dev/null
+++ 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
@@ -0,0 +1,39 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package org.apache.cassandra.stress.generate.values;
+
+import org.apache.cassandra.db.marshal.DecimalType;
+
+import java.math.BigDecimal;
+
+public class BigDecimals extends GeneratorBigDecimal
+{
+public BigDecimals(String name, GeneratorConfig config)
+{
+super(DecimalType.instance, config, name, BigDecimal.class);
+}
+
+@Override
+public BigDecimal generate()
+{
+return BigDecimal.valueOf(identityDistribution.next());
+}
+}

[01/10] cassandra git commit: Support BigDecimal in cassanra-stress

2015-08-23 Thread benedict 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 df9e798de - 5bee617fe
  refs/heads/cassandra-2.2 dd19a7fcf - a6d654044
  refs/heads/cassandra-3.0 6e48573be - 342e33ffb
  refs/heads/trunk 7aa49acf6 - b1fb6b939


Support BigDecimal in cassanra-stress

patch by Sebastian Estevez; reviewed by Benedict for CASSANDRA-10048


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bee617f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bee617f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bee617f

Branch: refs/heads/cassandra-2.1
Commit: 5bee617fea68e7b745c18d14b9b7076ab6fe219b
Parents: df9e798
Author: phact sdrumme...@gmail.com
Authored: Fri Aug 21 09:50:43 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:27:16 2015 +0100

--
 .../apache/cassandra/stress/StressProfile.java  |  3 +-
 .../stress/generate/values/BigDecimals.java | 39 
 2 files changed, 41 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java 
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 49c4682..192b535 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -502,6 +502,7 @@ public class StressProfile implements Serializable
 case BOOLEAN:
 return new Booleans(name, config);
 case DECIMAL:
+return new BigDecimals(name, config);
 case DOUBLE:
 return new Doubles(name, config);
 case FLOAT:
@@ -523,7 +524,7 @@ public class StressProfile implements Serializable
 case LIST:
 return new Lists(name, getGenerator(name, 
type.getTypeArguments().get(0), config), config);
 default:
-throw new UnsupportedOperationException();
+throw new UnsupportedOperationException(Because of this 
name: +name+ if you removed it from the yaml and are still seeing this, make 
sure to drop table);
 }
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
new file mode 100644
index 000..42758ed
--- /dev/null
+++ 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
@@ -0,0 +1,39 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package org.apache.cassandra.stress.generate.values;
+
+import org.apache.cassandra.db.marshal.DecimalType;
+
+import java.math.BigDecimal;
+
+public class BigDecimals extends GeneratorBigDecimal
+{
+public BigDecimals(String name, GeneratorConfig config)
+{
+super(DecimalType.instance, config, name, BigDecimal.class);
+}
+
+@Override
+public BigDecimal generate()
+{
+return BigDecimal.valueOf(identityDistribution.next());
+}
+}

[02/10] cassandra git commit: Support BigDecimal in cassanra-stress

2015-08-23 Thread benedict 
Support BigDecimal in cassanra-stress

patch by Sebastian Estevez; reviewed by Benedict for CASSANDRA-10048


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bee617f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bee617f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bee617f

Branch: refs/heads/cassandra-2.2
Commit: 5bee617fea68e7b745c18d14b9b7076ab6fe219b
Parents: df9e798
Author: phact sdrumme...@gmail.com
Authored: Fri Aug 21 09:50:43 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:27:16 2015 +0100

--
 .../apache/cassandra/stress/StressProfile.java  |  3 +-
 .../stress/generate/values/BigDecimals.java | 39 
 2 files changed, 41 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java 
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 49c4682..192b535 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -502,6 +502,7 @@ public class StressProfile implements Serializable
 case BOOLEAN:
 return new Booleans(name, config);
 case DECIMAL:
+return new BigDecimals(name, config);
 case DOUBLE:
 return new Doubles(name, config);
 case FLOAT:
@@ -523,7 +524,7 @@ public class StressProfile implements Serializable
 case LIST:
 return new Lists(name, getGenerator(name, 
type.getTypeArguments().get(0), config), config);
 default:
-throw new UnsupportedOperationException();
+throw new UnsupportedOperationException(Because of this 
name: +name+ if you removed it from the yaml and are still seeing this, make 
sure to drop table);
 }
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
--
diff --git 
a/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
new file mode 100644
index 000..42758ed
--- /dev/null
+++ 
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
@@ -0,0 +1,39 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package org.apache.cassandra.stress.generate.values;
+
+import org.apache.cassandra.db.marshal.DecimalType;
+
+import java.math.BigDecimal;
+
+public class BigDecimals extends GeneratorBigDecimal
+{
+public BigDecimals(String name, GeneratorConfig config)
+{
+super(DecimalType.instance, config, name, BigDecimal.class);
+}
+
+@Override
+public BigDecimal generate()
+{
+return BigDecimal.valueOf(identityDistribution.next());
+}
+}

[08/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

2015-08-23 Thread benedict 
Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/342e33ff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/342e33ff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/342e33ff

Branch: refs/heads/cassandra-3.0
Commit: 342e33ffb14d10c29d040fa12dbaf48d6f16e410
Parents: 6e48573 a6d6540
Author: Benedict Elliott Smith bened...@apache.org
Authored: Sun Aug 23 11:30:57 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Sun Aug 23 11:30:57 2015 +0100

--
 .../apache/cassandra/stress/StressProfile.java  |  3 +-
 .../stress/generate/values/BigDecimals.java | 39 
 2 files changed, 41 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/342e33ff/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--

[jira] [Commented] (CASSANDRA-9459) SecondaryIndex API redesign

2015-08-23 Thread Benedict (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708330#comment-14708330
 ] 

Benedict commented on CASSANDRA-9459:
-

A nit question relating to this ticket: In each of the main Transaction 
implementations we seem to be making unnecessary use of streams. These are a 
hot code paths on systems using indexes, and I'm not convinced we should be 
incurring the GC burden, especially when clarity isn't meaningfully improved. 
Compare

{code}
try (OpOrder.Group opGroup = Keyspace.writeOrder.start())
{
Index.Indexer[] indexers = Arrays.stream(indexes)
 .map(i - i.indexerFor(key, 
nowInSec, opGroup, Type.CLEANUP))
 .toArray(Index.Indexer[]::new);

Arrays.stream(indexers).forEach(Index.Indexer::begin);

if (partitionDelete != null)
Arrays.stream(indexers).forEach(indexer - 
indexer.partitionDelete(partitionDelete));

if (row != null)
Arrays.stream(indexers).forEach(indexer - 
indexer.removeRow(row));

Arrays.stream(indexers).forEach(Index.Indexer::finish);
}
{code}

with

{code}
try (OpOrder.Group opGroup = Keyspace.writeOrder.start())
{
for (Index index : indexes)
{
Index.Indexer indexer = index.indexerFor(key, nowInSec, 
opGroup, Type.CLEANUP);
indexer.begin();
if (partitionDelete != null)
indexer.partitionDelete(partitionDelete);
if (row != null)
indexer.removeRow(row);
indexer.finish();
}
}
{code} 

I'm pretty convinced the latter is clearer, and in the former I count at least 
8 unnecessary allocations for the first stream, and 4 for each of the rest. A 
few of these allocations are = 64Kb, and I estimate total allocation per row 
on the order of (but probably a little less than) 1Kb. Conversely, the 
old-style code performs no allocations besides that of the {{Indexer}}.

 SecondaryIndex API redesign
 ---

 Key: CASSANDRA-9459
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9459
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
 Fix For: 3.0 beta 1


 For some time now the index subsystem has been a pain point and in large part 
 this is due to the way that the APIs and principal classes have grown 
 organically over the years. It would be a good idea to conduct a wholesale 
 review of the area and see if we can come up with something a bit more 
 coherent.
 A few starting points:
 * There's a lot in AbstractPerColumnSecondaryIndex  its subclasses which 
 could be pulled up into SecondaryIndexSearcher (note that to an extent, this 
 is done in CASSANDRA-8099).
 * SecondayIndexManager is overly complex and several of its functions should 
 be simplified/re-examined. The handling of which columns are indexed and 
 index selection on both the read and write paths are somewhat dense and 
 unintuitive.
 * The SecondaryIndex class hierarchy is rather convoluted and could use some 
 serious rework.
 There are a number of outstanding tickets which we should be able to roll 
 into this higher level one as subtasks (but I'll defer doing that until 
 getting into the details of the redesign):
 * CASSANDRA-7771
 * CASSANDRA-8103
 * CASSANDRA-9041
 * CASSANDRA-4458
 * CASSANDRA-8505
 Whilst they're not hard dependencies, I propose that this be done on top of 
 both CASSANDRA-8099 and CASSANDRA-6717. The former largely because the 
 storage engine changes may facilitate a friendlier index API, but also 
 because of the changes to SIS mentioned above. As for 6717, the changes to 
 schema tables there will help facilitate CASSANDRA-7771.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10151) Fix TrackerTest.testTryModify

2015-08-23 Thread Benedict (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708349#comment-14708349
 ] 

Benedict commented on CASSANDRA-10151:
--

Thanks. Committed as 941a5dd8a1ca9b759e2091ca151bb339bc947c58

 Fix TrackerTest.testTryModify
 -

 Key: CASSANDRA-10151
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10151
 Project: Cassandra
  Issue Type: Test
Reporter: Stefania
Assignee: Benedict
 Fix For: 2.2.1, 3.0 beta 2


 TrackerTest.testTryModify has started failing on trunk and 3.0:
 http://cassci.datastax.com/job/cassandra-3.0_testall/53/testReport/org.apache.cassandra.db.lifecycle/TrackerTest/testTryModify/
 {code}
 junit.framework.AssertionFailedError
   at 
 org.apache.cassandra.utils.concurrent.Ref$State.assertNotReleased(Ref.java:165)
   at org.apache.cassandra.utils.concurrent.Ref.ref(Ref.java:121)
   at 
 org.apache.cassandra.io.sstable.format.SSTableReader$GlobalTidy.get(SSTableReader.java:2251)
   at 
 org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.setup(SSTableReader.java:2099)
   at 
 org.apache.cassandra.io.sstable.format.SSTableReader.setup(SSTableReader.java:2040)
   at 
 org.apache.cassandra.io.sstable.format.SSTableReader.internalOpen(SSTableReader.java:575)
   at org.apache.cassandra.MockSchema.sstable(MockSchema.java:124)
   at org.apache.cassandra.MockSchema.sstable(MockSchema.java:79)
   at org.apache.cassandra.MockSchema.sstable(MockSchema.java:74)
   at 
 org.apache.cassandra.db.lifecycle.TrackerTest.testTryModify(TrackerTest.java:86)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10155) 2i key cache load fails

2015-08-23 Thread Robert Stupp (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708338#comment-14708338
 ] 

Robert Stupp commented on CASSANDRA-10155:
--

Right, cfName is different for base table and 2i.
But the only parameter to 
{{AutoSavingCache#getCacheDataPath}}/{{getCacheCrcPath}} is cfId - so 
{{DatabaseDescriptor.getSerializedCachePath}} can only return the file name for 
the base table. I.e. a {{AutoSavingCache#loadSaved}} will access the same file 
for base table and 2i {{ColumnFamilyStore}}s.

 2i key cache load fails
 ---

 Key: CASSANDRA-10155
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10155
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
 Fix For: 3.0.0 rc1


 CASSANDRA-9265 changed how key cache content is serialized to disk. It uses 
 {{UUID cfId}} to generate the file path for each {{ColumnFamilyStore}}.
 Since {{cfId}} of a secondary index is the same as for the base table, the 
 key-cache files for 2i's and the base are the same. This will/may lead to 
 deserialization failures on restart for tables with at least one 2i.
 /cc [~aweisberg] [~danchia]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10155) 2i key cache load fails

2015-08-23 Thread Robert Stupp (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708338#comment-14708338
 ] 

Robert Stupp edited comment on CASSANDRA-10155 at 8/23/15 10:19 AM:


Right, cfName is different for base table and 2i.
But the only parameter to 
{{AutoSavingCache#getCacheDataPath}}/{{getCacheCrcPath}} is cfId - so 
{{DatabaseDescriptor.getSerializedCachePath}} can only return the file name for 
the base table. I.e. a {{AutoSavingCache#loadSaved}} will access the same file 
for base table and 2i {{ColumnFamilyStore}}s.

EDIT: I'm in favor of making cfId (or some other identifier) unique 
(CASSANDRA-10028) since there's a lot of special casing for 2i names.


was (Author: snazy):
Right, cfName is different for base table and 2i.
But the only parameter to 
{{AutoSavingCache#getCacheDataPath}}/{{getCacheCrcPath}} is cfId - so 
{{DatabaseDescriptor.getSerializedCachePath}} can only return the file name for 
the base table. I.e. a {{AutoSavingCache#loadSaved}} will access the same file 
for base table and 2i {{ColumnFamilyStore}}s.

 2i key cache load fails
 ---

 Key: CASSANDRA-10155
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10155
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
 Fix For: 3.0.0 rc1


 CASSANDRA-9265 changed how key cache content is serialized to disk. It uses 
 {{UUID cfId}} to generate the file path for each {{ColumnFamilyStore}}.
 Since {{cfId}} of a secondary index is the same as for the base table, the 
 key-cache files for 2i's and the base are the same. This will/may lead to 
 deserialization failures on restart for tables with at least one 2i.
 /cc [~aweisberg] [~danchia]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10155) 2i key cache load fails

2015-08-23 Thread Daniel Chia (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708617#comment-14708617
 ] 

Daniel Chia commented on CASSANDRA-10155:
-

Ah good call. That was actually introduced in CASSANDRA-7784, I'll investigate 
further.

 2i key cache load fails
 ---

 Key: CASSANDRA-10155
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10155
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
 Fix For: 3.0.0 rc1


 CASSANDRA-9265 changed how key cache content is serialized to disk. It uses 
 {{UUID cfId}} to generate the file path for each {{ColumnFamilyStore}}.
 Since {{cfId}} of a secondary index is the same as for the base table, the 
 key-cache files for 2i's and the base are the same. This will/may lead to 
 deserialization failures on restart for tables with at least one 2i.
 /cc [~aweisberg] [~danchia]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10155) 2i key cache load fails

2015-08-23 Thread Daniel Chia (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708635#comment-14708635
 ] 

Daniel Chia commented on CASSANDRA-10155:
-

It looks like switching to use cfId exclusively in CASSANDRA-7784 introduced 
this regression, so it will be found 2.1 as well.

Looks like the right fix is to re-introduce pathInfo - what do people think?

 2i key cache load fails
 ---

 Key: CASSANDRA-10155
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10155
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
 Fix For: 3.0.0 rc1


 CASSANDRA-9265 changed how key cache content is serialized to disk. It uses 
 {{UUID cfId}} to generate the file path for each {{ColumnFamilyStore}}.
 Since {{cfId}} of a secondary index is the same as for the base table, the 
 key-cache files for 2i's and the base are the same. This will/may lead to 
 deserialization failures on restart for tables with at least one 2i.
 /cc [~aweisberg] [~danchia]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10035) Windows utest 3.0: TransactionLogsTest failure

2015-08-23 Thread Stefania (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708767#comment-14708767
 ] 

Stefania commented on CASSANDRA-10035:
--

Since CASSANDRA-7066 was committed, the tests have been failing less but we 
still have problems. The first utest build on 3.0 with the patch committed was 
build #15. Since there:

* TransactionLogTest.testObsoletedDataFileUpdateTimeChanged failed on build 15
* TransactionLogTest.testWrongChecksumLastLineMissingFile failed on build 19
* TransactionLogTest.testRemoveUnfinishedLeftovers_abort and 
testRemoveUnfinishedLeftovers_commit failed on build 22

The failure in build 15 is odd, it seems we did not pick up the modification of 
the last updated time, we probably just need to release the sstables earlier.

The remaining failures are all related to listing temporary files, so I suspect 
the root cause is CASSANDRA-10109.

 Windows utest 3.0: TransactionLogsTest failure
 --

 Key: CASSANDRA-10035
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10035
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Joshua McKenzie
Assignee: Stefania
  Labels: Windows
 Fix For: 3.0.0 rc1


 {noformat}
 [junit] Testcase: 
 testUntrack(org.apache.cassandra.db.lifecycle.TransactionLogsTest):   
 FAILED
 [junit] 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf4-6742d2103f8911e58737831ef068609c\ma-1-big-Data.db
 [junit] junit.framework.AssertionFailedError: 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf4-6742d2103f8911e58737831ef068609c\ma-1-big-Data.db
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.testUntrack(TransactionLogsTest.java:215)
 [junit]
 [junit]
 [junit] Testcase: 
 testAbortOnlyNew(org.apache.cassandra.db.lifecycle.TransactionLogsTest):  
 FAILED
 [junit] 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf5-67456a203f8911e58737831ef068609c\ma-0-big-Data.db
 [junit] junit.framework.AssertionFailedError: 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf5-67456a203f8911e58737831ef068609c\ma-0-big-Data.db
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.testAbortOnlyNew(TransactionLogsTest.java:312)
 [junit]
 [junit]
 [junit] Testcase: 
 testNoPrepare(org.apache.cassandra.db.lifecycle.TransactionLogsTest): 
 FAILED
 [junit] 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf13-675681203f8911e58737831ef068609c\ma-1-big-Data.db
 [junit] junit.framework.AssertionFailedError: 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf13-675681203f8911e58737831ef068609c\ma-1-big-Data.db
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.access$200(TransactionLogsTest.java:56)
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest$TxnTest$Transaction.assertAborted(TransactionLogsTest.java:143)
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest$TxnTest.assertAborted(TransactionLogsTest.java:191)
 [junit] at 
 org.apache.cassandra.utils.concurrent.AbstractTransactionalTest.testNoPrepare(AbstractTransactionalTest.java:40)
 {noformat}
 taking the testPrepare case as an example: looks like it never passed on 
 Windows 
 ([history|http://cassci.datastax.com/view/trunk/job/trunk_utest_win32/lastCompletedBuild/testReport/org.apache.cassandra.db.lifecycle/TransactionLogsTest/testPrepare/history/])
 [~stefania_alborghetti]: annotate points to you on these tests. Care to take 
 a look at this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10035) Windows utest 3.0: TransactionLogsTest failure

2015-08-23 Thread Stefania (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708767#comment-14708767
 ] 

Stefania edited comment on CASSANDRA-10035 at 8/24/15 4:41 AM:
---

Since CASSANDRA-7066 was committed, the tests have been failing less but we 
still have problems. The first utest build on 3.0 with the patch committed was 
build #15. Since then:

* TransactionLogTest.testObsoletedDataFileUpdateTimeChanged failed on build 15
* TransactionLogTest.testWrongChecksumLastLineMissingFile failed on build 19
* TransactionLogTest.testRemoveUnfinishedLeftovers_abort and 
testRemoveUnfinishedLeftovers_commit failed on build 22

The failure in build 15 is odd, it seems we did not pick up the modification of 
the last updated time, we probably just need to release the sstables earlier.

The remaining failures are all related to listing temporary files, so I suspect 
the root cause is CASSANDRA-10109.


was (Author: stefania):
Since CASSANDRA-7066 was committed, the tests have been failing less but we 
still have problems. The first utest build on 3.0 with the patch committed was 
build #15. Since there:

* TransactionLogTest.testObsoletedDataFileUpdateTimeChanged failed on build 15
* TransactionLogTest.testWrongChecksumLastLineMissingFile failed on build 19
* TransactionLogTest.testRemoveUnfinishedLeftovers_abort and 
testRemoveUnfinishedLeftovers_commit failed on build 22

The failure in build 15 is odd, it seems we did not pick up the modification of 
the last updated time, we probably just need to release the sstables earlier.

The remaining failures are all related to listing temporary files, so I suspect 
the root cause is CASSANDRA-10109.

 Windows utest 3.0: TransactionLogsTest failure
 --

 Key: CASSANDRA-10035
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10035
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Joshua McKenzie
Assignee: Stefania
  Labels: Windows
 Fix For: 3.0.0 rc1


 {noformat}
 [junit] Testcase: 
 testUntrack(org.apache.cassandra.db.lifecycle.TransactionLogsTest):   
 FAILED
 [junit] 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf4-6742d2103f8911e58737831ef068609c\ma-1-big-Data.db
 [junit] junit.framework.AssertionFailedError: 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf4-6742d2103f8911e58737831ef068609c\ma-1-big-Data.db
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.testUntrack(TransactionLogsTest.java:215)
 [junit]
 [junit]
 [junit] Testcase: 
 testAbortOnlyNew(org.apache.cassandra.db.lifecycle.TransactionLogsTest):  
 FAILED
 [junit] 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf5-67456a203f8911e58737831ef068609c\ma-0-big-Data.db
 [junit] junit.framework.AssertionFailedError: 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf5-67456a203f8911e58737831ef068609c\ma-0-big-Data.db
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.testAbortOnlyNew(TransactionLogsTest.java:312)
 [junit]
 [junit]
 [junit] Testcase: 
 testNoPrepare(org.apache.cassandra.db.lifecycle.TransactionLogsTest): 
 FAILED
 [junit] 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf13-675681203f8911e58737831ef068609c\ma-1-big-Data.db
 [junit] junit.framework.AssertionFailedError: 
 build\test\cassandra\data;0\TransactionLogsTest\mockcf13-675681203f8911e58737831ef068609c\ma-1-big-Data.db
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest.access$200(TransactionLogsTest.java:56)
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest$TxnTest$Transaction.assertAborted(TransactionLogsTest.java:143)
 [junit] at 
 org.apache.cassandra.db.lifecycle.TransactionLogsTest$TxnTest.assertAborted(TransactionLogsTest.java:191)
 [junit] at 
 org.apache.cassandra.utils.concurrent.AbstractTransactionalTest.testNoPrepare(AbstractTransactionalTest.java:40)
 {noformat}
 taking the testPrepare case as an example: looks like it never passed on 
 Windows 
 ([history|http://cassci.datastax.com/view/trunk/job/trunk_utest_win32/lastCompletedBuild/testReport/org.apache.cassandra.db.lifecycle/TransactionLogsTest/testPrepare/history/])
 [~stefania_alborghetti]: annotate points to you on these tests. Care to take 
 a look at this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10086) Add a CLEAR cqlsh command to clear the console

2015-08-23 Thread Stefania (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708745#comment-14708745
 ] 

Stefania commented on CASSANDRA-10086:
--

Thanks for the patch, +1. [~jbellis] could you commit on 3.0?

The tests for cqlsh are located in _cqlsh_tests/cqlsh_tests.py_. We are missing 
some commands but we've started adding tests whenever we make any changes to 
existing commands or add new ones. A good example would be the DESCRIBE 
command, for which you find some tests in this file. 

The test is actually quite simple, we just run cqlsh and check that there is no 
error in stderr and that stdout contains one of the clear screen ANSI escape 
sequences. Have a look at the top commit 
[here|https://github.com/stef1927/cassandra-dtest/commits/10086].

It works on Linux. Are you by any chance running Windows or Mac OS, to see if 
it works there too? If not, don't worry about it, just review the test and 
confirm you are happy with it.
 

 Add a CLEAR cqlsh command to clear the console
 

 Key: CASSANDRA-10086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10086
 Project: Cassandra
  Issue Type: Improvement
Reporter: Paul O'Fallon
Priority: Trivial
  Labels: cqlsh, doc-impacting
 Attachments: 10086.txt, 10086v2.txt


 It would be very helpful to have a CLEAR command to clear the cqlsh 
 console.  I learned (after researching a patch for this) that lowercase 
 CTRL+L will clear the screen, but having a discrete command would make that 
 more obvious.  To match the expectations of Windows users, an alias to CLS 
 would be nice as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10109) Windows dtest 3.0: ttl_test.py failures

2015-08-23 Thread Stefania (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-10109:
-
Fix Version/s: (was: 3.0.x)
   3.0.0 rc1

 Windows dtest 3.0: ttl_test.py failures
 ---

 Key: CASSANDRA-10109
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10109
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Joshua McKenzie
Assignee: Stefania
  Labels: Windows
 Fix For: 3.0.0 rc1


 ttl_test.py:TestTTL.update_column_ttl_with_default_ttl_test2
 ttl_test.py:TestTTL.update_multiple_columns_ttl_test
 ttl_test.py:TestTTL.update_single_column_ttl_test
 Errors locally are different than CI from yesterday. Yesterday on CI we have 
 timeouts and general node hangs. Today on all 3 tests when run locally I see:
 {noformat}
 Traceback (most recent call last):
   File c:\src\cassandra-dtest\dtest.py, line 532, in tearDown
 raise AssertionError('Unexpected error in %s node log: %s' % (node.name, 
 errors))
 AssertionError: Unexpected error in node1 node log: ['ERROR [main] 2015-08-17 
 16:53:43,120 NoSpamLogger.java:97 - This platform does not support atomic 
 directory streams (SecureDirectoryStream); race conditions when loading 
 sstable files could occurr']
 {noformat}
 This traces back to the commit for CASSANDRA-7066 today by [~Stefania] and 
 [~benedict].  Stefania - care to take this ticket and also look further into 
 whether or not we're going to have issues with 7066 on Windows? That error 
 message certainly *sounds* like it's not a good thing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)

2015-08-23 Thread Stefania (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708687#comment-14708687
 ] 

Stefania commented on CASSANDRA-8630:
-

I've fixed {{ChecksummedDataInput}} so that the slow path in 
{{RebufferingInputStream}} is no longer required.

I've added a comment regarding copying of {{MmappedRegions}} and thread safety 
and filed a follow-up ticket: CASSANDRA-10158.

[~aweisberg], ready for another round of review.


 Faster sequential IO (on compaction, streaming, etc)
 

 Key: CASSANDRA-8630
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8630
 Project: Cassandra
  Issue Type: Improvement
  Components: Core, Tools
Reporter: Oleg Anastasyev
Assignee: Stefania
  Labels: compaction, performance
 Fix For: 3.x

 Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, 
 flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz, 
 mmaped_uncomp_hotspot.png


 When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot 
 of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int).
 This is because default implementations of readShort,readLong, etc as well as 
 their matching write* are implemented with numerous calls of byte by byte 
 read and write. 
 This makes a lot of syscalls as well.
 A quick microbench shows than just reimplementation of these methods in 
 either way gives 8x speed increase.
 A patch attached implements RandomAccessReader.readType and 
 SequencialWriter.writeType methods in more efficient way.
 I also eliminated some extra byte copies in CompositeType.split and 
 ColumnNameHelper.maxComponents, which were on my profiler's hotspot method 
 list during tests.
 A stress tests on my laptop show that this patch makes compaction 25-30% 
 faster  on uncompressed sstables and 15% faster for compressed ones.
 A deployment to production shows much less CPU load for compaction. 
 (I attached a cpu load graph from one of our production, orange is niced CPU 
 load - i.e. compaction; yellow is user - i.e. not compaction related tasks)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10158) Rationalize implementations of DataInputPlus and DataOutputPlus

2015-08-23 Thread Stefania (JIRA) 
Stefania created CASSANDRA-10158:


 Summary: Rationalize implementations of DataInputPlus and 
DataOutputPlus
 Key: CASSANDRA-10158
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10158
 Project: Cassandra
  Issue Type: Improvement
Reporter: Stefania
Assignee: Stefania


Following CASSANDRA-8630, we can further improve implementations of 
{{DataInputPlus}} and {{DataOutputPlus}} as follows:

* In {{MmappedRegions}}, compact the mmap ranges, at least on the final opening 
of the file

* Use the mmap extension logic for compressed files

* Consider renaming classes to more appropriate names, moving them in their own 
package and making their relations more simply defined.

* Consider unifying {{ChecksummedDataInput}} and 
{{ChecksummedRandomAccessReader}}.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)